Roberto Dip
fa89dff66f
The scheduled test run https://github.com/fleetdm/fleet/actions/runs/7764392848 failed with a panic because `TestWindowsMDMEnrollmentPrevented` timed out: ``` 2024-02-03T05:05:26.3041218Z === RUN TestWindowsMDMEnrollmentPrevented 2024-02-03T05:05:26.3044251Z === RUN TestWindowsMDMEnrollmentPrevented/{RenewEnrollmentProfile:false_RotateDiskEncryptionKey:false_NeedsMDMMigration:false_NeedsProgrammaticWindowsMDMEnrollment:true_WindowsMDMDiscoveryEndpoint:http://example.com/_NeedsProgrammaticWindowsMDMUnenrollment:false_PendingScriptExecutionIDs:[]_EnforceBitLockerEncryption:false} 2024-02-03T05:05:26.3047208Z coverage: 2.5% of statements in github.com/fleetdm/fleet/v4/... 2024-02-03T05:05:26.3047963Z panic: test timed out after 1h0m0s 2024-02-03T05:05:26.3048482Z running tests: 2024-02-03T05:05:26.3049005Z TestWindowsMDMEnrollmentPrevented (59m52s) 2024-02-03T05:05:26.3052172Z TestWindowsMDMEnrollmentPrevented/{RenewEnrollmentProfile:false_RotateDiskEncryptionKey:false_NeedsMDMMigration:false_NeedsProgrammaticWindowsMDMEnrollment:true_WindowsMDMDiscoveryEndpoint:http://example.com/_NeedsProgrammaticWindowsMDMUnenrollment:false_PendingScriptExecutionIDs:[]_EnforceBitLockerEncryption:false} (59m52s) [...] 2024-02-03T05:05:26.3068624Z goroutine 69 [chan receive]: 2024-02-03T05:05:26.3069997Z github.com/fleetdm/fleet/v4/orbit/pkg/update.TestWindowsMDMEnrollmentPrevented.func2.1({{0xe3ada3, 0x12}, {0x0, 0x0}, {0xe37311, 0xc}}) 2024-02-03T05:05:26.3072376Z /home/runner/work/fleet/fleet/orbit/pkg/update/notifications_test.go:295 +0x65 2024-02-03T05:05:26.3074514Z github.com/fleetdm/fleet/v4/orbit/pkg/update.(*windowsMDMEnrollmentConfigFetcher).attemptEnrollment(0xc0000f8cf0, {0x0, 0x0, 0x0, 0x1, {0xe3ada3, 0x12}, 0x0, {0x0, 0x0, ...}, ...}) ``` I was able to reproduce locally 1/4th of the times, after putting the following print statements: ```diff if cfg.NeedsProgrammaticWindowsMDMEnrollment { fetcher.execEnrollFn = func(args WindowsMDMEnrollmentArgs) error { - <-chProceed // will be unblocked only when allowed + fmt.Println("fetcher.execEnrollFn A: ", apiCallCount) + <-chProceed // will be unblocked only when allowed + fmt.Println("fetcher.execEnrollFn B: ", apiCallCount) apiCallCount++ // no need for sync, single-threaded call of this func is guaranteed by the fetcher's mutex return apiErr } @@ -301,7 +303,9 @@ func TestWindowsMDMEnrollmentPrevented(t *testing.T) { } } else { fetcher.execUnenrollFn = func(args WindowsMDMEnrollmentArgs) error { - <-chProceed // will be unblocked only when allowed + fmt.Println("fetcher.execUnenrollFn A: ", apiCallCount) + <-chProceed // will be unblocked only when allowed + fmt.Println("fetcher.execUnenrollFn B: ", apiCallCount) apiCallCount++ // no need for sync, single-threaded call of this func is guaranteed by the fetcher's mutex return apiErr } @@ -317,23 +321,33 @@ func TestWindowsMDMEnrollmentPrevented(t *testing.T) { started := make(chan struct{}) go func() { + fmt.Println("before close started") close(started) + fmt.Println("aftre close started") // the first call will block in enroll/unenroll func + fmt.Println("before inner fetchergetconfig") cfg, err := fetcher.GetConfig() + fmt.Println("after inner fetchergetconfig") assertResult(cfg, err) }() + fmt.Println("before started") <-started + fmt.Println("after started") // this call will happen while the first call is blocked in // enroll/unenrollfn, so it won't call the API (won't be able to lock the // mutex). However it will still complete successfully without being // blocked by the other call in progress. + fmt.Println("before first fetchergetconfig") cfg, err := fetcher.GetConfig() + fmt.Println("before first fetchergetconfig") assertResult(cfg, err) // unblock the first call and wait for it to complete + fmt.Println("before close chProceed 1") close(chProceed) + fmt.Println("after close chProceed 2") time.Sleep(100 * time.Millisecond) ``` This is the output I've got every time the test hung: ``` before started before close started aftre close started after started before first fetchergetconfig before inner fetchergetconfig after inner fetchergetconfig fetcher.execEnrollFn A: 0 ``` And this is the output when the tests passed ``` before started before close started aftre close started before inner fetchergetconfig fetcher.execUnenrollFn A: 0 after started before first fetchergetconfig before first fetchergetconfig before close chProceed 1 after close chProceed 2 fetcher.execUnenrollFn B: 0 after inner fetchergetconfig fetcher.execUnenrollFn A: 1 fetcher.execUnenrollFn B: 1 ``` Note how the deadlock occurs when `GetConfig` is called first outside of the goroutine. I added some logic to prevent this, but I'm confident there must be a better way to accomplish the same. cc: @mna you're the king of concurrency, do you have any ideas?
Orbit is a lightweight osquery installer and autoupdater. With Orbit, it's easy to deploy osquery, manage configurations, and keep things up-to-date. Orbit eases the deployment of osquery connected with a Fleet server, and is a (near) drop-in replacement for osquery in a variety of deployment scenarios.
Orbit is the recommended agent for Fleet. But Orbit can be used with or without Fleet, and Fleet can be used with or without Orbit.
Documentation
Bugs
To report a bug or request a feature, click here.
Orbit Development
Run Orbit From Source
To execute orbit from source directly, run the following command:
go run github.com/fleetdm/fleet/v4/orbit/cmd/orbit \
--dev-mode \
--disable-updates \
--root-dir /tmp/orbit \
--fleet-url https://localhost:8080 \
--insecure \
--enroll-secret Pz3zC0NMDdZfb3FtqiLgwoexItojrYh/ \
-- --verbose
Or, using a flagfile.txt for osqueryd:
go run github.com/fleetdm/fleet/v4/orbit/cmd/orbit \
--dev-mode \
--disable-updates \
--root-dir /tmp/orbit \
-- --flagfile=flagfile.txt --verbose
Generate Installer Packages from Orbit Source
The fleetctl package command generates installers by fetching the targets/executables from a TUF repository.
To generate an installer that contains an Orbit built from source you need to setup a local TUF repository.
The following document explains how you can generate a TUF repository, and installers that use it tools/tuf/test.
FAQs
How does Orbit compare with Kolide Launcher?
Orbit is inspired by the success of Kolide Launcher, and approaches a similar problem domain with new strategies informed by the challenges encountered in real world deployments. Orbit does not share any code with Launcher.
- Both Orbit and Launcher use The Update Framework specification for managing updates. Orbit utilizes the official go-tuf library, while Launcher has it's own implementation of the specification.
- Orbit can be deployed as a (near) drop-in replacement for osquery, supporting full customization of the osquery flags. Launcher heavily manages the osquery flags making deployment outside of Fleet or Kolide's SaaS difficult.
- Orbit prefers the battle-tested plugins of osquery. Orbit uses the built-in logging, configuration, and live query plugins, while Launcher uses custom implementations.
- Orbit prefers the built-in osquery remote APIs. Launcher utilizes a custom gRPC API that has led to issues with character encoding, load balancers/proxies, and request size limits.
- Orbit encourages use of the osquery performance Watchdog, while Launcher disables the Watchdog.
Additionally, Orbit aims to tackle problems out of scope for Launcher:
- Configure updates via release channels, providing more granular control over agent versioning.
- Manage osquery startup flags from a remote (Fleet) server.
- Support for deploying and updating osquery extensions (🔜).
- Manage osquery versions from a remote (Fleet) server (🔜).
Is Orbit Free?
Yes! Orbit is licensed under an MIT license and all uses are encouraged.
How does orbit update osquery? And how do the stable and edge channels get triggered to update osquery on a self hosted Fleet instance?
Orbit uses a configurable update server. We expect that many folks will just use the update server we manage (similar to what Kolide does with Launcher's update server). We are also offering tooling for self-managing an update server as part of Fleet Premium (the subscription offering).
Community
Chat
Please join us in the #fleet channel on osquery Slack.
