**Related issue:** Resolves#41910
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configurable env var to limit Android MDM profile
reconciliation batch size (FLEET_MDM_ANDROID_PROFILES_BATCH_SIZE;
default 1000).
* Reconciliation now processes hosts in cursor-based, batched windows
and persists a reconciliation cursor to resume/advance work, reducing
peak API load and enabling pagination.
* **Tests**
* Added validation tests for the batch-size config and tests verifying
cursor-based pagination and processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#46338
Changes load tested (osquery load test).
<img width="1532" height="282" alt="image"
src="https://github.com/user-attachments/assets/9eb38629-918e-4e05-bc0b-e2ae22e8b148"
/>
<img width="1414" height="937" alt="image"
src="https://github.com/user-attachments/assets/1b90afd3-087c-452e-939f-b495df3e59c8"
/>
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Extended host cache retention period from 60 seconds to 180 seconds to
improve overall cache efficiency and hit rates.
* Optimized host cache invalidation strategy to eliminate unnecessary
reverse-index lookups, significantly reducing database reader load and
Redis CPU consumption.
* **Tests**
* Added test coverage for host update cache invalidation edge cases to
ensure proper behavior.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/46458?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Closes#46153
This PR is big, but I found it worth it to include in the same PR to
keep the mental change context in one place.
This PR moves away from our previous version of a big SQL computing the
desired state and label membership with big union branches. It does so
by switching the model up completely, first:
- We batch read hosts (current hardcoded is 5k), and we always iterate
5k hosts and then decide if they have changes, so that means a tick
(30s) could read 5k hosts that DOES NOT require changes, but that is
computed in code after, rather than relying on a big SQL to do it
(twice).
- We then for those hosts, bulk fetch label memberships, their related
team profiles and current rows. This performs much better as we can
lookup everything we need by primary key or super fast indexed columns,
simple fetch all these calls.
- Then once gathered the information we move to the code to determine if
the operation is install, remove, NO-OP (Desired state calculation),
then we check the label membership to further determine it's final
action.
- We then move to what we did before, which is queue the correct command
etc.
It comes with some slight caveats, which is we now load a lot more data
into memory (but before we could spike worse), so when loadtesting we
watched CPU/Memory utilization, which never seemed to spike as the
datasets are kept as small as possible.
_Cleanup will come in a follow-up PR where we remove all the old code._
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance**
* Optimized Apple profile and DDM (Declarations) reconciliation engine
with batched processing for significantly improved performance in
environments with large numbers of Apple-enrolled hosts.
* Implemented cursor-based pagination for more efficient reconciliation
across large fleets.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45573?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude <noreply@anthropic.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43928
This PR adds a Redis-backed cache in front of the two host-by-key
lookups on the agent auth paths.
Docs: https://github.com/fleetdm/fleet/pull/44504
## What changes
**Read path (osquery/orbit auth):**
- `LoadHostByNodeKey` and `LoadHostByOrbitNodeKey` now check Redis
before falling through to MySQL.
- Successful lookups are cached for 60s ± 10% jitter (configurable via
`FLEET_REDIS_HOST_CACHE_TTL`).
- `NotFound` results are cached for 5s as a negative entry, dampening
repeated probes for keys that
do not exist (deleted hosts whose agents are still polling, attacker
scans, retry storms).
- Concurrent lookups for the same key collapse into one DB query via
`singleflight`. The shared
query runs under a context detached from any one caller's deadline so
the leader giving up does
not abort the work for joiners. The shared query is itself bounded by a
30s timeout so a wedged
DB call cannot pin the singleflight slot indefinitely.
**Write path (invalidations):**
- These methods now invalidate the cache after a successful inner call:
`UpdateHost`, `SerialUpdateHost`, `UpdateHostOsqueryIntervals`,
`UpdateHostRefetchRequested`,
`UpdateHostRefetchCriticalQueriesUntil`,
`UpdateHostIdentityCertHostIDBySerial`, `EnrollOsquery`,
`EnrollOrbit`, `NewHost`, `DeleteHost`, `DeleteHosts`,
`CleanupExpiredHosts`,
`CleanupIncomingHosts`, `AddHostsToTeam`.
- `AddHostsToTeam`, `DeleteHosts`, `CleanupExpiredHosts`, and
`CleanupIncomingHosts` use a pipelined
batch invalidator so 10k-host operations stay in the millisecond range
instead of taking minutes
of sequential round-trips.
- Inner-call errors are not invalidations: a failing write leaves cached
state intact.
**Configuration:**
- New flags `FLEET_REDIS_HOST_CACHE_ENABLED` (default `true`) and
`FLEET_REDIS_HOST_CACHE_TTL`
(default `60s`).
- Server refuses to start if the cache is enabled with `TTL <= 0`.
**Observability:**
- Three new OTEL counters under the `fleet` meter:
- `fleet.host_cache.lookups{result=hit|negative_hit|miss}`
- `fleet.host_cache.errors{op=get|set|del}`
-
`fleet.host_cache.invalidations{reason=update|enroll|team|delete|cert}`
- A pre-built SigNoz dashboard ships in
`tools/signoz/host_cache_dashboard.json`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional Redis-backed host lookup cache for osquery and orbit auth,
with automatic invalidation and metrics/monitoring dashboard.
* **Bug Fixes**
* Fixed host-removal batching so cache-related removals use correct
chunks.
* **Tests**
* Added comprehensive host-cache unit tests covering hits, negative
cache, invalidation, concurrency, and JSON round-trips.
* **Chores**
* New config flags to enable the cache and set TTL (default 60s ±10%
jitter).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44052
Improve performance by reducing the time for the synchronous API call to
update profiles or switch teams. And spreading out the application of
profiles by processing 2000 hosts every 30 seconds.
1. **Windows profile reconciliation is no longer synchronous to
bulk-set.**
Apple, Android, and Apple-declaration paths still write their pending
state inside the bulk-set transaction. The Windows path commits the
transactional inputs and lets the existing `mdm_windows_profile_manager`
cron pick the work up on its next tick. The visible effect is that
`host_mdm_windows_profiles` is no longer guaranteed to be populated by
the time bulk-set returns; it converges within one cron interval.
2. **The Windows reconciler now processes hosts in bounded batches, with
a persisted cursor.**
Previous behavior was "scan the universe of pending Windows hosts on
every tick." New behavior is a host-window query bounded by batch size
and a `host_uuid` cursor, advanced after the batch commits successfully
and persisted across ticks. A failed tick leaves the cursor untouched so
the same window is retried.
3. **Two replication races are now explicitly handled.**
- Admin-delete vs reconcile: the existence check the reconciler uses to
avoid touching a just-deleted profile reads from the primary, not a
replica.
- Insert lag in the reconciler's own listings: hosts that appear in the
cursor query but are not yet visible in the scoped listings advance the
cursor instead of jamming the loop.
4. **`updates.WindowsConfigProfile` from `BulkSetPendingMDMHostProfiles`
is now always false in production.**
The only consumer ORs it with the transactional signal from
`BatchSetMDMProfiles`, which is the accurate source. The bulk-set call
no longer attempts to compute or return that activity signal itself.
5. **Tests opt in to the old synchronous behavior via a named hook.**
Default test behavior matches production (deferred). Legacy tests whose
assertions require Windows rows immediately after bulk-set call an
explicit enable-hook and rely on `t.Cleanup` to restore.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Windows MDM profile reconciliation batching improvements enable large
team transfers and bulk profile change operations to complete faster,
with profile updates rolling out in the background without blocking host
check-ins or other MDM activity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#30877
We need to send `platform_like` during orbit enrollment for proper setup
experience for Linux
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
Fixes#30473
Refactore Datastore.EnrollHost and Datastore.EnrollOrbit methods to use
functional options. Doing this refactor before adding new options to
those methods. This should make the code more maintainable and easier to
understand.
No functional changes here. Just refactoring.
# Checklist for submitter
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Streamlined host and Orbit enrollment methods to use a flexible
options-based pattern instead of fixed parameter lists.
* Updated related tests and service logic to use the new options
approach, improving clarity and extensibility for enrollment operations.
* **New Features**
* Introduced configuration options for host and Orbit enrollment,
allowing more explicit and customizable parameter setting during
enrollment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #26713
# Details
This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.
Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:
> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.
Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.
The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
- [x] Orbit runs on macOS ✅ , Linux ✅ and Windows.
- [x] Manual QA must be performed in the three main OSs, macOS ✅,
Windows and Linux ✅.
Konstantin Sykulev
