Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 21 Packages Projects Releases Wiki Activity
25,953 Commits 2 Branches 0 Tags
bfc986df7e5117e2ebdd61589d908da2e4a58dab
Commit Graph
5267 Commits
Author SHA1 Message Date
George Karr 69fa5ca435 Fix VPP/in-house app install on manual-profile BYOD iOS hosts (#48879) (#48916) 
**Related issue:** Resolves #48879

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (parameterized queries only).
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes — N/A,
no endpoint/path changes.

## Summary

Installing an App Store (VPP) or in-house app on an iOS/iPadOS host
enrolled via the **manual (profile-driven) BYOD** enrollment profile
failed: Fleet routed the install down the **Account-Driven User
Enrollment (user-scoped)** licensing path, tried to look up/register a
VPP user keyed on the host's Managed Apple ID, and returned _"Fleet
hasn't received a Managed Apple ID for this host yet."_ — which never
resolves, because a device-channel host has no Managed Apple ID.

### Root cause

The device-vs-user licensing decision keyed off
`host_mdm.is_personal_enrollment`. That flag is set for **both**:
- **Account-Driven User Enrollment** — user channel, backed by a Managed
Apple ID → user-scoped licensing (correct).
- **Manual-profile BYOD** — device channel, no Managed Apple ID → must
install **device-scoped**, exactly like company-owned manual enrollment.

### Fix

Branch on the actual enrollment **channel** — the presence of a
user-channel `nano_enrollments` row (`type='User' AND enabled=1`), the
same signal the MDM profile reconcile cron already uses
(`GetNanoMDMUserEnrollment`). This is timing-robust: the user
nano-enrollment exists from enrollment time, whereas the Managed Apple
ID only arrives minutes later via `TokenUpdate` (so `managed_apple_id`
emptiness is deliberately **not** used as the discriminator).

Three sites updated:
| File | Change |
|---|---|
| `ee/server/service/software_installers.go` |
`InstallVPPAppPostValidation` routes on `GetNanoMDMUserEnrollment`
instead of `is_personal_enrollment` |
| `server/datastore/mysql/vpp.go` | InstallApplication builder derives
`IsUserEnrollment` (ChangeManagementState omission) from a user-channel
`nano_enrollments` row |
| `server/datastore/mysql/activities.go` | same, for in-house `.ipa`
installs |

## Testing

- [x] Added/updated automated tests:
- `ee/server/service`:
`TestInstallVPPAppPostValidation_AssociateAssetsRouting` — added a
regression subtest asserting manual-profile BYOD (personal flag set,
device channel) routes via `serialNumbers` and performs **no** VPP user
lookup; repointed routing to the user-channel signal.
- `server/datastore/mysql`: new
`TestVPP/VPPInstallEnrollmentChannelRouting` — manual BYOD includes
`ChangeManagementState` despite `is_personal_enrollment=1`;
account-driven User Enrollment omits it.
- [x] Automated tests simulate multiple hosts and test for host
isolation (two distinct hosts, device- vs user-channel).
- [ ] QA'd all new/changed functionality manually — pending (draft).

For unreleased bug fixes in a release candidate:

- [x] Confirmed that the fix is not expected to adversely impact load
test results (adds one indexed lookup per install enqueue; removes a
`host_mdm` join).

## Database migrations

- N/A — no schema changes. The fix reads existing `nano_enrollments`
rows.

## fleetd/orbit/Fleet Desktop

- N/A

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed app installation for manually enrolled BYOD iPhone and iPad
devices so App Store and in-house apps install correctly on the device.
* Improved enrollment handling so device-scoped installs no longer fail
when a device is marked personal in one place but uses device-channel
enrollment.
* Account-Driven User Enrollment continues to use user-scoped licensing
and installs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 07:32:37 -05:00
Victor Lyuboslavsky e1094096af Surface proxied Windows SCEP certificate failures (#45550) (#48842) 
Windows configuration profiles that Fleet proxies SCEP for previously
reported "verified" as soon as the device acknowledged the SyncML Exec
command, even when the asynchronous SCEP exchange later failed and no
certificate was ever issued.

- Proxied SCEP profiles (custom SCEP proxy, NDES) now move to
"verifying" on the device ACK and only reach "verified" once Fleet
observes the matching certificate on the host, keyed by the renewal-ID
marker (fleet-<profile_uuid>) in the certificate CN/OU.
- When Fleet's SCEP proxy observes an upstream CA error during
PKIOperation, it marks the profile "failed" with a detail naming the
operation and upstream status. If the device's own retry later succeeds,
the observed certificate flips the profile to "verified".
- Unconfirmed profiles stay "verifying" (offline host, agent that cannot
enumerate certificates, empty store, or a user-scoped profile before the
user logs in); absence is never treated as failure.

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #45550 

Demo: https://www.youtube.com/watch?v=WNGuFdeBmzA
Docs: https://github.com/fleetdm/fleet/pull/48933/changes

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* Added Windows SCEP failure tracking with clearer, categorized detail
when upstream operations fail.
* Added reconciliation backstops for “stuck” proxied SCEP profiles,
including automatic recovery to verified when the expected certificate
is observed.

* **Bug Fixes**
* Prevented proxied Windows SCEP installs from being marked “verified”
until matching certificate evidence arrives.
* Improved classification and persistence behavior for timeouts,
connection/DNS issues, and HTTP error responses without disturbing
existing retry state.

* **Tests**
* Expanded Windows SCEP scenarios to cover reconciliation, skipping
conditions, and error classification.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 07:38:47 +01:00
Jonathan Katz 6d1938b914 Fix test setting flag that caused further tests to fail (#48947) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #
Moves `TestInstallAllSelfServiceSoftware` from the enterprise
integration suite (`TestIntegrationsEnterprise`) to the MDM integration
suite (`TestIntegrationsMDM`), because setting the
`MDM.EnabledAndConfigured` flag in the enterprise suite was leaking into
and failing other tests (`TestLinuxDiskEncryption`, `TestTeamEndpoints`,
`TestTeamSpecs`, `TestMDMNotConfiguredEndpoints`,
`TestVPPAppsWithoutMDM`,
`TestOrbitSetupExperienceStatusChecksAuthBeforeMDM`).


# Checklist for submitter

## Testing

- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually
- checking this to avoid a CI failure, but there is nothing to actually
manually check

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Expanded integration coverage for self-service software installs
across team, label, category, and multi-host scenarios.
* Added checks for install ordering, idempotency, queue consistency, and
concurrent requests.
* Included coverage for VPP-backed apps and mixed install queues to
better validate real-world behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-08 13:11:01 -04:00
Luke HeathClaude Fable 5Lucas Manuel Rodriguez
 		003ab766d3 Filter cross-team memberships from user list responses (#48890) 
From Lucas:
- [X] QA'd all new/changed functionality manually

## Summary

A team-scoped admin listing users of a team they administer (`GET
/api/latest/fleet/users?team_id=A`) received the full team membership —
team IDs, names, and roles — of any user also shared with other teams,
disclosing teams the requester has no role in.

The single-user `GET /users/{id}` endpoint already blocks this: its
authorization requires the requester to administer *every* team the
target belongs to. The list endpoint authorizes against a synthetic
single-team object (correct, so team admins can manage their members),
but then returned each user's complete team list as loaded by the
datastore.

This filters each returned user's teams down to the requester's scope at
the response layer. Requesters with any global role are unchanged
(they're authorized to see all teams).

## Why the response layer, not `Service.User`

`ModifyUser` and the password-reset flow reuse `Service.User` and read
`user.Teams` to compute write diffs. Filtering there would silently drop
team memberships on edits, so the filter is applied in
`listUsersEndpoint` only.

`GET /users/{id}` is intentionally not changed — it is not exploitable
(authz already requires admin-of-all-the-target's-teams), and its
legitimate readers should keep seeing the full team list.

## Testing

- `TestListUsersFiltersTeamsToRequesterScope` — team-1 admin listing
team 1 sees only team 1 for a user shared with {1,2}.
- `TestListUsersGlobalRequesterSeesAllTeams` — global admin sees all
teams.
- Existing `TestUserAuth` / `TestAuthorizeUser` pass unchanged (no authz
regression).

Fixes fleetdm/confidential#16691

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Fixed user listing so returned team membership details are scoped to
the requesting user’s permissions, including fleet-scoped context.
* Team-scoped requesters now only see memberships for teams they’re
allowed to view; global-role requesters still see all memberships.
* When scoped viewer context is missing, team membership details are no
longer included in the response.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
 2026-07-08 13:43:14 -03:00
Victor Lyuboslavsky 19aac451e1 Fix Windows CSP bypass issue (#48843) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48752 

Stacked PR. Needs 48349-windows-modify branch to merge first.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed a Windows MDM loophole where scope-less or differently formatted
`LocURI` values could bypass Fleet restrictions.
* Strengthened detection and enforcement for reserved Windows targets,
including OS updates, remote wipe premium gating, and BitLocker
restrictions.
* Improved `LocURI` handling to be resilient to whitespace and alternate
formatting, including more consistent SCEP profile processing.

* **Tests**
* Added regression coverage for reserved `LocURI` matching, OS-update
targeting, and premium detection for wipe commands (including scope-less
cases).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-08 14:38:10 +01:00
CarloandCopilot Autofix powered by AI 9463a46c32 Fix Windows software ingest lock contention by matching titles on upgrade_code (#48902) 
**Related issue:** Resolves #48875

  # Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

  ## Testing

  - [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
  - [x] QA'd all new/changed functionality manually

  For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

**Functional:** `TestSoftwareTitleUpgradeCodeDriftMatch` (added here): a
host reporting a Windows program whose name has drifted from the stored
title but shares its `upgrade_code` must resolve to the existing title.
Fails on pre-fix code, passes with the fix.

**Load:** 25 concurrent hosts × 4 rounds, each reporting 50 drifted
programs (sharing the stored `upgrade_code`s) through
`UpdateHostSoftware`; doomed inserts counted via the MySQL general log:

  | concurrent-burst metric | without fix | with fix |
  |---|---|---|
  | doomed `INSERT IGNORE INTO software_titles` | 175 | 0 |
  | `Innodb_row_lock_waits` (Δ) | 715 | 0 |
  | burst wall time | ~4.9 s | ~0.2 s (~25× faster) |

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **Bug Fixes**
* Improved matching for Windows software programs when the displayed
name changes but the upgrade code remains the same.
* More reliably reuses existing software titles during ingest, reducing
duplicate title entries.
* Added a regression test to confirm upgrade-code matches take priority
and unknown upgrade codes don’t create false matches.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-08 08:00:05 -04:00
Magnus Jensen ace8cf046b SAAD: DDM Asset table migration (#48866) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48566 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information. Coming in bigger backend story.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for tracking Apple declaration assets, including a new
asset record and a link table for associating assets with declarations.
* Added a new timestamp on declarations to reflect the latest asset
update time.

* **Bug Fixes**
* Strengthened database constraints to prevent duplicate asset entries
and enforce valid asset/declaration references.
* Improved delete behavior so referenced declarations clean up related
links automatically.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-08 09:17:33 +02:00
Dante Catalfamo 4351f4cee5 escrow snapd TPM-backed FDE recovery keys from orbit (#48452) 
**Related issue:** Resolves #44428
 2026-07-07 16:25:20 -04:00
Dante Catalfamo 57dc28991a Add resolved-in-version override for CVE-2025-63389 on Ollama (#48525) 
**Related issue:** Resolves #44800
 2026-07-07 16:24:41 -04:00
Carlo dfe0f1c871 Fix App Store picker 403 for non-admin roles (#48856) 
**Related issue:** Resolves #46057

Authorize `GetVPPTokens` against `VPPApp` instead of admin-only
`AppleCSR`, so maintainer/technician roles no longer get a 403 that
broke the App Store picker.

  # Checklist for submitter

  - [x] Changes file added for user-visible changes in `changes/`.

  ## Testing

  - [x] Added/updated automated tests
  - [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Fixed the “Add software > App Store” picker so maintainer and
technician roles no longer encounter access errors when browsing VPP
tokens.
* Improved VPP token visibility for team-scoped users by restricting
listings to teams they can read, while including “all teams” tokens and
excluding unassigned/unauthorized ones.
* Ensured users without appropriate access receive the correct
authorization response instead of broader token listings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 16:15:58 -04:00
Victor Lyuboslavsky 4608e82481 Added anonymous usage statistics reporting the number of macOS and Windows hosts enrolled in Fleet's MDM (#48840) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48685

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
- Tested `ShouldSendStatistics` method manually against our DB, which
covers all our changes.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* Added anonymous usage statistics for the number of macOS and Windows
hosts currently enrolled in Fleet’s MDM.

* **Bug Fixes**
* Improved statistics accuracy by counting only actually enrolled,
non-server macOS and Windows hosts that are using Fleet’s MDM.

* **Tests**
* Updated and extended statistics tests to verify the new enrollment
counts are computed and reported correctly.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 19:22:31 +01:00
Jordan Montgomery 4c79d6bddd Add user-scoped declaration support (#48796) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for Apple declarative management declarations on both
System and User channels.
* User-scoped declarations are now delivered, reconciled, and
acknowledged independently from device-scoped declarations.
* **Bug Fixes**
* Prevented scope-mixing so declaration items and status updates no
longer affect the wrong channel.
* Tightened reconciliation behavior for scope changes and missing user
channels.
* **Tests**
* Expanded coverage for channel isolation, payload scope
parsing/validation, and correct delivery payload behavior (including
stripping the payload-scope field from delivered JSON).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 13:15:14 -04:00
Jordan MontgomeryandCopilot Autofix powered by AI 3b7c88fb87 Fix dupe profile enqueue bug (#48652) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48633

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Prevented duplicate profile enqueueing for hosts that share the same
hardware UUID.
* Reconcile processing now consistently picks the highest matching host
record when duplicates exist.
* Duplicate enrollment IDs are now filtered out before queueing,
reducing repeated work and avoiding queue conflicts.

* **Tests**
* Added regression coverage for duplicate-host and duplicate-enqueue
reconcile scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-07 13:08:26 -04:00
Victor Lyuboslavsky 8f3624cf0a Fixed Windows profile modify batch (#48474) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48349, as well as a few other minor issues
found during dev (such as canonical LocURI, ensuring we delete the CSP
version actually on the device, etc.).

Load tested the fix.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Windows profile edits and deletions now handle large environments more
reliably, with faster processing and no size-based timeouts.
* Removed profile content is now cleaned up asynchronously, improving
the responsiveness of profile changes.
* **Bug Fixes**
* Fixed Windows profile edits so removed settings are deleted correctly
even when profiles are updated instead of fully removed.
* Improved matching for Windows configuration targets, making cleanup
more consistent across profile versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 16:29:53 +01:00
Carlo 724835658a Follow-up fix for FMA counts (#48818) 
**Related issue:** Resolves #48528

Follow-up to #48783, which changed the Fleet-maintained apps "items"
count from per-platform entries to per-app, dropping it from 1,263 to
1,023. This restores the count to `COUNT(DISTINCT fma.id)`: macOS and
Windows entries are separately installable (each its own Add button), so
each counts (1,263 / 960 macOS / 303 Windows). The token-based
row-combining and pagination from #48783 are kept.

  # Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

  ## Testing

  - [x] Added/updated automated tests
  - [x] QA'd all new/changed functionality manually

  For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Corrected available-app counts in listings so pagination totals now
match what users can actually add.
* Improved pagination consistency for apps with multiple platform
variants, reducing confusion where totals did not align with visible
entries.
* Updated team-based filtering so already-added apps are excluded more
accurately from available results.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 10:30:47 -04:00
Jordan Montgomery 88ee1fee97 Fix re-enrollment with pending SCEP(and ACME) renewals (#48661) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48486 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Apple MDM devices manually re-enrolled during a pending SCEP renewal
are now handled as a fresh enrollment, so enrollment steps run
correctly.
* Renewal and re-enrollment flows are now better distinguished, reducing
cases where profile or app setup could be skipped.
* Enrollment certificates now carry clearer markers to help the system
apply the right lifecycle behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 09:26:14 -04:00
Nico 7dfcb76a02 Add POST /reports/run to the API endpoints catalog (#48790) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Relates to #43544

The Fleet MCP server runs multi-host live queries by creating an ad-hoc
campaign via `POST /api/v1/fleet/reports/run`, but that route is missing
from the API endpoints catalog. An api-only user restricted to a
specific endpoint allowlist therefore cannot be granted it and receives
a 403, so multi-host live queries fail under a least-privilege setup.
This adds the route to the catalog so it can be granted; it stays gated
by observer_plus RBAC and does not match any allowlist-bypass blocklist
rule.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] QA'd all new/changed functionality manually



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added support for an asynchronous live report run endpoint.
* API-only users on restricted allowlists can now be granted access to
run reports.
* **Changes**
* Updated the live report endpoint path and display name to reflect the
async behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 09:59:12 -03:00
Lucas Manuel Rodriguez b3b3a42fed Additional changes for Zorin OS support (#48779) 
Follow up PR for community PR:
https://github.com/fleetdm/fleet/pull/45712.

- [X] QA'd all new/changed functionality manually

<img width="1574" height="827" alt="Screenshot 2026-07-06 at 1 54 44 PM"
src="https://github.com/user-attachments/assets/672d7b84-155f-4dab-8246-fe88e391e416"
/>
<img width="1235" height="827" alt="Screenshot 2026-07-06 at 1 54 01 PM"
src="https://github.com/user-attachments/assets/51df3344-b002-45df-9cda-afa573652944"
/>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* OS settings and disk-encryption views now correctly include Zorin
devices in Linux-related results.
* Host filtering counts now account for Zorin alongside other supported
Linux platforms.
* Updated related checks so Zorin devices are handled consistently in
status and encryption reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-07 09:58:30 -03:00
Victor Lyuboslavsky d7692a43ef Add FLEET_MDM_ENABLE_DISK_ENCRYPTION alias for custom BitLocker profiles (#43518) (#48737) 
**Related issue:** Resolves #43518

Adds a cross-platform alias `FLEET_MDM_ENABLE_DISK_ENCRYPTION`
(`mdm.enable_disk_encryption`) for the existing
`FLEET_MDM_ENABLE_CUSTOM_FILEVAULT` server configuration. When either
option is set, Fleet allows both custom Apple MDM profiles for FileVault
and custom Windows configuration profiles for BitLocker. Behavior
matches FileVault: no special conflict handling between Fleet's built-in
disk encryption controls and a custom profile. The setting remains Fleet
Premium only.

Both the single-add API/UI path and the batch/GitOps path are covered.
The existing `FLEET_MDM_ENABLE_CUSTOM_FILEVAULT` name continues to work
for backward compatibility.

Demo: https://www.youtube.com/watch?v=5naGaZKLZ8o
Docs: https://github.com/fleetdm/fleet/pull/48738/changes

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## New Fleet configuration settings

- [x] Setting(s) is/are explicitly excluded from GitOps


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a cross-platform disk encryption setting that can enable custom
management for both macOS FileVault and Windows BitLocker profiles.

* **Bug Fixes**
* Windows BitLocker profile uploads are now accepted when custom disk
encryption is enabled.
* Startup now disables custom disk encryption management when the
license does not support it, and logs a warning.

* **Tests**
* Added coverage for BitLocker profile handling with custom disk
encryption enabled and disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 22:38:45 +01:00
Jordan Montgomery b526909b7a Persist byod=true enroll param through IdP redirects (#48808) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48805

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Preserve a user’s BYOD selection through IdP authentication so it no
longer gets lost mid-flow.
* Enrollment redirects to IdP SSO now retain the correct enrollment
query settings (including BYOD and fully managed) for consistent
enrollment behavior.
* **Tests**
* Added coverage to ensure the SSO initiation redirect preserves the
expected query parameters and returns the correct redirect response.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 16:19:42 -05:00
Carlo b3c2a6368d Count FMAs by slug (#48783) 
**Related issue:** Resolves #48528

This keys the count, pagination, and the frontend row-combining on the
app's slug token (the prefix before `/`, shared across an app's platform
entries but distinct across apps). The count now equals the rows shown
in every view (macOS, Windows, All), and name-colliding apps stay as
separate rows.

  # Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

  ## Testing

  - [x] Added/updated automated tests
  - [x] QA'd all new/changed functionality manually

  For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Software listings now group platform-specific installers into a single
app row based on the app identifier, improving how macOS and Windows
entries appear together.

* **Bug Fixes**
* Apps with the same display name but different identifiers now stay
separate instead of being merged incorrectly.
* List counts and pagination now match the combined app view more
accurately across the software pages.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 17:05:59 -04:00
Victor Lyuboslavsky a7c21caa32 Removed the unused /api/mdm/microsoft/auth Windows MDM STS endpoint (#48734) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #41056 

Docs: https://github.com/fleetdm/fleet/pull/48735/changes

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary

* **Bug Fixes**
* Removed the obsolete Windows MDM authentication (unauthenticated STS)
endpoint; it now returns **HTTP 404**.
* Streamlined the Windows enrollment flow so only the supported
Microsoft MDM endpoints are exposed.

* **Testing**
* Added an integration test to confirm the removed endpoint remains
inaccessible.
* Removed now-irrelevant unit tests and helpers related to the deleted
authentication behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 20:29:05 +01:00
Victor Lyuboslavsky bf94df6e6f Show certificates on host details page for Windows (#31294) (#48469) 
Surface the existing "Certificates" card on the host details page for
Windows hosts, with parity to macOS. Requires osquery 5.23.1 or higher.

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #31294

Demo video: https://www.youtube.com/watch?v=kGRp-YtnnJc
Docs: https://github.com/fleetdm/fleet/pull/48493/changes

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* Windows host certificates now display on the host details page (gated
by minimum agent/osquery version), including scope (**System** vs
**User**) and improved scope-aware certificates list details.

* **Bug Fixes**
* Certificate table labeling and help text are now platform-appropriate
(with “Keychain” renamed to “Scope”).
* Windows certificate reconciliation is more resilient, preserving
certificates for scopes not observed during a collection run and
preventing row collapsing when ids repeat across scopes.

* **Tests**
* Expanded coverage for Windows/malformed DN parsing and scope-aware
reconciliation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 20:28:21 +01:00
Lucas Manuel Rodriguez 9f3e05c06c Fix data race detected by Splunk tests in CI (#48778) 
Fixes data race detected in
https://github.com/fleetdm/fleet/actions/runs/28769705097/job/85300822820.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved reliability of log delivery by ensuring buffered log data is
copied before being sent, preventing intermittent issues when batches
are processed.
* Reduced the risk of log entries being corrupted or lost during
transmission.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 14:42:03 -03:00
Victor Lyuboslavsky 51f1e85c05 Improved the performance of Windows MDM profile installation (#48733) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #45650 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **Bug Fixes**
* Improved MySQL migration handling for MDM command results by safely
removing an outdated foreign key when present, preventing issues during
upgrade and re-run scenarios.
* Updated the database schema definition to keep related response
foreign key behavior consistent.
* **Chores**
* Added the latest migration version to the migration status seed data
to ensure version tracking stays in sync.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 18:34:10 +01:00
fletcher-rudra dfc8c272d3 Add Zorin OS as a recognized Linux platform (#45712) 
**Related issue:** Resolves #45710

# Checklist for submitter

- [x] Changes file added (`changes/45710-zorin-os-support`).
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops.
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes.

## Testing

- [x] Added/updated automated tests —
`server/vulnerabilities/oval/oval_platform_test.go` extended with Zorin
→ Ubuntu LTS mapping cases (16/17/18) plus an unknown-version case
(`Zorin OS 99` → `zorin_99`, which `IsSupported()` rejects).
- [x] QA'd all new/changed functionality manually — Zorin OS 17.0 and
18.1 hosts enrolled against a patched Fleet server, host details show
`platform=zorin`, software inventory populates, and OVAL CVE matching
produces results against the corresponding `ubuntu_2204` / `ubuntu_2404`
feeds.

## Database migrations

- N/A. No schema changes.

## New Fleet configuration settings

- N/A. No new settings.

## fleetd/orbit/Fleet Desktop

- N/A. Server + frontend only; no fleetd/orbit changes.

---

## Summary

Fleet previously logged `unrecognized platform` for Zorin OS hosts
(osquery reports `platform=zorin` from `/etc/os-release` `ID=zorin`).
The common workaround was running osquery with
`--force_platform=ubuntu`, which masquerades the host. This change adds
`zorin` as a first-class Linux platform alongside Ubuntu:

- **`server/fleet/hosts.go`** — register `zorin` in `HostLinuxOSs` and
`HostDebPackageOSs`
- **`server/datastore/mysql/linux_mdm.go`** — include Zorin in the Linux
disk-encryption summary query
- **`server/vulnerabilities/oval/oval_platform.go`** — map Zorin major
version to the underlying Ubuntu LTS OVAL feed (16 → 20.04, 17 → 22.04,
18 → 24.04). Unknown future versions fall through to an unsupported
`zorin_<major>` identifier so vulnerability scanning is skipped rather
than served stale data from an aging LTS feed.
- **frontend** — add `zorin` to `HOST_LINUX_PLATFORMS`, the
disk-encryption support list and type guard, the label platform
dropdown, and the icon mapping (Ubuntu icon, since no Zorin-specific
asset exists in the repo).

No new dependency, schema migration, or config setting. Reuses existing
Ubuntu OVAL feeds and the existing Ubuntu icon.

Diff is ~30 lines net across 9 files (8 patched + 1 `changes/` file).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added Zorin OS as a supported Linux platform.
* Zorin hosts included in Linux disk-encryption summaries and treated as
disk-encryption capable.
* Zorin OS available as a selectable/filterable platform label and
considered DEB-install compatible.
* Vulnerability scanning enabled for Zorin 16→Ubuntu 20.04, 17→22.04,
18→24.04; unknown/future Zorin versions are marked unsupported and
skipped for CVE matching.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45712?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 13:00:10 -03:00
Andrew Mellor 2abc49ba02 46235 dep profile assigner context cancelled (#48473) 
**Related issue:** Resolves #46235

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests

- [ ] QA'd all new/changed functionality manually:  Pending if possible


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed DEP sync so progress is only saved after device data is written
successfully, preventing missed enrollment events during interrupted
syncs.
* Improved handling of sync errors so the next run can safely replay
affected devices instead of skipping them.
* Added end-to-end and scenario coverage to verify cursor behavior after
successful syncs, errors, and expired cursors.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 14:16:14 +01:00
Lucas Manuel Rodriguez 1c1fae8e93 Add CachyOS support (part 2/2) (#48688) 
**Related issue:** Fully resolves
https://github.com/fleetdm/fleet/issues/34591.

## Testing

- [X] QA'd all new/changed functionality manually.

<img width="533" height="454" alt="Screenshot 2026-07-03 at 10 40 37 AM"
src="https://github.com/user-attachments/assets/892fb548-21c6-467c-b270-65f1c9338fdc"
/>
<img width="1287" height="259" alt="Screenshot 2026-07-03 at 10 41
55 AM"
src="https://github.com/user-attachments/assets/d3528b0c-0d05-4ace-8512-ab363241b97c"
/>
<img width="1077" height="123" alt="Screenshot 2026-07-03 at 10 41
46 AM"
src="https://github.com/user-attachments/assets/249e80de-320c-48f3-962a-59c98c736c54"
/>
<img width="725" height="208" alt="Screenshot 2026-07-03 at 10 41 32 AM"
src="https://github.com/user-attachments/assets/361764cf-26fc-4a44-b5d6-489d883a392b"
/>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added CachyOS Linux to rolling-release OS detection and reporting.
* Added a CachyOS fleetd package/image variant and a new CachyOS fleetd
service for local testing.

* **Bug Fixes**
* Improved rolling-release OS version labeling for host “Vitals”
display.
* Updated OS inventory normalization so CachyOS is aggregated with Arch
Linux, including correct “rolling” version handling.

* **Tests**
* Expanded OS version ingest test coverage for rolling-release and
CachyOS scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-06 09:12:15 -03:00
Rachael ShawJordan MontgomeryVictor LyuboslavskyMarko LisicaMike ThomasScott Gressmelpike
 		480847b7f5 v4.88.0 doc changes (#46357) 
Documentation changes for 4.88.0

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added a new chart data API endpoint for retrieving metric-based chart
information.
* **Tests**
* Updated test server setup so chart-related routes are included in
endpoint validation, improving coverage and consistency.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Co-authored-by: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com>
Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: melpike <79950145+melpike@users.noreply.github.com>
 2026-07-03 17:22:51 -05:00
Sharon KatzandClaude Opus 4.6 6ba04b0d20 Optimize query aggregated stats cron to skip queries without execution data (#48698) 
**Related issue:** Resolves #48697

## Summary

The hourly `UpdateQueryAggregatedStats` cron job currently walks **every
query ID** in the `queries` table and runs 5 expensive
percentile-calculation queries per query against
`scheduled_query_stats`, plus 1 INSERT/UPDATE to store results. Most
queries have no execution data at all (they are saved queries,
live-only, or de-scheduled), so this work is pure waste.

This PR changes the cron to only process queries that actually have
execution data, by querying `scheduled_query_stats` directly instead of
the `queries` table. The now-unused `walkIdsInTable` helper function is
also removed.

### How the calculations work

`CalculateAggregatedPerfStatsPercentiles` computes performance
statistics for each query that has been scheduled and executed by hosts.
For each qualifying query ID, it runs these operations against the read
replica:

1. **P50 user_time** -- Calculates the median (50th percentile) of
per-host average user-mode CPU time. The query groups
`scheduled_query_stats` rows by `host_id`, computes `SUM(user_time) /
SUM(executions)` per host, sorts them, then picks the row at position
`FLOOR(total_rows * 0.5) + 1` using a `@rownum` session variable.

2. **P95 user_time** -- Same calculation but picks the 95th percentile
row (`FLOOR(total_rows * 0.95) + 1`).

3. **P50 system_time** -- Same percentile calculation for kernel/system
CPU time.

4. **P95 system_time** -- 95th percentile of system CPU time.

5. **Total executions** -- `SELECT COALESCE(SUM(executions), 0) FROM
scheduled_query_stats WHERE scheduled_query_id = ?`

6. **INSERT/UPDATE** -- Writes the JSON result (`user_time_p50`,
`user_time_p95`, `system_time_p50`, `system_time_p95`,
`total_executions`) into the `aggregated_stats` table via `INSERT ... ON
DUPLICATE KEY UPDATE`.

### What changed

**Before:** `SELECT id FROM queries` -- walks every query (200-400+ in a
typical deployment).

**After:** `SELECT DISTINCT scheduled_query_id FROM
scheduled_query_stats WHERE executions > 0` -- walks only queries that
have actual execution data (typically 10-20).

### Benchmark results (MySQL 8.0, 300 queries seeded, only 15 with
stats)

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Avg time per cron run | 3.36s | 0.28s | **12.2x faster** |
| DB operations per run | 1,800 | 90 | **95% fewer** |
| DB operations per day | 43,200 | 2,160 | **41,040 eliminated** |
| `aggregated_stats` rows written | 300 (285 empty) | 15 (all
meaningful) | Less table bloat |
| Correctness | baseline | byte-identical JSON | **Zero regression** |

At 500+ queries the current approach **drops MySQL connections**
(`unexpected EOF` / `invalid connection`) because the cursor is held
open across thousands of heavy serial queries. The optimized version
handles any scale trivially.

### Impact analysis

Verified safe across all consumers: all query endpoints use `LEFT JOIN
aggregated_stats` (NULL-safe for missing rows), the frontend explicitly
handles null stats as "Undetermined", live query stats
(`service_campaigns.go`) call `CalculateAggregatedPerfStatsPercentiles`
directly and are unaffected, and query deletion already cleans up both
`scheduled_query_stats` and `aggregated_stats` rows.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Confirmed that the fix is not expected to adversely impact load
test results

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-03 13:29:49 -04:00
Lucas Manuel Rodriguez 1ceca6ad8e Cleanup policy_membership stale entries in distributed/write (#48674) 
Resolves #47241.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [X] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Better host policy results by automatically cleaning up out-of-scope
`policy_membership` records.
* Refreshes host failing-policy counts after cleanup, including when
distributed writes report “no policies in scope.”
* Preserves existing safeguards by skipping this cleanup during
setup/initial configuration to prevent premature updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-03 14:12:08 -03:00
Lucas Manuel Rodriguez 171504dc18 Fix nondeterministic device_mapping order in ListHosts (#48696) 
Fixes the `device_mapping` ordering flake in
`TestIntegrations/TestListHostsByLabel` and
`TestIntegrations/TestHostsReportDownload`, seen across integration-core
jobs on `main` since July 1 (e.g. [this
run](https://github.com/fleetdm/fleet/actions/runs/28567132474)).

#48488 replaced the derived-table `GROUP_CONCAT` join in `ListHosts`
with a correlated subquery, but the `GROUP_CONCAT` has no `ORDER BY`, so
MySQL returns `device_mapping` entries in arbitrary order. The old plan
happened to read `idx_host_emails_host_id_email` in index order, which
masked this; the new access path doesn't, so the order now varies
between endpoints and runs — the tests compare `GET /hosts` output
against `GET /labels/{id}/hosts` (and CSV report) output for the same
host and intermittently see `[b@b.c, a@b.c]` vs `[a@b.c, b@b.c]`.

This adds `ORDER BY he.email, he.source` inside the `GROUP_CONCAT`,
matching the ordering of the single-host `listHostDeviceMappingDB`
query. The sort applies only within each host's few email rows, so it
doesn't affect the perf improvement from #48488.

No changes file: #48488 is unreleased (not in any RC branch), so this is
a fix to an unreleased change.

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

Covered by existing tests: `TestIntegrations/TestListHostsByLabel` and
`TestIntegrations/TestHostsReportDownload` assert the (now
deterministic) ordering. Ran both 4× locally with `MYSQL_TEST=1
REDIS_TEST=1`, plus the `TestHosts` device-mapping/ListHosts datastore
tests — all green.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved consistency of host device mapping results by making the
ordering deterministic.
* Fixed an issue where device mapping entries could appear in different
orders between requests.
* **Tests**
* Added coverage to verify the device mapping order remains stable and
predictable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-03 14:09:07 -03:00
Sharon KatzandClaude Opus 4.6 b36be84e85 Add native Splunk HEC log destination (#48455) 
**Related issue:** Resolves #25574

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

---

## Summary

- Adds a new `splunk` log plugin that sends osquery logs directly to
Splunk's HTTP Event Collector (HEC) endpoint
- Eliminates the need for middleware like AWS Firehose when using Splunk
as a log destination
- Follows the same pattern as existing log destinations (Firehose, Kafka
REST, NATS, etc.)
- Includes `insecure_skip_verify` option for environments with
self-signed TLS certs

## UI changes

Follows the same pattern as the NATS log destination PR (#36527) --
adding "Splunk" to the display name, tooltip, and TypeScript type union.
No new components, pages, or styles.

### Manage automations modal -- "Log destination: Splunk"
<img width="822" height="527" alt="image"
src="https://github.com/user-attachments/assets/2533207f-fa95-4364-8ee0-3c39cd3e8e4d"
/>


### Query details page -- "Log destination: Splunk"
<img width="1905" height="662" alt="image"
src="https://github.com/user-attachments/assets/069a5005-f95c-4562-a819-fd8bdcc349f7"
/>



### Tooltip on hover
<img width="639" height="348" alt="image"
src="https://github.com/user-attachments/assets/809a47a6-b82a-4f45-b731-77b2d2c87947"
/>



### Edit query form -- "sent to your log destination: Splunk"
<img width="451" height="814" alt="image"
src="https://github.com/user-attachments/assets/b78b9a57-1f0c-4413-8b7c-654de1fd40a2"
/>



### Save new query modal -- "sent to your log destination: Splunk"
<img width="536" height="698" alt="image"
src="https://github.com/user-attachments/assets/d0a0ab01-66fe-4d63-9190-9c5e840e456d"
/>

---

### How it works

The Splunk writer (`server/logging/splunk.go`) implements the
`fleet.JSONLogger` interface. On startup it performs a health check
against the HEC `/services/collector/health` endpoint. On each `Write()`
call, it wraps each log entry in Splunk's HEC event format (adding
`time`, `index`, `source`, `sourcetype`), batches them up to 1 MB, and
POSTs to `/services/collector/event` with the `Authorization: Splunk
<token>` header. If a batch exceeds 1 MB it flushes and starts a new
one. Events over 1 MB are dropped with a log warning. Transient errors
(HTTP 503) are retried with exponential backoff (up to 8 retries).

### Configuration

```yaml
osquery:
  status_log_plugin: splunk
  result_log_plugin: splunk

splunk:
  url: https://splunk.example.com:8088
  token: <HEC token>
  index: main
  source: fleet
  source_type: fleet:json
  insecure_skip_verify: false  # set true for self-signed certs
```

Or via environment variables:
```
FLEET_OSQUERY_STATUS_LOG_PLUGIN=splunk
FLEET_OSQUERY_RESULT_LOG_PLUGIN=splunk
FLEET_SPLUNK_URL=https://splunk.example.com:8088
FLEET_SPLUNK_TOKEN=<HEC token>
FLEET_SPLUNK_INDEX=main
FLEET_SPLUNK_SOURCE=fleet
FLEET_SPLUNK_SOURCE_TYPE=fleet:json
```

### Files changed
- `server/logging/splunk.go` -- Splunk HEC log writer with batching,
retry, and health check
- `server/logging/splunk_test.go` -- 9 unit tests
- `server/logging/splunk_integration_test.go` -- 3 integration tests
against real Splunk (gated by env var)
- `server/logging/logging.go` -- Added `SplunkConfig` and `case
"splunk"` to factory
- `server/config/config.go` -- Added `SplunkConfig` struct and config
flags
- `cmd/fleet/logging.go` -- Wired Splunk config into logging builder
- `server/fleet/app.go` -- Added `SplunkConfig` type for API responses
(excludes token)
- `server/service/service_appconfig.go` -- Added `case "splunk"` to
logging plugin validation
- `frontend/interfaces/config.ts` -- Added `"splunk"` to LogDestination
type
-
`frontend/components/LogDestinationIndicator/LogDestinationIndicator.tsx`
-- Added Splunk display name and tooltip
- `docs/Configuration/fleet-server-configuration.md` -- Splunk config
documentation
- `docs/Get started/FAQ.md` -- Updated plugin list
- `articles/log-destinations.md` -- Updated Splunk section with native
HEC docs
- `changes/25574-splunk-log-destination` -- Change file

## Test plan

### Unit tests (9 tests)
- [x] `TestSplunkWrite` -- sends 3 events, verifies HEC format, auth
header, index/source/sourcetype
- [x] `TestSplunkWriteEmpty` -- empty logs don't trigger HTTP request
- [x] `TestSplunkServerError` -- HEC 403 propagates as error
- [x] `TestSplunkHealthCheckFailure` -- constructor fails on bad health
- [x] `TestSplunkRecordTooBig` -- oversized events (>1MB) are dropped,
normal events still sent
- [x] `TestSplunkSplitBatchBySize` -- logs exceeding 1MB batch limit are
split into multiple requests
- [x] `TestSplunkRetryOnServiceUnavailable` -- 503 retried with backoff,
succeeds on 3rd attempt
- [x] `TestSplunkRetryExhausted` -- after 9 attempts (1 + 8 retries)
returns error
- [x] `TestSplunkMissingConfig` -- empty URL/token returns descriptive
error

### Integration tests (3 tests, gated by `SPLUNK_INTEGRATION_TEST=1`)
- [x] `TestSplunkIntegration` -- 3 events sent via writer, queried back
from Splunk REST API
- [x] `TestSplunkIntegrationBatch` -- 100 events in one Write(), all
confirmed indexed
- [x] `TestSplunkIntegrationBadToken` -- bad token Write() returns 403

### End-to-end test (macOS ARM64, real osquery agent)

1. Started Splunk Enterprise, MySQL, Redis via Docker
2. Built Fleet server from this branch with
`--osquery_status_log_plugin=splunk`
3. Set up Fleet, enrolled a real osquery 5.23.0 agent on this MacBook
4. **83 real osquery status log events indexed in Splunk** with correct
source/sourcetype/index
5. Each event contained full osquery data (`hostIdentifier`,
`host_uuid`, `calendarTime`, `severity`, `message`, `decorations`)

### Splunk showing real osquery events from Fleet
<img width="1910" height="861" alt="image"
src="https://github.com/user-attachments/assets/192490bf-d594-4424-a3e3-a18306892873"
/>


Generated with [Claude Code](https://claude.ai/code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added native Splunk HEC logging destination for status, result, and
audit logs.
* Updated the log destination UI to display **Splunk** with a dedicated
tooltip.
* Added Splunk HEC configuration (URL/token/index/source/source type)
including TLS verification control.
* **Bug Fixes**
* Improved log delivery with batching, retries for temporary HTTP
failures, and safeguards for oversized events.
* **Tests**
* Added unit tests and optional integration tests covering routing,
batching, retries, and error scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-07-03 12:14:24 -04:00
Lucas Manuel Rodriguez 34af79e98a Fix performance regression in software_macos query (#48649) 
Resolves #47894

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [X] QA'd all new/changed functionality manually

---

Performance results on my macOS host (between the old an new query):

Clean, dramatic result. Subtracting the ~0.23 s / ~27.5 MB osqueryd
startup baseline to isolate the query cost:
```
┌─────────────────────┬───────────┬──────────┬──────────────────────────┐
│                     │ Wall time │ Peak RSS │ Query-attributable work¹ │
├─────────────────────┼───────────┼──────────┼──────────────────────────┤
│ Baseline (SELECT 1) │ 0.23 s    │ 27.5 MB  │ —                        │
├─────────────────────┼───────────┼──────────┼──────────────────────────┤
│ OLD (recursive %%)  │ ~1.46 s   │ 128 MB   │ +1.23 s, +100 MB         │
├─────────────────────┼───────────┼──────────┼──────────────────────────┤
│ NEW (bounded 2+3)   │ 0.24 s    │ 27.8 MB  │ +0.01 s, +0.3 MB         │
└─────────────────────┴───────────┴──────────┴──────────────────────────┘

¹ over baseline
```

Takeaways:
- Memory: ~128 MB → ~28 MB peak (–100 MB). The recursive walk alone
added ~100 MB; the bounded version adds essentially nothing.
- Time: ~1.46 s → ~0.24 s (~6× faster wall clock; the query-attributable
work dropped ~1.23 s → ~0.01 s, effectively free).
- System time tells the story: OLD spends 0.88–0.97 s in sys (the
readdir/stat syscalls from walking the tree); NEW spends ~0.00 s.

And this is with only 6 casks, dominated by gcloud-cli's ~98k-entry SDK
tree (walked twice via the latest → version symlink, plus following the
app back-symlinks into /Applications bundles). The recursive query hit
128 MB peak from a single well-stocked host — already within striking
distance of osquery's 200 MB watchdog limit. On hosts with more or
larger casks (or the /Library//Applications patterns from the issue),
that's exactly what tips it over and kills the worker. The bounded
version is flat regardless.
 2026-07-03 11:04:30 -03:00
plop28andplop28 292fe61301 Add CachyOS support (#47757) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Should Resolve #34591

# Checklist for submitter
- [x] Changes file added 

## User Story
CachyOS lacks from vitals information such as :
* disk encryption status
* disk space
* IP & MAC Addresses
* Installed packages


## Summary
  - Add CachyOS as a recognized Linux platform

## Tests
  - [x] Enroll a CachyOS host and verify it appears as Linux in Fleet
  - [x] Verify disk encryption status displays correctly
- [x] Verify pacman packages are queryable via `fleetd_pacman_packages`
table
  - [x] Disk space, mac address, Public/Private IP are well reported
  - [x] Script are well executed
- [x] No more errors in fleet service logs (level=error
msg="unrecognized platform" hostID=169 platform=cachyos)
  - [ ]  QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* CachyOS (Arch-based Linux distribution) is now recognized as a
supported platform, including disk encryption detection and LUKS
support.
* **Bug Fixes**
* Updated host vitals disk-encryption tooltip messaging so CachyOS uses
the correct Linux-specific copy.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: plop28 <plop28@noreply.com>
 2026-07-03 10:28:08 -03:00
Jonathan Katz 8b1e806754 Fix GitOps creating duplicate software titles (#48664) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48054 
Changes:
- Changes batch add installer path to reuse
`getOrGenerateSoftwareInstallerTitleID`
- Adds migration to retroactively fix duplicate titles created by this
bug

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- The tables will actually be updated, so it makes sense for that to
change if it happens

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Resolved a case where GitOps uploads of Windows software could create
duplicate software titles when a host had already reported the same
program.
* Improved deduplication and reassociation so related records
(installers and icons) are merged into the retained title, preserving
the correct upgrade code.
* **Tests**
* Added regression coverage for the duplicate-title scenario to prevent
future repeats.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-02 19:16:28 -04:00
kilo-code-bot[bot]Kilo Codekiloconnect[bot]
 		e95a8dfb8e Better error message: Configuration profiles has characters that need escaping (#40073) 
- @noahtalerman: For the following quick win:
  - #40074

---------

Co-authored-by: Kilo Code <kilo@fleetdm.com>
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
 2026-07-02 14:32:49 -05:00
George Karr dea65b824c Bump migration timestamps after 4.88 cherry-pick (#48617) 
**Related issue:** Resolves NA (release hygiene — migration ordering)

## What & why

The `4.88.0` patch cherry-picked two migrations,
`20260624210253_AddHostMDMAppleEnrollmentPermissions` and
`20260624210311_RenamePersonalEnrollmentStatus`. Eight migrations on
`main` were **not** cherry-picked into 4.88 but had **earlier**
timestamps than those two:

| Old timestamp | Migration |
|---|---|
| 20260611202649 | AddWindowsMDMConfigProfilesPendingDelete |
| 20260615135619 | AddSetupExperienceSoftwareInstallers |
| 20260617172853 | CreateSoftwareTitleTeamPins |
| 20260617194413 | AddAndroidProfileVariableTracking |
| 20260622124714 | AddPolicyGateToSetupExperienceResults |
| 20260622124734 | AddBYODFleetAndADUEEnrollment |
| 20260623140135 | AddSupportSoftwareCategory |
| 20260624152755 | AddCertAndAndroidAppVariableTracking |

This violates the rule in
`docs/Contributing/workflows/releasing-fleet.md`:

> Any migrations that are not cherry-picked in a patch must have a
_later_ timestamp than migrations that were cherry-picked.

Left as-is, a customer on `4.88.0` (who applied migrations through
`20260624210311`) upgrading to `4.89.0` would hit these 8 as
out-of-order/missing migrations older than their highest-applied
version.

## Fix

Bumped the 8 non-cherry-picked migrations to new timestamps
(`20260702013055`–`20260702013102`) using `tools/bump-migration`,
**preserving their relative order**, so they now sort after the
cherry-picked migrations and
`20260626120000_CompressWindowsMDMResponsesColumn`. Regenerated
`schema.sql`.

Verified `20260626120000_CompressWindowsMDMResponsesColumn` (the only
other non-cherry-picked migration, already correctly ordered) touches
only `windows_mdm_responses` — none of the 8 moved migrations touch that
table, so no dependency inversion is introduced by the reorder. None of
these 10 migrations shipped in `4.87.1`, so no released database is
affected.

`rc-patch-fleet-v4.88.0` needs no change. This lands on `main` and
should be reflected on `rc-minor-fleet-v4.89.0`.

# Checklist for submitter

## Database migrations

- [x] Migration files renamed via `tools/bump-migration`; function names
updated to match new timestamps.
- [x] Regenerated `schema.sql` via `make dump-test-schema`; migrations
apply cleanly in the new order.
- [x] No schema/content changes to the migrations themselves — timestamp
renumber only.

## Testing

- [x] `go build ./server/datastore/mysql/migrations/...` and `go vet`
pass; schema regeneration ran all migrations successfully in order.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for additional device/profile tracking across Android,
Apple, Windows, certificates, and apps.
* Added new setup and software management records, including support
software categories and team pins.
  * Added a policy-gating flag for setup experience results.

* **Bug Fixes**
* Improved database consistency with stronger uniqueness and
cascade-delete behavior.
* Updated schema tracking so migrations apply cleanly with the latest
database state.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-02 12:00:56 -05:00
Lucas Manuel Rodriguez ad0a39e067 Fix panic in GetClientConfig with null agent options config (#47388) (#48584) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #47388

I'll be doing some separate research on how agent options ends up as
`null` in the first place.
Obviously you can set `config:` in the agent options and hit `Save` and
the issue is reproduced but seems unlikely (one theory is GitOps doing
some overriding).

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Summary

`GetClientConfig` (`server/service/osquery.go`) panicked with
`assignment to entry in nil map` (returning 5XX on
`/api/v1/osquery/config`) when a host's resolved agent options had a
null `config`.

Root cause: `config` is initialized as an empty map, but
`json.Unmarshal([]byte("null"), &config)` silently sets the map to `nil`
(no error). When the host also had packs or scheduled queries, the later
`config["packs"] = ...` assignment panicked.

This adds a nil-guard that re-initializes the map after the unmarshal.

## Testing

- [x] Added/updated automated tests

Added `TestGetClientConfigNullConfig`, which sets `{"config":null}`
agent options plus a pack and asserts no panic/error and that `packs`
still serialize correctly.

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed a server crash that could occur when generating osquery
configuration for hosts with a null agent config.
* Improved config handling so hosts with packs and scheduled queries now
receive their configuration reliably, even when the base config is
empty.
  * Added regression coverage to help prevent this issue from returning.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-02 11:17:04 -03:00
Juan Fernandez 4afd59833d Fix installed_software status in policy automation activities 
Relates to #38670

The policies/:id/automation_activities endpoint derived the top-level
status for installed_software activities from the live
host_software_installs.status generated column. That column becomes NULL
when the install row is marked removed=1 (e.g. after the installer
package is edited/updated or the software is re-installed), so a
historically-successful install was miscategorized as "error".

Derive the outcome from the activity's recorded details.status instead,
which reflects the install result at the time the activity was created.
The install output still comes from host_software_installs. This applies
to both the displayed status and the ?status=error|success filter.

Also fixed alignment with the info icon on the policy automations table.
 2026-07-02 09:59:59 -04:00
Juan Fernandez 7023c5be9a Fix cron jobs stuck in "expired" when a run is interrupted mid-flight 
Fixes #48497

When a cron run's context was cancelled mid-flight (e.g. the instance
received SIGTERM during graceful shutdown), the stats row was left
"pending" because the terminal-status write failed on the cancelled
context. CleanupCronStats would later reap it to "expired", hiding the
fact that the run was interrupted and discarding the captured job
errors.

Record the terminal status on a context detached from cancellation
(context.WithoutCancel with a bounded timeout) so an interrupted run
persists its outcome. The run is marked "canceled" only when the context
was cancelled AND a job actually reported an error, so a run whose jobs
all finished cleanly is still "completed" even if cancellation merely
raced the end of the run.
 2026-07-02 07:38:11 -04:00
Juan Fernandez 013718aacb Fix Redis MOVED errors from query results counts in cluster mode 
Fixes #47303

GetQueryResultsCounts and IncrQueryResultsCounts pipelined commands
across multiple query_results_count:<id> keys on a single connection.
These keys have no hash tag, so in a Redis Cluster they scatter across
hash slots. A pipelined connection binds to the first key's slot, so
every other key returned a MOVED redirect, producing recurring error log
noise on host check-ins. IncrQueryResultsCounts additionally used
ConfigureDoer, whose RetryConn does not support Send, so increments
failed entirely in cluster mode.

Group the keys by hash slot with redis.SplitKeysBySlot and run one
pipeline per slot group, mirroring the existing QueriesForHost and
CleanupInactiveQueries patterns in the same file. The write path uses a
plain pooled connection (not ConfigureDoer) since all keys in a slot
group share a slot and no redirect handling is needed.
 2026-07-02 07:37:42 -04:00
Lucas Manuel Rodriguez c4a66e6303 Update osquery schema and flags to 5.23.1 (#48587) 
osquery [5.23.1](https://github.com/osquery/osquery/releases/tag/5.23.1)
was released by osquery publicly today, this updates our schema with the
changes in it.

- [X] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Updated schema support for certificate `subject2` and `issuer2` fields
on Linux and macOS.
* Documentation generation and download tooling now target osquery
`5.23.1`.

* **Bug Fixes**
* Clarified the `process_open_handles` table behavior by removing
outdated default-process wording.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-01 19:34:00 -03:00
George Karr 80b883a2e7 Adding in check to disable recovery lock on personal macos since it doesn't have the required permissions (#48598) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #48594

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Recovery-lock password checks now skip personally owned (BYOD) Apple
devices, avoiding failures on eligible hosts.
* Recovery-lock clear actions are no longer applied to personally owned
enrollments.

* **Tests**
* Added coverage to verify BYOD devices are excluded from both
recovery-lock enforcement and clear workflows.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-01 16:20:50 -05:00
Victor Lyuboslavsky 7e8b03cd1c Reject unsupported OnPremise Windows MDM enrollment with an actionable message (#46387) (#48300) 2026-07-01 19:21:02 +01:00
Juan Fernandez e8f26ec4ef Fix S3 file carve cleanup hang and rework reconciliation 
Relates to #48549

The S3 carve cleanup (server/datastore/s3, run by the
cleanups_then_aggregation cron) advanced ListObjectsV2 pagination using
the response's ContinuationToken — an echo of the request token —
instead of NextContinuationToken. On any bucket with more than one page
of objects this looped forever, hanging the entire serial cleanup cron
and stalling every cleanup/aggregation job ordered after it.

Replace the bucket-listing reconciliation with a direct HeadObject probe
per carve, which is exact and independent of listing order or object
counts:

- Only carves older than 24h with a completed upload are reconciled
(mirrors the MySQL carve store's floor; skips in-flight multipart
uploads). A carve is expired only on a definitive not-found; transient
or other probe errors leave it for a future run, so a carve whose object
still exists is never expired.
- Probes run with bounded concurrency; expirations are written in one
batched, retryable UPDATE (new ExpireCarves datastore method) rather
than one per carve.
- The number of carves reconciled per run is capped so a large backlog
drains across runs without any single run making unbounded S3 requests.

Add S3-carve-store-only server settings (the MySQL carve store is
unaffected):
- s3.carves_cleanup_disabled       — skip reconciliation entirely
- s3.carves_cleanup_max_per_run    — per-run cap (default 1000)
- s3.carves_cleanup_concurrency    — concurrent probes (default 32)

Also log the expired count per run and fix the test bucket cleanup
helper to paginate. Adds unit tests (transient-error safety, partial
failure, concurrency) and a MySQL integration test for ExpireCarves.
 2026-07-01 14:00:59 -04:00
Jordan Montgomery 6223af892e Fix manual-personal enrollment for iOS/iPadOS (#48534) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

Unreleased bug, no changes file

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [ ] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Personal enrollment status is now preserved and updated correctly when
MDM device records change.
* macOS MDM ingestion now keeps the BYOD/personal enrollment flag for
Fleet devices instead of defaulting it away.
* Incoming server URLs continue to have query parameters removed while
still retaining the enrollment status used for processing.

* **Tests**
* Added coverage for personal enrollment updates and macOS ingestion
scenarios, including BYOD and non-BYOD cases.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-01 09:59:55 -05:00
Lucas Manuel RodriguezandCopilot Autofix powered by AI 2d70a7b500 Associate all matching hosts with a SCIM/IdP user (not just the first) (#48351) 
Resolves https://github.com/fleetdm/fleet/issues/48378 (issue found
while working on the Google Workspace IdP integration).

## Summary

Fixes a bug where an IdP user associated with **multiple hosts** only
had IdP host vitals populated on **one** of them.

`maybeAssociateScimUserWithHostMDMIdP` (called when a SCIM/IdP user is
created) matched all hosts whose MDM IdP account corresponds to the
user, but then deliberately linked only `hostIDs[0]` (with a `// TODO:
confirm desired behavior` / "just use the first one"). So when a user is
created *after* the hosts already enrolled — e.g. a directory sync
creating users for people who each have a laptop and a desktop — only
the first host got a `host_scim_user` row, and therefore only that host
received the user's IdP host vitals and profile-variable resends.

The fix links **every** matching host. `associateHostWithScimUser` is
keyed on `host_id` (`INSERT … ON DUPLICATE KEY UPDATE`) and triggers its
own per-host profile resend, so calling it once per host is safe and
idempotent.

This is shared SCIM linking code, so the fix benefits all IdP sources
(Okta/Entra SCIM as well as the Google Workspace directory sync that
surfaced it). Deletes and updates already handled multiple hosts
correctly; only the initial reverse-link was capped.

## Testing

Added `testScimUserCreateAssociatesAllMatchingHosts`
(`server/datastore/mysql/scim_test.go`): two hosts share one MDM IdP
account, then a SCIM user is created — both hosts must resolve to it via
`ScimUserByHostID`. Fails before the fix (host #2 unlinked), passes
after.

**Related issue:** Resolves #48378

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements).

## Testing

- [x] Added/updated automated tests
- [X] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* SCIM/IdP user provisioning now associates a new SCIM user with **all**
matching hosts, not just the first match.
* Host end-user details (including IdP username/full name) are now
populated consistently on every associated host.
* **Tests**
* Added SCIM integration and datastore regression coverage to ensure
multiple hosts linked to the same IdP account are all associated during
user creation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-01 10:40:31 -03:00
Lucas Manuel Rodriguez bec3b0dc2a Reduce MySQL reader load on GET /hosts with device_mapping + search query (#47722) (#48488) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #47722

The issue was from a customer running `GET
/api/v1/fleet/hosts?device_mapping=true&page=1&per_page=100&query=<ADDRESS>%40example.com`
on a script in a for loop. This change reduces the impact of the API on
such workflows.

Results from my local load test:

EXPLAIN ANALYZE:
```
┌───────────────────────────────────┬────────────────┬─────────────┬─────────────────────────────────────────────┐
│                                   │ optimizer cost │ actual time │         device_mapping aggregation          │
├───────────────────────────────────┼────────────────┼─────────────┼─────────────────────────────────────────────┤
│ Old (derived-table GROUP BY join) │ ~23,179        │ ~73 ms      │ materialized dm derived table, cost ~7,125  │
├───────────────────────────────────┼────────────────┼─────────────┼─────────────────────────────────────────────┤
│ New (correlated subquery)         │ ~1,260         │ ~25 ms      │ Aggregate … loops=1 (only the returned row) │
└───────────────────────────────────┴────────────────┴─────────────┴─────────────────────────────────────────────┘
```
Tests with 10k hosts:
```
┌───────────────────────────────────┬────────────┬───────────────┬───────┐
│              dataset              │ OLD (main) │ NEW (this PR) │ ratio │
├───────────────────────────────────┼────────────┼───────────────┼───────┤
│ 10k hosts × 3 emails (30k rows)   │ 4.6s       │ 1.1s          │ ~4×   │
├───────────────────────────────────┼────────────┼───────────────┼───────┤
│ 10k hosts × 30 emails (300k rows) │ 35.9s      │ 1.2s          │ ~30×  │
└───────────────────────────────────┴────────────┴───────────────┴───────┘
```

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## What & why

`GET
/api/v1/fleet/hosts?device_mapping=true&page=1&per_page=100&query=<email>`
caused high MySQL **reader** load on instances with ~10k hosts. Each
page load ran an expensive aggregation over the entire `host_emails`
table even though only ~100 rows are returned.

**Root cause:** with `device_mapping=true`, `applyHostFilters` added a
`LEFT JOIN` on a derived table with `GROUP BY host_id` over
`host_emails`. Because of the `GROUP BY`, MySQL must fully materialize
that derived table (aggregating every row for all hosts) before the
outer `WHERE`/`LIMIT 100` can be applied, so the full cost is paid on
every page request regardless of result size. `CountHosts` reused the
same options, materializing the aggregation a **second** time per page
load.

**Fixes (both in `server/datastore/mysql/hosts.go`):**

1. Replaced the derived-table join with a correlated subquery in the
`SELECT` list (only when `opt.DeviceMapping`), so it is evaluated only
for the rows actually returned, each as an indexed lookup on
`idx_host_emails_host_id_email`. This matches the existing
`host_additional` pattern in the same query.
2. Set `opt.DeviceMapping = false` in `CountHosts` — the column is never
selected for counting — mirroring the existing `opt.DisableIssues`
handling.

## Notes

- The composite index `idx_host_emails_host_id_email (host_id, email)`
already exists, so the correlated subquery resolves via an indexed
lookup per returned row.
- `TestHosts` (full suite) passes, including `HostDeviceMapping`,
`CustomHostDeviceMapping`, and `IDPHostDeviceMapping` (the last two
verify the `custom_*` → `custom` and `idp` → `mdm_idp_accounts` source
translation still works through the new subquery).
- Recommend validating with `EXPLAIN ANALYZE` on a ~10k-host dataset
before/after, per the issue. I did not have access to such a dataset.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Performance**
* Improved host list responsiveness when using search filters alongside
device mapping.
* Reduced database load during host listing by retrieving device mapping
more efficiently per host.
* Improved host counting speed by avoiding device-mapping evaluation for
count queries.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-01 10:28:30 -03:00
Victor Lyuboslavsky a90eab6f62 Improved GitOps consistency for Windows BatchSetMDMProfiles (#48467) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves
https://github.com/fleetdm/confidential/issues/16293

Test failures are not related to this change. They are currently failing
on main.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **Bug Fixes**
* Improved consistency when applying Windows configuration profiles in
batch by validating against the latest server MDM state.
* Fixed an issue where a temporary “assume enabled” setting could affect
real configuration updates; it now applies only to dry runs.
* Ensured team profile validation uses the freshly persisted server
state during the same GitOps execution.
* Added a regression test covering Windows MDM “assume enabled” behavior
for dry-run vs real runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-06-30 21:33:07 +01:00