Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 34 Packages Projects Releases Wiki Activity
24,124 Commits 2 Branches 0 Tags
af9c488cf61d79ffbdb559642c98671c4ffc8a22
Commit Graph
1596 Commits
Author SHA1 Message Date
Andrew Mellor 119feeda02 42218 updated ios version number to include supplemental extra (#44727) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #42218

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

Note: Sim update included and validated with and without supplemental,
screen shots attached

<img width="760" height="87" alt="Host List"
src="https://github.com/user-attachments/assets/c55f0ace-a205-4242-95da-510e8e6ec4ad"
/>
<img width="1511" height="523" alt="Standard"
src="https://github.com/user-attachments/assets/74a42e57-9391-4ce0-8b0a-ad3de6ab4745"
/>
<img width="1505" height="526" alt="Supplimental"
src="https://github.com/user-attachments/assets/392fc603-c7a2-4d6f-8ae0-87767cab7e3c"
/>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* iOS/iPadOS devices managed via MDM now include reported supplemental
OS version text (e.g., Rapid Security Response suffixes) in the
displayed OS version string.
* **Bug Fixes**
* Supplemental extras are validated; invalid values are ignored.
Combined version strings are length-limited and safely truncated.
* **Tests**
* Added tests for supplemental handling, validation, fallback, and
truncation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 15:16:31 +01:00
Scott Gress 684becade8 Allow disabling chart datasets: backend (#44769) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #44077 

# Details

This PR implements enforcement of the "disable dataset" feature.  

When a dataset is disabled globally, we:

* Stop collecting all data for that dataset (the `Collect` method for
that dataset is not called in the cron job)
* Remove all previously-collected data for the dataset via an
asynchronous job

When a dataset is disabled for one or more fleets, we:

* Provide the list of disabled fleets as an argument to each dataset's
`Collect` method. Each dataset is responsible for filtering out hosts in
the most efficient way possible
* Scrub the data for the relevant datasets using a bitmask, so that all
hosts from the disabled fleets are removed from the data. This is done
via an asynchronous job.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, unreleased

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [X] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [ ] QA'd all new/changed functionality manually

  ### Prerequisites / Test Setup

- [ ] Fleet running with at least 3 teams (call them T1, T2, T3) and ≥3
hosts in each, plus ≥2 hosts with no team
- [ ] At least one host on each team has reported recent uptime (within
the bucket window)
- [ ] At least one host in each team is affected by a tracked CVE (so
`host_scd_data` for `dataset='cve'` will have non-empty bitmaps)
- [ ] AppConfig: both `features.historical_data.uptime` and
`features.historical_data.vulnerabilities` start as `true`; same for
every team
- [ ] Let the collection cron run at least one full tick to populate
baseline rows in `host_scd_data` for both `uptime` and `cve`
- [ ] Note the current row count per dataset: `SELECT dataset, COUNT(*)
FROM host_scd_data GROUP BY dataset;`

  ---

  ### 1. Cron Skips Globally-Disabled Datasets

  #### 1.1 Global disable of `uptime`

- [x] Disable globally: `PATCH /api/v1/fleet/config` with
`features.historical_data.uptime = false`
- [x] Verify activity feed shows `disabled_historical_dataset` for
`uptime` (existing behavior)
- [x] Wait for next collection tick (or trigger it via fleetctl debug if
available)
  - [x] Confirm **no new rows** appear for `dataset='uptime'`:
`SELECT MAX(valid_from) FROM host_scd_data WHERE dataset='uptime';`
        should not advance after the disable
- [x] Confirm cron still writes `cve` rows on the same tick (per-dataset
isolation)
  - [x] Re-enable: PATCH `historical_data.uptime = true`
  - [x] Verify next tick resumes writing `uptime` rows

  #### 1.2 Global disable of `vulnerabilities`

  - [x] Repeat 1.1 with `features.historical_data.vulnerabilities`
  - [x] Confirm `cve` writes stop, `uptime` continues

  #### 1.3 Both disabled globally

  - [x] Disable both globally
  - [x] Confirm cron tick produces zero new rows for either dataset
  - [x] Confirm cron does not error or get stuck
  - [x] Re-enable both

  ---

  ### 2. Per-Fleet Disable — Cron Filters at SQL

  #### 2.1 Single team disabled for one dataset

  - [x] Disable uptime for T1 only: PATCH team T1 with
        `features.historical_data.uptime = false`
- [x] Verify scoped `disabled_historical_dataset` activity emitted for
T1
  - [x] Wait for next cron tick / trigger cron
- [x] Pick a host known to be in T1 (call it `H_T1`); confirm its bit is
NOT set in any `uptime` row written *after* the disable by filtering the
chart to that host
- [x] Pick a host in T2 (`H_T2`); confirm its bit IS still set in the
same rows (T2 is not disabled)
- [x] Pick a no-team host (`H_none`); confirm its bit IS still set
(no-team hosts follow the global value)

  #### 2.2 Same fleet, different dataset

- [x] With T1's uptime disabled, confirm T1's hosts ARE still written
into `cve` rows on subsequent ticks (per-dataset isolation)

  #### 2.3 All teams disabled, global on, no-team hosts

  - [x] Disable uptime on every team (T1, T2, T3)
- [x] Confirm next tick still writes a row containing only no-team
hosts' bits (global is on, no-team hosts always count)
  - [x] Re-enable uptime on all teams

  ---

  ### 3. Global Scrub — DELETE

  #### 3.1 Successful global scrub

  - [x] Note baseline:
        `SELECT COUNT(*) FROM host_scd_data WHERE dataset='uptime';`
        (should be > 5000 to exercise the loop; if not, manually
        insert filler rows or run multiple cron ticks)
  - [x] Disable uptime globally via the API
  - [x] Wait for the worker to pick up the scrub / trigger the job
  - [x] Confirm the count drops to 0:
        `SELECT COUNT(*) FROM host_scd_data WHERE dataset='uptime';`
  - [x] Confirm rows for **other datasets** are untouched
  - [ ] Test again but disable via GitOps

  ---

  ### 4. Per-Fleet Scrub — ANDNOT

  #### 4.1 Single-fleet scrub clears bits

  - [x] Identify hosts in T1 and record their IDs (call this set `S`)
- [x] Pre-disable, confirm at least one `host_scd_data` row for
`dataset='uptime'` has bits set at positions in `S` by filtering the
chart to those hosts
  - [x] Disable uptime on T1 only, via the API
  - [x] Wait for the scrub to run / trigger it
- [x] Confirm: every existing row for `dataset='uptime'` now has NO bits
set at any position in `S`. Spot-check by filtering the chart to those
hosts
- [x] Confirm rows for `dataset='cve'` (different dataset) are untouched
  - [x] Confirm bits for hosts in T2/T3 (not disabled) are still set
  - [x] Run test again but disable via GitOps

  #### 4.2 Multi-fleet scrub via GitOps batch

- [x] Apply a GitOps spec that flips cve to false on T1 and T3 in a
single apply
  - [x] Wait for scrub(s) to complete
- [x] Confirm bits for the union of T1∪T2 hosts are cleared from every
row of `dataset='cve'`
  - [x] Confirm T2 hosts' bits remain set

  ---

  ### 5. Activity Feed Cross-Check

  - [x] Each global flip emits exactly one `disabled_historical_dataset`
        activity (existing behavior, unchanged)
  - [x] Each per-team flip emits one scoped activity with the team's
        ID and name
  - [x] PATCH submitting unchanged values emits **no** activity and
        causes **no** scrub (no `host_scd_data` data change observed
        after the cron tick)
  - [x] No new "scrub completed" or "scrub started" activity is
        emitted (out of scope for v1)
  - [x] Re-enable flips emit `enabled_historical_dataset` activities
        and do NOT emit any scrub-related activity

  ---

  ### 6. Regression Spot Checks

  - [x] With everything enabled (default), the chart UI renders the
        same data as before this change (no behavior change in the
        "all on" case)
  - [x] AppConfig YAML round-trip (`fleetctl apply`) is benign:
        applying the unchanged config produces no scrub jobs and no
        activities
  - [x] GitOps apply with `historical_data` omitted from team specs
        defaults to `true` (per the gitops-api change) and does not
        trigger spurious scrubs
  - [x] After a full disable+scrub of cve, the `host_scd_data` table
        has no `dataset='cve'` rows; the chart UI for "vulnerable
        hosts over time" shows an empty/zero state without errors

  ---


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Chart collection now supports per-dataset scoping and honors
team-level disables; new scrub jobs are registered and worker handlers
added.
* New dataset scrub operations: global and fleet-scoped scrubs; scrubs
can be enqueued and are deduplicated to avoid duplicate pending jobs.
Historical-data changes enqueue scrubs after save (errors logged,
non-blocking).
* **Tests**
* Added unit tests for scope resolution, scrub enqueue/dedup behavior,
scrub workers, scrub application, and low-level blob scrub logic.
* **Documentation**
  * Added OpenSpec metadata for the chart scrub change.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 08:52:35 -05:00
Steven Palmesano 359408b6b9 Fix broken links for setup_experience (#44903) 2026-05-07 06:50:01 -05:00
Jordan Montgomery 4910c450a4 43887 MLAPR backend (#44726) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43887

Adds the password rotation state machine for macOS local admin accounts.
Changes file covered in prior PR

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Automatic macOS managed-local-account password rotation (5‑minute
scheduler) with queued SetAutoAdminPassword device commands
* Manual rotation API: POST /hosts/{id}/managed_local_account/rotate
(returns 204)
* API now reports auto-rotation timing and pending-rotation state
(auto_rotate_at, pending_rotation)
  * Activity records for successful and failed rotations

* **Behavior Changes**
* Password availability is based on stored encrypted password (broader
than before)
  * Rotate-while-in-flight is rejected to prevent duplicate rotations

* **Tests**
* New unit and integration tests for rotation flows, cron behavior, and
failure paths
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 06:36:30 -04:00
Magnus Jensen 9d96d6c76a add script output to GitOps (#44728) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #44082

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Enhanced GitOps script logging: reports how many scripts would be
applied in dry‑run mode or were actually applied, with per-team and
per-fleet breakdowns.

* **Tests**
* Added test coverage validating logging output for both dry‑run and
real execution, ensuring reported script counts and per-team/fleet
messages are accurate.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-07 09:38:25 +02:00
Victor Lyuboslavsky c79d33a3a6 Add support for SAN in Android certificate templates. (#44690) 
2/3rds of this PR is OpenSpec and tests. Use OpenSpec files as a
reference (if needed). They're there to help the review, and not to be a
review surface themselves.

- Backend implementation for `subject_alternative_name` in certificate
templates.
- Includes schema migration, variable expansion, GitOps support.
- Limits SAN types to `DNS`, `EMAIL`, `UPN`, `IP`, and `URI`.

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #41472 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.

## New Fleet configuration settings

- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Android certificate templates support Subject Alternative Name (SAN)
with validation (DNS, EMAIL, UPN, IP, URI), Fleet-variable substitution,
runtime expansion, and delivery; SAN use is gated by Premium license
  * GitOps now validates and includes SAN in Android certificate flows

* **Chores**
  * Database schema updated to store SAN on certificate templates
  * Changelog entry added

* **Tests**
* Added unit and end-to-end tests covering SAN validation, variable
expansion, and GitOps behavior
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-06 14:43:09 -05:00
Victor LyuboslavskyKonstantin SykulevCopilot
 		d38163db94 Setup experience for Windows. (#44306) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43859 

This PR brings the Windows Autopilot setup experience to parity with
macOS DEP. Windows hosts that enroll
through Autopilot now coordinate with Fleet during the OOBE Enrollment
Status Page (ESP), so admin-defined
software installs run while the device is still waiting at the ESP
screen, before the user can sign in. Fleet
holds the device on the ESP until profiles and setup-experience software
all reach a terminal state, then
either releases the device to login or blocks it on a Reset PC failure
screen. A new team-level setting
controls the policy: when enabled, any critical software install failure
during ESP blocks the device with a
software-specific error message; when disabled, the device releases
regardless of install outcomes (best
effort). A pure 3-hour timeout also forces a finalize, with a
timeout-specific error message on the block
screen. The setting is premium-only and rejected when Windows MDM is not
configured.

Beyond the gating itself, the PR adds the supporting machinery:
orbit-driven setup-experience initialization
on Windows so installs are enqueued at the right moment,
defense-in-depth cancellation of pending software
installs (both queue rows and status rows) whenever the device is going
to block or time out, idempotent
re-enrollment cleanup so a device that resets and re-enrolls during ESP
starts from a clean state. Internally, finalize is structured so a
transient failure at any step (cancel, persist, or the state-machine
transition) leaves the device retriable on the next management session
rather than permanently stuck on "Working on it...". The behavior is
exercised by example-based tests, a property-based test that randomly
samples the wait/block/release decision matrix, and manual VM testing
across Autopilot edge cases.

<img width="1184" height="776" alt="image"
src="https://github.com/user-attachments/assets/5e48660d-235d-40bd-80b6-f8591c579279"
/>

# Checklist for submitter

- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Re-enrollment now clears stale setup-experience results and pending
activities so devices aren’t blocked by old work.
* Insert operations tolerate missing enrollments and return clear
not-found behavior.

* **New Features**
* ESP finalization waits for software installation results and can block
or release based on configurable “require all” behavior; blocking
cancels pending steps and shows prioritized error text.
  * Finalization persists batched final commands for consistent retries.
* Orbit config exposes setup-experience notification for pending/active
Windows hosts.

* **Tests**
* Expanded coverage for ESP flows, datastore awaiting-configuration,
Orbit config, and re-enrollment cascades.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-05-06 14:26:58 -05:00
Sharon Katz 9611bb87b7 Improve error message when referencing a bad label for a configuration profile (#44839) 
Closes #39739

## Local reproduction

Reproduced the bug locally by running `fleetctl gitops --dry-run`
against a local Fleet server with a GitOps config that references a
nonexistent label on a configuration profile.

**Setup:**
1. Started a local Fleet server (`fleet serve` against Docker
MySQL/Redis on `https://localhost:8080`, fresh database).
2. Created a minimal `.mobileconfig` profile
(`test-profile.mobileconfig`).
3. Created a `default.yml` that references it under
`controls.macos_settings.custom_settings` with `labels_include_all:
["this-label-does-not-exist"]`.

**Reproduction:**
```bash
fleetctl gitops -f /tmp/repro-39739/default.yml --dry-run
```

**Result (before fix):**
```
[!] Unknown label 'this-label-does-not-exist' is referenced by MDM Profile '/tmp/repro-39739/profiles/test-profile.mobileconfig'
Error: Please create the missing labels, or update your settings to not refer to these labels.
```

Two problems visible:
- Says "MDM Profile" — internal jargon, not the user-facing term
- Shows the full absolute path — noisy and unhelpful

---

## Code changes

**Summary:** Two lines changed in `cmd/fleetctl/fleetctl/gitops.go`
inside the `getLabelUsage()` function. Both fix how configuration
profile label errors are displayed to the user during `fleetctl gitops`
runs.

### `cmd/fleetctl/fleetctl/gitops.go`

**Line 851 — "multiple label keys" error message:**
Changed `"MDM profile"` → `"configuration profile"` and wrapped
`setting.Path` in `filepath.Base()` so the error shows just the filename
instead of the full absolute path.

```diff
- err := fmt.Errorf("MDM profile '%s' has multiple label keys; ...", setting.Path)
+ err := fmt.Errorf("configuration profile '%s' has multiple label keys; ...", filepath.Base(setting.Path))
```

**Line 869 — label usage tracking entry:**
Changed the type string from `"MDM Profile"` → `"configuration profile"`
and the identifier from the full `setting.Path` to
`filepath.Base(setting.Path)`. This feeds into the error message on line
458:
`[!] Unknown label '<name>' is referenced by <type> '<identifier>'`

```diff
- updateLabelUsage(labels, setting.Path, "MDM Profile", result)
+ updateLabelUsage(labels, filepath.Base(setting.Path), "configuration profile", result)
```

**After fix:**
```
[!] Unknown label 'this-label-does-not-exist' is referenced by configuration profile 'test-profile.mobileconfig'
```

---

## Testing

### Manual testing

1. Started a local Fleet server (fresh DB, `fleet serve` on
`https://localhost:8080`).
2. Created a minimal `.mobileconfig` profile and a `default.yml` GitOps
config that references it with `labels_include_all:
["this-label-does-not-exist"]`.
3. Built `fleetctl` from the **unfixed** code (`git stash`) and ran
`fleetctl gitops -f default.yml --dry-run`. Confirmed the old error
message:
   ```
[!] Unknown label 'this-label-does-not-exist' is referenced by MDM
Profile '/tmp/repro-39739/profiles/test-profile.mobileconfig'
   ```
4. Built `fleetctl` from the **fixed** code and ran the same command.
Confirmed the new error message:
   ```
[!] Unknown label 'this-label-does-not-exist' is referenced by
configuration profile 'test-profile.mobileconfig'
   ```

### Unit tests added

New file: `cmd/fleetctl/fleetctl/gitops_label_usage_test.go` — two tests
that exercise `getLabelUsage()` directly (no Redis/MySQL needed):

- **`TestGetLabelUsageProfilePathShortened`**: Creates a `GitOps` config
with a macOS profile using a full absolute path and a nonexistent label.
Asserts the label usage entry has the basename (not the full path) and
the type is `"configuration profile"` (not `"MDM Profile"`).
- **`TestGetLabelUsageMultipleLabelKeysError`**: Creates a config with
both `labels_include_all` and `labels_include_any` on the same profile.
Asserts the error contains `"configuration profile"` and the short
filename, and does **not** contain the directory path.

Both tests were verified to **fail on unfixed code** and **pass on the
fix** via a `git stash` round-trip.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced error messages for MDM configuration profile label validation
to display concise filenames instead of full file paths, improving user
experience.

* **Refactor**
* Updated internal label usage tracking to use configuration profile
base filenames for consistency and clarity.

* **Tests**
* Added test coverage for configuration profile path shortening and
error message validation in label key scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-06 13:08:27 -04:00
Marko LisicaandCarlo e988dd4756 Support VPP apps from non-US App Store regions (#44368) 
**Related issue:** Resolves #43846

---------

Co-authored-by: Carlo <1778532+cdcme@users.noreply.github.com>
 2026-05-06 12:01:07 -04:00
Carlo b9933f45a2 Fix gitops 500 when software title icon bytes are missing (#44735) 
Fixes #43511
 2026-05-05 18:47:16 -04:00
Sharon Katz 1c522097d0 Fix missing GitOps label validation for invalid field combinations (#44410) 
**Related issue:** Closes #34229

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

---

`fleetctl gitops` silently accepted labels with invalid parameter
combinations (e.g. manual labels with query/criteria/platform). Added
per-type field validation in a centralized
`fleet.ValidateLabelMembershipFields` function, called from the GitOps
parser, `ApplyLabelSpecs`, and `NewLabel`.

| Type | Allowed | Now rejects |
|------|---------|-------------|
| `manual` | `name`, `description`, `hosts` | `query`, `criteria`,
`platform` |
| `dynamic` | `name`, `description`, `query`, `platform` | `criteria`,
`hosts`; validates platform value |
| `host_vitals` | `name`, `description`, `criteria` | `query`,
`platform`, `hosts` |

### Automated tests

- `TestLabelInvalidFieldCombinations` in `pkg/spec/gitops_test.go` — 17
sub-tests covering every invalid combination per label type, plus 3
valid happy-path cases.
- `TestNewLabelFieldValidation` in `server/service/labels_test.go` — 4
cases for NewLabel validation.
- `TestApplyLabelSpecsManualLabelNilHosts` — 10 sub-cases for
ApplyLabelSpecs field validation.
- `TestWhenCreatingNewLabelsPlatformIsValidated` — platform validation
across NewLabel and ApplyLabelSpecs.

All existing `pkg/spec` and `server/service` label tests pass.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Labels now reject invalid field combinations for manual, dynamic, and
host_vitals types with clear error responses instead of failing
silently.
* **Tests**
* Added comprehensive tests covering valid and invalid label
configurations across membership types.
* **Documentation**
  * Changelog entry describing the behavioral fix.  
* **Chores**
* Removed an unnecessary platform constraint from a label configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---

### Manual test results

Ran against a local Fleet server with the built binary.

**API - NewLabel (POST /api/latest/fleet/labels)**

| Test | Input | Expected | Result |
|------|-------|----------|--------|
| 1 | manual + platform=darwin | 422, field=`platform` | PASS |
| 2 | dynamic + platform=invalidplatform | 422, field=`platform` | PASS
|
| 3 | dynamic + platform=darwin + query | 200 | PASS |
| 4 | manual (no platform) | 200 | PASS |
| 5 | host_vitals + platform=darwin | 422, field=`platform` | PASS |
| 6 | dynamic + whitespace-only query | 422, field=`query` | PASS |

**API - ApplyLabelSpecs (POST /api/latest/fleet/spec/labels)**

| Test | Input | Expected | Result |
|------|-------|----------|--------|
| 7 | manual + query | 422, field=`query` | PASS |
| 8 | dynamic + hosts | 422, field=`hosts` | PASS |
| 9 | valid dynamic | 200 | PASS |

**Round-trip: get labels --yaml then apply**

| Test | Scenario | Result |
|------|----------|--------|
| 10 | Legacy manual label with platform=darwin in DB | Platform
stripped from YAML, re-apply succeeds — PASS |
| 11 | Dynamic label with platform=darwin | Platform preserved in YAML,
re-apply succeeds — PASS |

**GitOps parser (fleetctl gitops --dry-run)**

| Test | Input | Result |
|------|-------|--------|
| 12 | manual + query + platform + criteria | All 3 errors surfaced at
once — PASS |
| 13 | valid manual label | No validation errors — PASS |
| 14 | dynamic + invalid platform | Error surfaced — PASS |

---

### Code walkthrough

**`server/fleet/labels.go`** — Added
`ValidateLabelMembershipFields(*LabelSpec) *InvalidArgumentError`. This
is the single source of truth for label field validation, returning
field-specific errors (`platform`, `query`, `criteria`, `hosts`). Lives
here because this package defines the label types both callers import.
Also uses `strings.TrimSpace` to reject whitespace-only queries.

**`server/service/labels.go`** — Three changes: (1) Removed the early
blanket platform check from `NewLabel` that ran before the membership
type was known. (2) Added `ValidateLabelMembershipFields` call in
`NewLabel` after type inference, so the API rejects invalid combos at
creation time. (3) Replaced three incomplete inline checks in
`ApplyLabelSpecs` with a single call to the centralized function, using
`err.WithStatus(422)` to preserve field-specific error shape in the API
response.

**`pkg/spec/gitops.go`** — Replaced the inline validation switch and a
standalone `ValidLabelPlatformVariants` check with a call to
`ValidateLabelMembershipFields`. Unwraps the returned errors
individually into `multiError` so all validation problems are reported
to the user at once.

**`cmd/fleetctl/fleetctl/generate_gitops.go`** — Gated platform emission
on `LabelMembershipTypeDynamic` so legacy manual/host_vitals labels with
a stored platform don't produce YAML that fails re-import.

**`cmd/fleetctl/fleetctl/get.go`** — Added `stripMismatchedLabelFields`
which clears type-inappropriate fields (query, platform, criteria,
hosts) per membership type before YAML output. Called in both code
paths: listing all labels and fetching a single label by name. Ensures
the `get labels --yaml` → `apply` round-trip works for legacy data.

**`server/datastore/mysql/labels.go`** — Added missing `l.criteria`
column to `GetLabelSpec` SELECT, matching `GetLabelSpecs`. Without it,
host_vitals labels fetched by name lost their criteria in the YAML
output, causing re-import to fail with the new validation.
 2026-05-05 16:19:41 -04:00
Juan Fernandez 7088dfa32c Add include_all label scope to GitOps and fleetctl (#41566) 
Resolves #41566 

Wires labels_include_all to GitOps and fleetctl for policies and reports.
 2026-05-05 15:13:22 -04:00
Nico f2b2e23b0a GitOps changes for custom org's logo uploads (#44550) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #44333

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests. Also added some integration tests
as a follow-up of the first PR
(https://github.com/fleetdm/fleet/pull/44390).

- [x] QA'd all new/changed functionality manually

#### generate-gitops

- Branched off to main, no URLs set, then ran generate-gitops on this
branch. Deprecated keys gone, new keys present.

<img width="447" height="170" alt="nourls_new"
src="https://github.com/user-attachments/assets/61931615-d61b-44d3-8095-f7a2b9bd8871"
/>

- Branched off to main, set external URLs for both light and dark modes,
then ran generate-gitops on this branch. Deprecated keys gone, new keys
set with the external URLs.

<img width="637" height="471" alt="externalurl_main"
src="https://github.com/user-attachments/assets/c3782756-acc2-4b99-812d-86e145f11ad5"
/>

<img width="459" height="168" alt="externalurl_new"
src="https://github.com/user-attachments/assets/aa2d8825-3c47-40ba-ab91-bb8202afe81a"
/>

- Within this branch, after uploading a custom logo for light mode, ran
generate-gitops. The logo was saved in lib/org_logo/light.webp

<img width="1510" height="639" alt="Screenshot 2026-05-04 at 4 06 59 PM"
src="https://github.com/user-attachments/assets/13318c24-8fa4-4e29-b629-ff723d4afe5a"
/>
<img width="786" height="172" alt="Screenshot 2026-05-04 at 4 07 30 PM"
src="https://github.com/user-attachments/assets/b46bd1df-7dcd-4489-b7da-4cbad77b25b8"
/>


#### gitops

- Applied gitops with two external URLs. Verified in the UI that those
are still present

<img width="944" height="189" alt="Screenshot 2026-05-04 at 7 54 53 AM"
src="https://github.com/user-attachments/assets/a34813ca-beb1-403e-9793-d42cc9c72f8b"
/>
<img width="637" height="259" alt="Screenshot 2026-05-04 at 8 01 04 AM"
src="https://github.com/user-attachments/assets/74c2cd56-ab1d-4ddd-9b8e-22c49e9ae9d5"
/>

- Applied gitops with "" as the URLs to clear them. Verified the default
fleet logo is shown.

<img width="460" height="201" alt="Screenshot 2026-05-04 at 8 15 11 AM"
src="https://github.com/user-attachments/assets/dcbafea3-b4ea-44aa-9045-08c4f5a64e98"
/>
<img width="648" height="269" alt="Screenshot 2026-05-04 at 8 15 50 AM"
src="https://github.com/user-attachments/assets/451a28f9-e929-4b84-93d3-a7dd9afd5eca"
/>

- Applied gitops with a custom logo for light theme, using
**org_logo_path_light_mode**:

<img width="948" height="207" alt="Screenshot 2026-05-04 at 4 10 05 PM"
src="https://github.com/user-attachments/assets/b1418cd4-31cc-4e53-b566-9af11ec21970"
/>
<img width="774" height="168" alt="Screenshot 2026-05-04 at 4 10 35 PM"
src="https://github.com/user-attachments/assets/63f596eb-308f-4122-ad86-e1d718e9b525"
/>



## New Fleet configuration settings

- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- See https://github.com/fleetdm/fleet/pull/43808.
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* GitOps support for uploading custom org logos (dark/light) via local
files.
* `fleetctl generate-gitops` exports Fleet-hosted logos as local files
and inserts path references.
  * New API endpoints to upload, delete, and fetch org logos.

* **Deprecated**
* Legacy logo keys consolidated into mode-specific URL keys
(`org_logo_url_dark_mode`, `org_logo_url_light_mode`).

* **Bug Fixes / Validation**
* Validation/error when both a path and URL are provided for the same
mode; file size and image-format checks enforced.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-05 18:18:08 +02:00
Scott Gress c3b82539a5 Allow disabling historical data collection (GitOps / API) (#44488) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #44077

# Details

* Adds `historical_data` key to app and team config (and gitops) with
`uptime` and `vulnerabilities` subkeys. Keys default to `true`, meaning
"collect this data"
* Adds `enabled_historical_dataset` and `disabled_historical_dataset`
activities when these values are flipped via GitOps or the config APIs

The majority of the file changes in here are GitOps test files that need
to be updated to have the new config in them.

**This PR does _not_ implement using these configs to actually disable
data collection or purge data; that will come in a follow-up PR (as well
as the front-end)**

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, unreleased

## Testing

- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
  #### Defaults
- [X] Fresh install: `GET /api/v1/fleet/config` returns
`features.historical_data.uptime: true` and
`features.historical_data.vulnerabilities: true`
- [X] Created a new fleet via `POST /api/v1/fleet/teams`, then `GET
/api/v1/fleet/fleets/{id}` returns `features.historical_data.uptime:
true` and `features.historical_data.vulnerabilities: true`

  #### Global PATCH (`POST /api/v1/fleet/config`)
- [X] PATCHed `{"features": {"historical_data": {"vulnerabilities":
false}}}` — `vulnerabilities` flipped to `false`, `uptime` unchanged at
`true`
- [X] PATCHed `{"features": {"historical_data": {"uptime": false,
"vulnerabilities": true}}}` — both values applied as sent
- [X] PATCHed `{"features": {"historical_data": {"vulnerabilites":
false}}}` (typo in sub-key) — request rejected with 4xx, stored config
unchanged

  #### Fleet PATCH (`PATCH /api/v1/fleet/fleets/{id}`)
- [X] PATCHed a fleet with `{"features": {"historical_data": {"uptime":
false}}}` — fleet's `uptime` flipped to `false`, `vulnerabilities`
unchanged
- [X] Subsequent `GET /api/v1/fleet/fleets/{id}` returns the toggled
values under `features.historical_data` (storage shape is symmetric with
global)
- [X] PATCHed a fleet with `{"features": {"enable_host_users": false}}`
(a non-`historical_data` features sub-field) — request returned 200 but
the fleet's `enable_host_users` is unchanged (silently ignored, per
existing endpoint convention)

  #### GitOps — global (`fleetctl gitops -f global.yml`)
- [X] Applied a YAML with `features.historical_data: {uptime: true,
vulnerabilities: false}` — `vulnerabilities` is `false` after apply,
`uptime` is `true`
- [X] Applied a YAML whose `org_settings` omits `features` entirely —
both sub-keys are `true` after apply (defaults injected even if
previously disabled)
- [X] Applied a YAML where `historical_data` only contains `uptime:
false` — `uptime: false` is honored, `vulnerabilities` defaults to
`true`
- [X] Disabled `vulnerabilities` via the API, then ran `fleetctl gitops`
with a YAML that doesn't pin it — `vulnerabilities` flips back to `true`
(this is intentional; gitops is the source of truth)

  #### GitOps — fleet
- [X] Applied a fleet YAML with `features.historical_data: {uptime:
false}` — that fleet has `uptime: false`, `vulnerabilities: true` after
apply
- [X] Applied a fleet YAML whose `team_settings.features` omits
`historical_data` — both sub-keys are `true` after apply
- [X] Applied a fleet YAML that omits `features` entirely — both
sub-keys are `true` after apply

  #### `fleetctl apply` (legacy, partial-merge)
- [ ] Disabled `vulnerabilities` via the API, then ran `fleetctl apply`
with a YAML that doesn't mention `historical_data` — `vulnerabilities`
is still `false` (apply leaves omitted fields alone)

  #### Activities — global
- [X] After PATCHing global to disable `vulnerabilities`, the latest
activity is `disabled_historical_dataset` with payload `{"dataset":
"vulnerabilities", "fleet_id": null, "fleet_name": null}`
- [X] After PATCHing global with both sub-keys flipping in one request,
two activities are emitted (one per sub-key)
- [X] After PATCHing global with the same values that are already
stored, zero new activities are emitted
- [X] After re-enabling a previously disabled dataset, the activity type
is `enabled_historical_dataset`

  #### Activities — per fleet
- [X] After PATCHing fleet `workstations` to disable `uptime`, the
activity is `disabled_historical_dataset` with payload `{"dataset":
"uptime", "fleet_id": <workstations id>, "fleet_name": "workstations"}`
- [X] Toggling the same dataset on two different fleets produces two
distinct activities, one per fleet
- [X] After a fleet PATCH with the same values already stored, zero new
activities are emitted

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed

## New Fleet configuration settings

- [ ] Setting(s) is/are explicitly excluded from GitOps

If you didn't check the box above, follow this checklist for
GitOps-enabled settings:

- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
  - https://github.com/fleetdm/fleet/pull/44703
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [X] Verified that any relevant UI is disabled when GitOps mode is
enabled


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Historical-data controls: per-org and per-team toggles for uptime and
vulnerability time‑series, with defaults applied when keys are omitted
and enable/disable activities emitted on changes.

* **Bug Fixes**
* Partial updates and PATCH/GitOps flows preserve unspecified
historical-data sub-keys instead of clearing them.

* **Tests**
* Expanded unit and integration tests covering defaults, partial
PATCH/GitOps behavior, idempotency, and activity emission.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-05 11:12:23 -05:00
Nico b4a207fb5a Add ability to upload custom org logos (#44390) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #44330, Resolves #44331

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests. (I'd defer integration tests to a
separate PR since this one is pretty large already.)

- [x] QA'd all new/changed functionality manually. I've tested this on
both the setup flow and the organization settings page. I haven't had
the time to test this on other places where we render the logo (macOS
setup experience / MDM migration dialog).


https://github.com/user-attachments/assets/95d4eae5-3da6-40f4-98a1-8575b97d96b3

## New Fleet configuration settings

- [x] Setting(s) is/are explicitly excluded from GitOps.

Will handle GitOps in a separate PR.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Organizations can upload custom logos for light and dark modes.
* Registration and Org Settings support logo file upload, preview,
per-mode replace/delete, and validation (size & image formats).
* Activity feed records logo changes/deletions; site nav displays
uploaded logos per theme.
* File uploader/preview adds a Fleet logo graphic option and improved
logo validation.
  * Config/GitOps outputs now include separate dark/light logo fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-05 14:42:52 +02:00
Sharon Katz beca71e674 Fix gitops dry-run to catch manual_agent_install + macos_script conflict (#44432) 
**Related issue:** Resolves #34464

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

---

## What

GitOps `--dry-run` was succeeding when `macos_manual_agent_install` was
set to `true` and a `macos_script` was configured under
`setup_experience`, but the actual GitOps run would fail with:

```
Couldn't add setup experience script. To add script, first disable macos_manual_agent_install.
```

## Why

The `manual_agent_install` conflict validation only existed server-side
in `ee/server/service/setup_experience.go:SetSetupExperienceScript()`.
The script upload call (`uploadMacOSSetupScript()`) was gated by
`!opts.DryRun` in `server/service/client.go`, so during dry-run the
upload was skipped entirely and the validation never fired.

## Fix

Added client-side validation in `server/service/client.go` at the point
where the YAML-parsed `MacOSSetup` struct is processed — before the
script file is validated and loaded. This check runs for **both dry-run
and real runs**, catching the conflict early. Two code paths were fixed:

1. **Team path** (~line 803): Checks `setup.ManualAgentInstall.Value`
when `setup.Script.Value` is set
2. **No-team path** (~line 2603): Checks
`macOSSetup.ManualAgentInstall.Value` when `macOSSetup.Script.Value` is
set

## How I reproduced the issue locally

### Prerequisites
- MySQL and Redis running via Docker: `docker compose up -d mysql_test
redis`

### Steps
1. Wrote an integration test
(`TestDryRunMacOSSetupScriptWithManualAgentInstallConflict`) that:
   - Creates a GitOps user and fleetctl config
   - Creates a bootstrap package server serving `testdata/signed.pkg`
   - Creates a `.sh` script file with `echo "setup script"`
   - Creates a **global config** YAML (minimal server settings)
- Creates a **team config** YAML with `macos_manual_agent_install:
true`, `macos_script: <path>`, and `macos_bootstrap_package: <url>`
   - Runs `fleetctl gitops --dry-run` and asserts it fails
   - Runs `fleetctl gitops` (no dry-run) and asserts it fails

2. Ran the test **before the fix** — confirmed the bug:
   ```
Dry-run error: <nil> ← BUG: should have failed
Real run error: ...status 422...first disable macos_manual_agent_install
← correctly fails
   ```

3. Applied the fix and re-ran — **both dry-run and real run now fail**
with the `macos_manual_agent_install` conflict error.

### Test command
```bash
MYSQL_TEST=1 REDIS_TEST=1 go test -v \
  -run TestIntegrationsEnterpriseGitops/TestDryRunMacOSSetupScriptWithManualAgentInstallConflict \
  ./cmd/fleetctl/integrationtest/gitops/... -count=1 -timeout 600s
```

Both sub-tests (team and no-team paths) pass. All related existing tests
continue to pass:
- `TestMacOSSetup`, `TestMacOSSetupScriptWithFleetSecret`,
`TestDeletingNoTeamYAML`, `TestDisallowSoftwareSetupExperience`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* GitOps dry-run now correctly fails when a macOS setup configuration
combines manual agent installation with a provided setup script,
preventing false-positive dry-run success.

* **Tests**
* Added unit and integration regression tests to verify dry-run and
real-run rejection of conflicting macOS setup configurations for both
team-scoped and unassigned host scopes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-04 15:32:21 -04:00
Konstantin Sykulevcopilot-swe-agent[bot]ksykulev
 		5c18f726b2 Add Info logging around vulnerability scanning phases (#44653) 
related to [#44391](https://github.com/fleetdm/fleet/issues/44391)

This will add 10 info level log statements during vulnerability
scanning.
Example:
`ts=2026-05-03T18:32:26Z level=info msg="phase completed"
cron=vulnerabilities phase=nvd elapsed=59.450125s`


## Testing

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Improvements**
* Vulnerability scanning now logs overall elapsed time and per-phase
durations so operators can see how long full scans and each scanner
phase take.

* **Chores**
* Repository ignore settings updated to exclude an additional path
(non-functional housekeeping).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ksykulev <230639+ksykulev@users.noreply.github.com>
 2026-05-04 14:17:23 -05:00
Konstantin SykulevandCopilot 779cdd663b Periodic background job to cleanup Windows MDM command queue (#44458) 
**Related issue:** Resolves #44190

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a periodic cleanup job that removes aged, acknowledged Windows
MDM command-queue entries to reduce write pressure during ACK
processing.

* **Bug Fixes**
* Pending-command detection now excludes already-ACKed commands from
dispatch; queue rows are retained after ACK and cleaned later.

* **Tests**
* Added and updated tests to validate cleanup behavior and revised
ACK/queue semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-05-04 11:32:45 -05:00
Juan Fernandez 2ee5404ed3 Validate label platform during gitops --dry-run (#42477) (#44594) 
Resolves #42477 

Move the platform check into pkg/spec parseLabels so both --dry-run and
apply hit the same validation and surface the same error.
 2026-05-04 07:18:07 -04:00
Juan Fernandez 376f602088 Fixed bug with about to expire CLI banner (#34924) 
Resolves #34924

Updated the message shown on about to expire license to point to
https://fleetdm.com/learn-more-about/downgrading.
 2026-05-04 07:17:08 -04:00
Noah Talerman 7cd2a1a34a fleetctl preview: Clarify that this is for trying Fleet (#44133) 
- Add link to deploy Fleet for long-lived instances (shown in `fleetctl
preview` Description field)
- Add helpful message after preview setup completes: "Use the stop and
reset subcommands to manage the server and dependencies once started."
 2026-05-01 15:47:59 -04:00
Carlo ce5640c99e Prevent silent corruption of software title icons (#44540) 2026-05-01 14:19:48 -04:00
Noah Talerman 8755a4dddc fleetctl new: Update automatic enrollment profile (#44441) 
- @noahtalerman: We're updating the default profile for new Fleet
instances as part of this story:
  - https://github.com/fleetdm/fleet/issues/40905


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Configuration Updates**
* Updated the default macOS enrollment profile name to "Fleet default
enrollment profile."
* Enrollment onboarding now shows the full setup flow (no setup items
are auto-skipped).
  * Removed region-specific configuration constraints.

* **Behavioral Changes**
  * MDM profile can be removed after enrollment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-01 11:18:58 -06:00
Victor Lyuboslavsky de86536f42 Redis-backed cache for host-by-key lookups (#43936) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43928 

This PR adds a Redis-backed cache in front of the two host-by-key
lookups on the agent auth paths.

Docs: https://github.com/fleetdm/fleet/pull/44504

## What changes

**Read path (osquery/orbit auth):**

- `LoadHostByNodeKey` and `LoadHostByOrbitNodeKey` now check Redis
before falling through to MySQL.
- Successful lookups are cached for 60s ± 10% jitter (configurable via
`FLEET_REDIS_HOST_CACHE_TTL`).
- `NotFound` results are cached for 5s as a negative entry, dampening
repeated probes for keys that
do not exist (deleted hosts whose agents are still polling, attacker
scans, retry storms).
- Concurrent lookups for the same key collapse into one DB query via
`singleflight`. The shared
query runs under a context detached from any one caller's deadline so
the leader giving up does
not abort the work for joiners. The shared query is itself bounded by a
30s timeout so a wedged
  DB call cannot pin the singleflight slot indefinitely.

**Write path (invalidations):**

- These methods now invalidate the cache after a successful inner call:
`UpdateHost`, `SerialUpdateHost`, `UpdateHostOsqueryIntervals`,
`UpdateHostRefetchRequested`,
`UpdateHostRefetchCriticalQueriesUntil`,
`UpdateHostIdentityCertHostIDBySerial`, `EnrollOsquery`,
`EnrollOrbit`, `NewHost`, `DeleteHost`, `DeleteHosts`,
`CleanupExpiredHosts`,
  `CleanupIncomingHosts`, `AddHostsToTeam`.
- `AddHostsToTeam`, `DeleteHosts`, `CleanupExpiredHosts`, and
`CleanupIncomingHosts` use a pipelined
batch invalidator so 10k-host operations stay in the millisecond range
instead of taking minutes
  of sequential round-trips.
- Inner-call errors are not invalidations: a failing write leaves cached
state intact.

**Configuration:**

- New flags `FLEET_REDIS_HOST_CACHE_ENABLED` (default `true`) and
`FLEET_REDIS_HOST_CACHE_TTL`
  (default `60s`).
- Server refuses to start if the cache is enabled with `TTL <= 0`.

**Observability:**

- Three new OTEL counters under the `fleet` meter:
  - `fleet.host_cache.lookups{result=hit|negative_hit|miss}`
  - `fleet.host_cache.errors{op=get|set|del}`
-
`fleet.host_cache.invalidations{reason=update|enroll|team|delete|cert}`
- A pre-built SigNoz dashboard ships in
`tools/signoz/host_cache_dashboard.json`.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Optional Redis-backed host lookup cache for osquery and orbit auth,
with automatic invalidation and metrics/monitoring dashboard.

* **Bug Fixes**
* Fixed host-removal batching so cache-related removals use correct
chunks.

* **Tests**
* Added comprehensive host-cache unit tests covering hits, negative
cache, invalidation, concurrency, and JSON round-trips.

* **Chores**
* New config flags to enable the cache and set TTL (default 60s ±10%
jitter).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-01 12:06:16 -05:00
Lucas Manuel Rodriguez 1e4a9f292f Add activities for user actions on labels (#44522) 
Resolves #36976

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Label operations (create, edit, delete) now generate activities shown
in the activity feed with label and optional fleet context.
* Host label add/remove operations emit corresponding label edited
activities; duplicate label names are deduplicated.
* Label activity types are selectable/filterable in the activity
dashboard.

* **Tests**
* Added unit, integration, and UI tests covering label activity
emission, rendering, filtering, and GitOps label lifecycle scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-05-01 10:19:45 -03:00
Tim Lee 13cec63bd1 Add RHEL 8/9/10 simulated hosts to osquery-perf (#44453) 2026-04-30 11:01:04 -06:00
Tim Lee 2d586cb2ff fleetctl vulnerability-data-stream to download OSV data (#44260) 2026-04-30 10:46:51 -06:00
Carlo 07e4e7afe6 Short-circuit for empty software config in Gitops dry run (#44405) 
Fixes #42607
 2026-04-29 14:54:49 -04:00
Magnus Jensen 98cad56716 redirect to correct URL, and allow both URLs for MDM SSO SAML validation if set (#44156) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #41592 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Fixed SSO failures when a custom Apple MDM URL is configured: callback
requests are now redirected to the configured MDM URL when needed, and
SAML validation correctly considers the configured MDM/server URLs so
authentication succeeds for custom MDM setups.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-29 08:43:58 -06:00
Scott Gress 4334017b38 Add Vulnerabilities exposure dataset (#44124) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #43769

# Details

Adds methods to collect data for the `cve` dataset. As with all sets
this is collected at hourly granularity, but unlike the `uptime` set,
the `cve` set uses the "snapshot" strategy so that we record at most one
change (the most recent) per hour.

For this first iteration, we are _recording_ data for all CVEs (i.e.,
which hosts were exposed to which CVEs at a given time), but we are only
_reporting_ a subset of CVEs for the dashboard chart. See [this
comment](https://github.com/fleetdm/fleet/pull/44124#discussion_r3155554405)
for more info.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
- [X] Spot-checked the CVEs chosen by the `trackedCVESoftwareMatchers`
and didn't find any outside of the expected
- [X] With [front-end PR](https://github.com/fleetdm/fleet/pull/44261),
generated chart:
<img width="706" height="421" alt="image"
src="https://github.com/user-attachments/assets/539d9877-6573-4406-a159-1d2a711a045f"
/>



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Host vulnerability (CVE) chart added to the dashboard; CVE chart data
collection is now active.
  * Critical CVE tracking surfaces high-severity vulnerabilities.

* **Improvements**
* CVE chart refreshes every 3 hours (was daily) for more timely
insights.
* Snapshot collection reconciles and closes prior data during empty runs
to keep charts accurate.
* CVE queries may produce zero datapoints when no tracked CVEs exist,
without affecting other metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-29 09:30:31 -05:00
Lucas Manuel Rodriguez 3cadfa1714 Fix issue with fleet's docker image in k8s environments (#44373) 
Resolves #44298

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [X] QA'd all new/changed functionality manually

1. Running the docker image pushed by this PR with the user 3333 doesn't
fail anymore:
```sh
docker run --platform linux/amd64 -it --user 3333:3333 fleetdm/fleet@sha256:1a06bcae25e13e37f871378c7c156f5a2cdf67bc3c3e3bcdc95b6afc0c6decbb
[...]
ts=2026-04-29T13:25:56Z level=warn msg="could not connect to db" err="dial tcp [::1]:3306: connect: connection refused" sleep_interval=0s
[...]
```
4.84.0 fails with:
```sh
docker run --platform linux/amd64 -it --user 3333:3333 fleetdm/fleet:v4.84.0@sha256:51b56ad59a840b28e074ff9b06d6d5b232b0ca2f0d999bb164820da69c7cbe15

Failed to fetch user info for home directory: user: unknown userid 33332026/04/29 13:28:08 71 <nil>
```
2. `strings ./build/fleet | rg github.com/AbGuthrie/goquery/v2` returns
nothing in this branch and returns plenty of matches in `main`.
3. Smoke tested `fleetctl goquery` functionality.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Resolved Docker image startup failures in Kubernetes environments
caused by a dependency side effect.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-29 11:19:13 -03:00
Martin Angers 2c609ae78e CSAH: appconfig/gitops/DB migration to add preserve_host_activities_on_reenrollment field (#44212) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43943 

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

See
https://github.com/fleetdm/fleet/issues/43943#issuecomment-4329658412

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.

## New Fleet configuration settings

- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
(see https://github.com/fleetdm/fleet/pull/43877/changes)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled (should be done by
https://github.com/fleetdm/fleet/issues/43947)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a configuration option to preserve host activities during host
re-enrollment, letting admins choose whether activity history is
retained when hosts re-enroll.

* **Chores**
* Updated defaults and database migration state so the new setting is
present in stored and generated configs and in GitOps outputs.

* **Tests**
* Added unit, integration, migration, and GitOps fixtures to validate
behavior, serialization, and upgrade semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-28 08:47:38 -04:00
Scott Gress 9ae4373f89 Don't ignore GitOps secrets on free tier (#44148) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #44118

# Details

On free tier, ignore exceptions and always apply enroll secrets when
present.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, unreleased

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
@AndreyKizimenko QA'd manually

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed GitOps to correctly apply enrollment secrets and labels on free
tier licenses, even when exception flags are configured.

* **Tests**
* Added tests validating that GitOps properly applies secrets and labels
for free tier customers.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-27 16:31:58 -05:00
Lucas Manuel Rodriguez bd18bac797 Adding gitOpsModeEnabled and gitOpsModeExceptions to anonymous statistics payload (#44161) 
**Related issue:** Resolves #42240.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Statistics now include GitOps mode: whether it’s enabled and the
ordered list of configured exception categories (serializes as an empty
list when none).

* **Tests**
* Added tests for GitOps-related statistics transitions and made
statistics-timing tests deterministic for reliable behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-27 08:28:49 -03:00
Scott Gress 9b01710a80 Don't throw gitops-exceptions-related errors on Free tier (#44118) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #44098 

# Details

We set the "secrets" exception on for all new instances (and the label
exception for existing instances), but you can't turn them off in the
free tier. That means GitOps runs (including the one we use to
initialize new instances) would fail with the "you can't use this key
because the exception is on" error. This PR fixes the issue by not
enforcing that rule for free tier instances.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, unreleased

## Testing

- [X] Added/updated automated tests
- [X] added test verifying that the free tier can run gitops using
excepted keys w/out error, and verified that it fails on main and passes
on this branch
- [X] QA'd all new/changed functionality manually
  - [X] spun up a new free-tier server successfully

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* GitOps exception enforcement no longer blocks free-tier users;
enforcement is applied only for premium licenses, allowing GitOps
applies on free tiers.
* **Tests**
* Added an integration test validating free-tier GitOps behavior to
prevent regressions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-24 10:46:45 -05:00
Juan Fernandez 1539c6b094 Enforce consistent fleet name uniqueness across UI and GitOps (#33557) 
Resolves #33557 

The tems.name column uses utf8mb4_unicode_ci, so names like "ABC" and
"abc" compare as equal at the database level. Before this change name
collisions were handled in different ways in the UI and in GitOps.

The changes introduced here, consolidates the logic used for detecting
name collisions in all code path. All conflicts return 409 with the
canonical copy "Fleet names must differ by at least one non-special
character (case-insensitive).
 2026-04-23 16:44:09 -04:00
Jonathan Katz ba0f6b3c72 Update GitOps for managed local account fields (#44058) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #42948 
- Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup)
SetDefaultsIfNeeded()` to account for new fields
- Updated default creation and editing for team edit/creation paths
- Updated `generate-gitops` warning message from `macos_setup` to
`setup_experience`
- Updated fields types to optjson and updated test files

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually
- Team edit and team creation through GitOps, validated config with `
curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'`
- New error message says `setup_experience` instead of `macos_setup`


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added macOS MDM settings to control local account behavior: enable
managed local accounts (default false) and specify end-user local
account type (default "admin") for fleet and team configs. GitOps output
now highlights unsupported setup-experience cases.

* **Tests**
* Updated fixtures and integration tests to assert and persist the new
macOS local-account settings across config, team, and GitOps scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-23 15:05:27 -04:00
Scott Gress 28908e6083 Dashboard charts backend (#43910) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #42812 

# Details

This PR implements a new bounded context, `chart`, with a single
endpoint `/charts`. The context encompasses a framework for recording
and querying and aggregating historical data for Fleet hosts, and
returning that data via the API for the purpose of charting.

This initial iteration has a full implementation of a dataset called
"uptime" which captures which hosts were online hour-by-hour (online
meaning, having been "seen" at some point during that hour). It has a
partial implementation of a "cve" dataset which will capture which hosts
were vulnerable to which CVEs during a given day.

### Data storage

Data is stored in an SCD (slowly-changing dimension) format in the
`host_scd_data` table, where the main "value" in a row is stored in the
`host_bitmap` column, which is a `mediumblob` where each bit encodes a
host ID (bit one represents host ID 1, bit 1444 represents host ID 1444,
etc.). The set of bits set on a row represents that hosts for which that
dataset is "on" during a given time period represented by the
`valid_from` (inclusive) and `valid_to` (exclusive) dates, where a
`valid_to` can have the special "sentinel" value 9999-12-31T00:00:00.000
meaning that the row is still "open" (the value represents everything
from `valid_from` to the present). Additionally an `entity_id` column
can be used for datasets with multiple dimensions, e.g. CVE exposure or
software usage which would have entity IDs representing CVEs or software
items respectively.

### Data collection

Data is collected via a cron job that runs every 10 minutes. Each
dataset has its own `Collect` method which will sample the data for the
given moment. For example the "uptime" dataset gathers the set of hosts
that are online at the moment, and the "cve" dataset will gather the set
of hosts that are vulnerable to each CVE at that moment. The sample can
then be recorded using one of two strategies:

* `accumulate`: bitwise OR the sample with any data already recorded for
the current hour, or add a new pre-closed row for that hour.
* `snapshot`: if there is no open row, create one with the sample and
`valid_to set` to the sentinel. Otherwise:
  * If the sample has the same value as the current open row, do nothing
* If the sample has a different value and the current open row's
`valid_from` is within the same hour, update the current row's value
* If the sample has a different value and the current open row's
`valid_from` is not within the same hour, close the current open row and
start a new one with `valid_from` = the start of the current hour

### Data retrieval 

1. Gets the set of host IDs to retrieve data for. This starts with the
set of host IDs in the requested fleet (or all the hosts a user has
access to if no `fleet_id` param was passed to the `/charts` endpoint),
and further whittled down by any filter options supplied with the
request (labels, platforms, etc.).
2. Finds all `host_scd_data` rows for the requested dataset and date
range (i.e. all rows whose `valid_from` is < the date range end and
`valid_to` is > the date range start).
3. Calculates the date ranges of the "buckets" to return datapoints for.
For the uptime chart we default to 3-hour buckets, so we want 8 buckets
per day.
4. Iterates over each bucket and finds the row or rows from
host_scd_data that cover that bucket range. For datasets using the
"accumulate" strategy, the values for those rows are ORed together. For
"snapshot"s, we take the one active at the bucket end time to represent
the bucket (e.g. "which hosts had a given CVE at the end of the day")

### Tools

This PR includes two dev tools that don't require deep review:

* **chart-backfill** - used to backfill data to various datasets for
testing
* **charts-collect** - used to collect data from a live server via the
API and put into a local hosts_scd_data table

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
  - With [front-end branch](https://github.com/fleetdm/fleet/pull/43878)
<img width="712" height="434" alt="image"
src="https://github.com/user-attachments/assets/b2ccce49-b5fd-4076-b47f-0eea6a53260c"
/>

## Database migrations

- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added charting bounded context: HTTP API for metrics (uptime, CVE),
dataset registry, hosted dataset collection, background
collection/cleanup with opt-out env.
* New utilities: host bitmap operations and string-list/uint-list
parsers.
  * New CLI tools to collect and backfill chart data.

* **Database**
  * Migration and schema to store host time-series SCD chart data.

* **Tests**
* Extensive unit and integration tests for service, storage, caching,
cron, and utilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-23 12:43:23 -05:00
Martin Angers 5da912a33e Bugfix: escape characters not supported in JSON when resolving variables (#43955) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #38013 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

See
https://drive.google.com/file/d/1zeFNLuf_rT5FWzDiYyL2_hbIBW2neba-/view?usp=drive_link

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* GitOps variables in JSON configuration profiles (Apple DDM
declarations and Android profiles) are now automatically escaped for
JSON special characters, ensuring proper handling of sensitive values.

* **Tests**
* Added JSON configuration profile escaping validation to the enterprise
GitOps integration test suite.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-23 10:34:12 -06:00
Tim Lee 79da2f0028 Add RHEL OSV vulnerability scanning (#43377) 2026-04-22 15:34:23 -06:00
Tim Lee 9feb9c2be0 Fix Recovery Lock password desync on MDM re-enrollment (#43827) 2026-04-22 15:17:48 -06:00
Victor Lyuboslavsky d79dc50883 Remove Win MDM "Status on a Status" from osquery perf (#43940) 
Estimate ~5% load improvement.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed MDM command handling in the performance testing agent to
properly skip duplicate status responses.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-22 10:42:31 -05:00
Carloandjkatz01 39e4f616ea macOS managed local account foundations (#43381) 
Implements both #42942 and #42943 

Co-authored-by: jkatz01 <yehonatankatz@gmail.com>
 2026-04-22 09:05:07 -04:00
Scott Gress 91d9b25924 Allow conditional downloads across fleets (#43679) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43417

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
done in https://github.com/fleetdm/fleet/pull/42216

## Testing

- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- Using a local fileserver, added the same software to two fleets and
ran `fleetctl gitops`. Verified that the first fleet downloaded the
file, the second fleet used the cache, and both fleet showed the
software installer in the UI.

## Summary by CodeRabbit

* **Chores**
* Updated software installer lookup mechanism to support optional
team-scoped searches, enabling fallback to cross-team installer cache
when team-specific installers are unavailable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-21 17:06:23 -05:00
Konstantin Sykulev ba7720f7de Use UTC time for osv processing (#43889) 
**Related issue:** Resolves #39900

## Testing
- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved timestamp handling for OS vulnerability data synchronization
to use UTC timezone when synchronization is enabled, ensuring consistent
timing behavior across different system configurations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-21 14:42:16 -05:00
Tim Lee e0d7e0c6b8 Add RHEL support to osv-processor (#43277) 2026-04-21 13:39:22 -06:00
Martin Angers ade597b5e1 DDMV: improve osquery-perf simulation of DDM traffic (#43607) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43050 

## Testing

- [x] QA'd all new/changed functionality manually

Confirmed the traffic sequence with ngrok and DDM stats are as expected
in the osquery-perf logs:

### Adding a new DDM

* `DeclarativeManagement` command and Ack
* `tokens` request
* `declaration-items` request
* `activation` for the DDM
* `configuration` for the DDM
* `tokens` request confirms changes settled
* `status` request

### Remove/re-add DDM (no global change)

* `DeclarativeManagement` command and Ack
* `tokens` request

### Adding a second DDM

* `DeclarativeManagement` command and Ack
* `tokens` request
* `declaration-items` request
* `activation` for the new DDM only
* `configuration` for the new DDM only
* `tokens` request confirms changes settled
* `status` request

### Removing a DDM

* `DeclarativeManagement` command and Ack
* `tokens` request
* `declaration-items` request
* `tokens` request
* `status` request

### Remove all DDMs

* `DeclarativeManagement` command and Ack
* `tokens` request
* `declaration-items` request
* `tokens` request
* `status` request

---

State correctly updates on the host's profiles:

<img width="1246" height="512" alt="image"
src="https://github.com/user-attachments/assets/0d289d4e-1e9b-4283-aef0-fd1ab3ecb355"
/>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Improved macOS Declarative Management sync: faster convergence,
fetches only changed declarations, detects removals, and sends
consolidated status updates.

* **Monitoring**
* Added metrics to track declaration token fetch success and error
rates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-21 13:26:46 -04:00
Juan Fernandez 2b35eabd5d Added middleware for api-only users auth (#43772) 
Fixes #42885

Added new middleware (APIOnlyEndpointCheck) that enforces 403 for
API-only users whose request either isn't in the API endpoint catalog or
falls outside their configured per-user endpoint restrictions.
 2026-04-21 07:11:33 -04:00
Tim LeeandKonstantin Sykulev 81ea7436c3 Fix OSV sync shallow clone failing on quiet weekends (#43450) 
## Summary

The nightly OSV artifact generation in `fleetdm/vulnerabilities` failed
over the weekend with:

```
fatal: error processing shallow info: 4
```

at `cmd/osv-processor/sync-and-detect-changes.sh` during:

```bash
git fetch --shallow-since="3 days ago" origin main
```

Root cause: `git fetch --shallow-since` errors out when the upstream
(`canonical/ubuntu-security-notices`) has zero commits newer than the
cutoff. Canonical didn't push anything over the weekend, so the 3-day
window returned empty and upload-pack produced an unusable shallow
response.

Fix:
- Fall back to `git fetch --depth=3` if `--shallow-since` still returns
empty, so the initial clone always succeeds.

Subsequent runs reuse the existing clone and take the other branch of
the script (plain `git fetch origin main`), which doesn't have this
failure mode.

Failing run:
https://github.com/fleetdm/vulnerabilities/actions/runs/24330589309/job/71035337352

## Test plan

- [x] Re-run the Ubuntu OSV artifact generation workflow; initial clone
succeeds regardless of upstream push frequency.
- [x] Manually exercise the cold-cache path locally: `rm -rf
ubuntu-security-notices &&
./cmd/osv-processor/sync-and-detect-changes.sh` — completes without
error.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved initial repository sync: if the primary shallow fetch returns
no commits, the process now falls back to a limited-depth fetch, warns
the user, and shows recent commit history before continuing. Downstream
change detection and existing behavior for already-cloned repos remain
unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
 2026-04-20 10:30:55 -05:00
Martin Angers 2a8803884b DDMV: Support Fleet variables in DDM (#43222) 
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #43047 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually
See
https://github.com/fleetdm/fleet/issues/42960#issuecomment-4244206563
and subsequent comments.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Apple DDM declarations support a vetted subset of Fleet variables with
per-host substitution; premium license required. Declaration tokens and
resend behavior now reflect variable changes; unresolved host
substitutions mark that host’s declaration as failed.

* **Bug Fixes**
* Clearer errors for unsupported or license-restricted Fleet variables
and more consistent DDM resend/update semantics when variables change.

* **Tests**
* Added extensive unit and integration tests covering Fleet variable
validation, substitution, token changes, resends, and failure states.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-04-20 09:14:52 -04:00