227e94de5b279decf0dec83550dd2eaa273aed56
3146
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
227e94de5b |
🤖 Chore: remove deprecated appendListOptionsWithCursorToSQL (#44385)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44723 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Strengthened validation of sorting/order parameters across many list and cursor-based endpoints — unsupported sort keys now return explicit errors and prevent unsafe queries. * Labels listing: label-list pagination query name changed; ordering by host_count is rejected when host counts are disabled (validated at request parsing). * **Tests** * Added/expanded tests covering allowed order keys, rejection of unknown keys, and pagination behavior for multiple listing APIs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com> |
||
|
|
b4a207fb5a |
Add ability to upload custom org logos (#44390)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44330, Resolves #44331 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests. (I'd defer integration tests to a separate PR since this one is pretty large already.) - [x] QA'd all new/changed functionality manually. I've tested this on both the setup flow and the organization settings page. I haven't had the time to test this on other places where we render the logo (macOS setup experience / MDM migration dialog). https://github.com/user-attachments/assets/95d4eae5-3da6-40f4-98a1-8575b97d96b3 ## New Fleet configuration settings - [x] Setting(s) is/are explicitly excluded from GitOps. Will handle GitOps in a separate PR. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Organizations can upload custom logos for light and dark modes. * Registration and Org Settings support logo file upload, preview, per-mode replace/delete, and validation (size & image formats). * Activity feed records logo changes/deletions; site nav displays uploaded logos per theme. * File uploader/preview adds a Fleet logo graphic option and improved logo validation. * Config/GitOps outputs now include separate dark/light logo fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8d37ec690c | Revert "Fix SCEP autorenew failing for offline hosts (#44250)" (#44535) | ||
|
|
beca71e674 |
Fix gitops dry-run to catch manual_agent_install + macos_script conflict (#44432)
**Related issue:** Resolves #34464 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually --- ## What GitOps `--dry-run` was succeeding when `macos_manual_agent_install` was set to `true` and a `macos_script` was configured under `setup_experience`, but the actual GitOps run would fail with: ``` Couldn't add setup experience script. To add script, first disable macos_manual_agent_install. ``` ## Why The `manual_agent_install` conflict validation only existed server-side in `ee/server/service/setup_experience.go:SetSetupExperienceScript()`. The script upload call (`uploadMacOSSetupScript()`) was gated by `!opts.DryRun` in `server/service/client.go`, so during dry-run the upload was skipped entirely and the validation never fired. ## Fix Added client-side validation in `server/service/client.go` at the point where the YAML-parsed `MacOSSetup` struct is processed — before the script file is validated and loaded. This check runs for **both dry-run and real runs**, catching the conflict early. Two code paths were fixed: 1. **Team path** (~line 803): Checks `setup.ManualAgentInstall.Value` when `setup.Script.Value` is set 2. **No-team path** (~line 2603): Checks `macOSSetup.ManualAgentInstall.Value` when `macOSSetup.Script.Value` is set ## How I reproduced the issue locally ### Prerequisites - MySQL and Redis running via Docker: `docker compose up -d mysql_test redis` ### Steps 1. Wrote an integration test (`TestDryRunMacOSSetupScriptWithManualAgentInstallConflict`) that: - Creates a GitOps user and fleetctl config - Creates a bootstrap package server serving `testdata/signed.pkg` - Creates a `.sh` script file with `echo "setup script"` - Creates a **global config** YAML (minimal server settings) - Creates a **team config** YAML with `macos_manual_agent_install: true`, `macos_script: <path>`, and `macos_bootstrap_package: <url>` - Runs `fleetctl gitops --dry-run` and asserts it fails - Runs `fleetctl gitops` (no dry-run) and asserts it fails 2. Ran the test **before the fix** — confirmed the bug: ``` Dry-run error: <nil> ← BUG: should have failed Real run error: ...status 422...first disable macos_manual_agent_install ← correctly fails ``` 3. Applied the fix and re-ran — **both dry-run and real run now fail** with the `macos_manual_agent_install` conflict error. ### Test command ```bash MYSQL_TEST=1 REDIS_TEST=1 go test -v \ -run TestIntegrationsEnterpriseGitops/TestDryRunMacOSSetupScriptWithManualAgentInstallConflict \ ./cmd/fleetctl/integrationtest/gitops/... -count=1 -timeout 600s ``` Both sub-tests (team and no-team paths) pass. All related existing tests continue to pass: - `TestMacOSSetup`, `TestMacOSSetupScriptWithFleetSecret`, `TestDeletingNoTeamYAML`, `TestDisallowSoftwareSetupExperience` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps dry-run now correctly fails when a macOS setup configuration combines manual agent installation with a provided setup script, preventing false-positive dry-run success. * **Tests** * Added unit and integration regression tests to verify dry-run and real-run rejection of conflicting macOS setup configurations for both team-scoped and unassigned host scopes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
888ee793e6 |
add missing IDP required check for OTA profiles (#44644)
Found this missed TODO from some old story work. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced OTA enrollment profile validation to properly enforce end-user authentication requirements when configured, now returning appropriate error responses for requests missing required authentication credentials. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c2dda6a16c |
Wipe host cancels all upcoming activities (#44323)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40459 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually Recording: https://drive.google.com/file/d/1_XqLyy-oY-WnIa97R4t9HihiBq3Fui6n/view?usp=drive_link <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Wiping a host now cancels all upcoming and queued activities for that host in a single, atomic operation to avoid intermediate activations. * **Bug Fixes** * Wipe response handling now distinguishes success vs failure and reliably cancels queued activities; datastore errors during host lookup or cancellation are surfaced. * Device lock/erase flows consistently update and propagate datastore errors. * **Tests** * Added integration and datastore tests validating wipe clears upcoming activities across macOS, Windows, Linux, and mixed-host scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Magnus Jensen <magnus@fleetdm.com> |
||
|
|
275b266ca1 |
produce failed enrollment renewal activity (#44511)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41418 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually To manually QA, I put an early return with `msg.Fail` in the `mdm_scep.go` file under PKIOperation method, and then triggered a SCEP renewal. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Activity logging for Apple MDM enrollment profile renewal failures to improve auditing and diagnostics. * Host display enhancements: include computer name and hardware model to improve host identification in activities and UI. * **Tests** * Integration tests verifying enrollment renewal failure activity creation, association to the correct host, and activity payload contents. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
62b716cc4e |
Enable disk encryption when only Windows MDM is configured. (#44462)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44194 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Team-level disk encryption can be toggled when at least one MDM platform (Windows or Apple) is configured, enabling BitLocker control for Windows-only deployments. * **Bug Fixes** * Updates validation to reject disk-encryption changes only when no MDM platforms are configured. * **Tests** * Added coverage for platform combinations and expected behavior, including Apple-specific profile creation when applicable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
1e4a9f292f |
Add activities for user actions on labels (#44522)
Resolves #36976 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Label operations (create, edit, delete) now generate activities shown in the activity feed with label and optional fleet context. * Host label add/remove operations emit corresponding label edited activities; duplicate label names are deduplicated. * Label activity types are selectable/filterable in the activity dashboard. * **Tests** * Added unit, integration, and UI tests covering label activity emission, rendering, filtering, and GitOps label lifecycle scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2723c132c2 |
Fixed GET /api/v1/fleet/commands timeout in large Fleet deployments (#44297)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44170 and Resolves #44422 Pagination is now pushed into each branch of the merged query, so per-tick work scales with page size instead of total commands. The Windows side was rewritten to avoid a disjunctive join that forced a nested-loop plan. `per_page` is capped (default 10), `page` is capped, and `order_key` is enforced against a closed allowlist on both code paths. Cursor pagination is fixed and is the recommended way to traverse beyond the page cap. This PR improves but does not fix the use case of fetching commands from all hosts. Deprecate usage without host_identifier: https://github.com/fleetdm/fleet/pull/44392/changes API doc updates: https://github.com/fleetdm/fleet/pull/44292 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enforced pagination on MDM commands list: per_page defaults to 10 (max 1,000) and page is capped at 100; traversal beyond page 100 requires cursor pagination via after. * **Bug Fixes / Performance** * Improved MDM command listing performance and de-duplication for large queries; fixed SQL error when combining host identifier with cursor pagination. * **Validation** * Requests exceeding pagination caps return 400; invalid sort keys return 422. * **Tests** * Added tests for pagination boundaries, cursor behavior, sort-key validation, and error responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
38a6129d0a |
Add include_all label scope to policies and reports (#44305)
**Related issue:** Resolves #41564 - Added include_all label scope to policies. - Added include_all and include_any scope to reports. |
||
|
|
e58fdca545 |
Skip membership catalog test for API-only users if they have access to everything (#44473)
Fixes #44403 Skip membership catalog test for API-only users if they have access to everything. |
||
|
|
96569a9c1c |
Fix SCEP autorenew failing for offline hosts (#44250)
**Related issue:** Resolves #44111 Customers reported certificates deployed via custom SCEP proxy were silently failing to auto-renew, leaving devices with expired certs. Five compounding bugs were causing this: ### 1. Cert metadata was wiped on every reconcile re-render `BulkUpsertMDMManagedCertificates` unconditionally overwrote `not_valid_before`, `not_valid_after`, and `serial` in `ON DUPLICATE KEY UPDATE`. Since the SCEP-proxy render-time payload has those fields nil (cert details aren't known until the device completes the handshake and osquery reports), every renewal trigger wiped them. Once NULL, the renewal cron's `HAVING validity_period IS NOT NULL` clause excluded the row — silently disabling future renewal attempts. Fixed by switching those columns to `COALESCE(VALUES(col), col)` so a nil incoming value preserves the existing value. DigiCert's flow (which does set the fields) and osquery's separate UPDATE in `updateHostMDMManagedCertDetailsDB` are unaffected. ### 2. 1-hour challenge TTL was too short for offline devices The challenge is generated at profile-render time but consumed when the device makes its SCEP request — which can be hours or days later (laptop asleep, on a plane, etc.). Devices that didn't pick up the InstallProfile push within the hour hit `challenge not found: sql: no rows in result set` and the renewal failed. Bumped `OneTimeChallengeTTL` from 1 hour to 7 days. Once consumed, the challenge is deleted immediately regardless of TTL. ### 3. Renewal cron re-fired on in-flight deliveries `WHERE hp.status IS NOT NULL` matched `'pending'` and `'verifying'` too, so a host whose delivery was still in flight (e.g., offline laptop) would have its profile re-rendered with a fresh challenge every cron tick — generating orphan nano commands and challenge rows hourly. Pre-fix this was masked by bug 1; once the COALESCE preserves cert metadata, the loop becomes visible. Tightened the filter to `WHERE hp.status IN ('verified', 'failed')` — settled states only. ### 4. iOS/iPadOS managed-cert profiles short-circuited to verified before cert metadata synced iOS/iPadOS profiles short-circuit `pending` → `verified` directly on MDM ack (no `verifying` step), since osquery isn't available to drive the standard verification cycle. That's correct for non-cert profiles, but for managed-cert profiles it created a window where the renewal cron saw `status='verified'` paired with stale cert metadata still in the renewal window — and the new `IN ('verified', 'failed')` filter from bug 3 kept matching, re-firing renewal each tick until `CertificateList` ingestion eventually caught up. Fixed by parking iOS/iPadOS managed-cert profiles at `'verifying'` on MDM ack and flipping them to `'verified'` from `updateHostMDMManagedCertDetailsDB` once fresh cert metadata arrives — i.e., reusing the existing state machine instead of inventing a parallel "renewal in flight" tracking column. The `EXISTS(SELECT 1 FROM host_mdm_managed_certificates ...)` check is folded into the existing platform-detection query, so no extra round-trip. macOS is unaffected: the new flip is redundant with `VerifyHostMDMProfiles` but idempotent. **Trade-off worth flagging:** if `CertificateList` ingestion never runs for an iOS managed-cert profile (broken cron, device offline indefinitely), the profile sits at `'verifying'` and the renewal cron's filter excludes it. In practice both run on the same Apple MDM cron loop — if one is broken, much else is too — but it's a sharper failure mode than letting renewals re-fire wastefully. ### 5. Permanent-failure profiles loop hourly through the renewal cron Once `'failed'` was added to the cron's status filter (bug 3), there was no longer any circuit breaker for profiles that fail at render time for non-transient reasons — CA deleted from app config, IDP variables missing from host, premium license downgraded. Each cron tick (1h interval) the cron flips `'failed'` → NULL, reconcile re-renders and immediately re-fails via `fleet.MarkProfilesFailed`, status returns to `'failed'`, repeat. Pre-fix this was masked by bug 1 (metadata wipe acted as accidental circuit breaker); once metadata is preserved (bug 1 fix), the loop becomes real and produces a profile-render attempt + nano command per failed cert per hour. Added a `renewalFailedRetryBackoff` constant (24h) and gated the `'failed'` branch on `hp.updated_at < DATE_SUB(NOW(), INTERVAL renewalFailedRetryBackoff SECOND)`. Transient SCEP-server outages still recover (within at most 24h, well under any cert validity window). Permanent failures still get retried daily (so a customer fixing the underlying issue eventually auto-recovers), but they don't churn nano commands hourly. `'verified'` rows in the renewal window are unaffected — they bypass the gate. ## Tests - `testMDMManagedSCEPCertificates`: three new sub-tests covering (a) cert-metadata preservation across reconcile re-renders, (b) in-flight-status skip behavior, (c) the permanent-failure backoff. Exercised against both NDES and Custom SCEP via the existing table-driven harness. - New `testIOSManagedCertProfileStaysVerifying`: verifies that on iOS, a managed-cert profile stays at `'verifying'` after MDM ack and only flips to `'verified'` once `UpdateHostCertificates` ingests fresh cert metadata. - New `challenges_test.go` covering `NewChallenge`/`ConsumeChallenge` lifecycle and TTL boundaries. - `TestCustomSCEPIntegration`: updated the hardcoded 2-hour challenge backdate to use `fleet.OneTimeChallengeTTL` so it stays correct as the constant evolves. - New `TestCustomSCEPRenewalPreservesCertMetadata` end-to-end test: drives the full reconcile path (rather than calling the bare datastore method) so a future change to the render-time payload structure can't silently regress the COALESCE preservation. |
||
|
|
07e4e7afe6 |
Short-circuit for empty software config in Gitops dry run (#44405)
Fixes #42607 |
||
|
|
78c0b0c651 |
43885: MLAPR migration + UUID capture (#44244)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43885 Adds a migration and code to capture the value of the fleet managed admin account if one exists. Changes file added for entire feature # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Automatic password rotation for managed local admin accounts on macOS, triggered after viewing activity. * Provisioning now captures and persists the managed admin account identifier (UUID) to support rotation and prevents that account from being stored as a regular user. * Hosts will request a best-effort recheck when the managed admin identifier is not yet available. * **Chores** * Database schema updated to store rotation scheduling and pending credential state. * **Tests** * Added tests covering UUID capture, conditional updates, migration, and ingest behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
98cad56716 |
redirect to correct URL, and allow both URLs for MDM SSO SAML validation if set (#44156)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41592 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed SSO failures when a custom Apple MDM URL is configured: callback requests are now redirected to the configured MDM URL when needed, and SAML validation correctly considers the configured MDM/server URLs so authentication succeeds for custom MDM setups. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
52caba768c |
Fix filtering in /api/v1/fleet/labels/:id/hosts endpoint (#44293)
- [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed filtering in the /api/v1/fleet/labels/:id/hosts endpoint and tightened validation to reject invalid sort/order keys with HTTP 422 responses. * Enforced ordering restrictions tied to feature flags (issues and device-mapping), rejecting unsupported order_key values. * **Tests** * Added extensive integration tests for order_key validation, deterministic sorting across allowed keys, and cursor pagination. * **Documentation** * Added a changelog entry noting the hosts-in-label filtering fix. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c0ecbfc1d8 |
Return Windows Enrollment Status Page (ESP) (#43454)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42843 This change shows Windows Enrollment Status Page (ESP) during OOBE enrollment. It does not track/update the status of that page, so the end user does not actually see any progress on it. Its purpose is to block the user from proceeding to desktop until all the profiles have been sent to the device. Software apps are not being tracked/blocked in this PR. This is what the final ESP screen looks for this PR before it takes the user to set up Windows Hello: <img width="646" height="549" alt="image" src="https://github.com/user-attachments/assets/748a2710-9388-4d04-93d1-8f2a518965a1" /> # Checklist for submitter ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enrollment Status Page (ESP) support for Windows Autopilot: sends hold/release commands and advances enrollment states during setup. * Scoped profile installation checks per host and a default ESP timeout (3 hours). * **Bug Fixes** * Clears prior profile delivery state during reenrollment cleanup to avoid stale delivery state. * Safer state transitions for "awaiting configuration" with guarded compare-and-swap updates. * **Tests** * New unit and integration tests validating ESP flows and awaiting-configuration transitions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Konstantin Sykulev <konst@sykulev.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
9628f49cb8 |
Improved the performance of Windows MDM profile reconciliation (#44075)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44052 Improve performance by reducing the time for the synchronous API call to update profiles or switch teams. And spreading out the application of profiles by processing 2000 hosts every 30 seconds. 1. **Windows profile reconciliation is no longer synchronous to bulk-set.** Apple, Android, and Apple-declaration paths still write their pending state inside the bulk-set transaction. The Windows path commits the transactional inputs and lets the existing `mdm_windows_profile_manager` cron pick the work up on its next tick. The visible effect is that `host_mdm_windows_profiles` is no longer guaranteed to be populated by the time bulk-set returns; it converges within one cron interval. 2. **The Windows reconciler now processes hosts in bounded batches, with a persisted cursor.** Previous behavior was "scan the universe of pending Windows hosts on every tick." New behavior is a host-window query bounded by batch size and a `host_uuid` cursor, advanced after the batch commits successfully and persisted across ticks. A failed tick leaves the cursor untouched so the same window is retried. 3. **Two replication races are now explicitly handled.** - Admin-delete vs reconcile: the existence check the reconciler uses to avoid touching a just-deleted profile reads from the primary, not a replica. - Insert lag in the reconciler's own listings: hosts that appear in the cursor query but are not yet visible in the scoped listings advance the cursor instead of jamming the loop. 4. **`updates.WindowsConfigProfile` from `BulkSetPendingMDMHostProfiles` is now always false in production.** The only consumer ORs it with the transactional signal from `BatchSetMDMProfiles`, which is the accurate source. The bulk-set call no longer attempts to compute or return that activity signal itself. 5. **Tests opt in to the old synchronous behavior via a named hook.** Default test behavior matches production (deferred). Legacy tests whose assertions require Windows rows immediately after bulk-set call an explicit enable-hook and rely on `t.Cleanup` to restore. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Windows MDM profile reconciliation batching improvements enable large team transfers and bulk profile change operations to complete faster, with profile updates rolling out in the background without blocking host check-ins or other MDM activity. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
7aea2e3fde |
Add gitops endpoints to api_endpoints catalog (#44291)
Resolves #44279 Add gitops endpoints to api_endpoints catalog |
||
|
|
a59ffd9288 |
Revert "Partial revert of #38785 work-in-progress (#44061)" (#44285)
This reverts commit
|
||
|
|
51dca83dec |
Fix script-only packages not setting install script file (#44299)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43659 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Preserves install scripts for script-only software installers when using hash-based references in GitOps, preventing self-service installs from silently no‑opping. * **Tests** * Added an integration regression test to verify batch installer resolution by hash preserves uploaded install script contents. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
a1b4833a82 |
updated default profile, added endpoint for seeing what default is applied (#44236)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43789 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * View and download the default automatic MDM (Apple Setup Assistant) enrollment profile via a new endpoint. * Shows a last-updated timestamp when present; returns the in‑app default with no timestamp if none is stored. * **Access** * Access follows existing team and global permission rules; not available on Free-tier licenses. * **Tests** * Added unit and integration tests covering endpoint behavior and access controls. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
45d49c5ab3 |
Better Apple server URL validation (#44163)
Small PR to improve URL validation for Apple Server URL, frontend now also requires a protocol, and does require a TLD (aka. no localhost). Backend requires a scheme/protocol, and no empty hostname, we previously did not have these validations in place, which breaks if you don't use a scheme for the URL. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Strengthened validation for MDM Apple Server URL and SSO User URL settings. MDM Apple Server URL now rejects localhost and non-HTTP/HTTPS schemes, and reports clearer errors for malformed URLs to reduce misconfiguration. * **Tests** * Added test cases covering malformed MDM Apple Server URL inputs to ensure validation behaves as expected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2c609ae78e |
CSAH: appconfig/gitops/DB migration to add preserve_host_activities_on_reenrollment field (#44212)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43943 # Checklist for submitter - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually See https://github.com/fleetdm/fleet/issues/43943#issuecomment-4329658412 ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) (see https://github.com/fleetdm/fleet/pull/43877/changes) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [ ] Verified that any relevant UI is disabled when GitOps mode is enabled (should be done by https://github.com/fleetdm/fleet/issues/43947) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a configuration option to preserve host activities during host re-enrollment, letting admins choose whether activity history is retained when hosts re-enroll. * **Chores** * Updated defaults and database migration state so the new setting is present in stored and generated configs and in GitOps outputs. * **Tests** * Added unit, integration, migration, and GitOps fixtures to validate behavior, serialization, and upgrade semantics. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
24e04a41c2 |
Move script request and response types from server/service to server/fleet (#43868)
For #36087 ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added script execution API supporting asynchronous and synchronous operations with timeout handling. * Introduced batch script execution capabilities including batch run creation, status querying, and execution cancellation. * Added host management API endpoints for locking, unlocking, and wiping devices. * Enhanced script management with create, update, delete, list, and retrieval operations. * Improved file download responses with proper content headers and attachment handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
9ae4373f89 |
Don't ignore GitOps secrets on free tier (#44148)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44118 # Details On free tier, ignore exceptions and always apply enroll secrets when present. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a, unreleased ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually @AndreyKizimenko QA'd manually For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed GitOps to correctly apply enrollment secrets and labels on free tier licenses, even when exception flags are configured. * **Tests** * Added tests validating that GitOps properly applies secrets and labels for free tier customers. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
899dc5aa57 |
Check for duplicate linux software installers (#44234)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43959 #44038 Refactored `checkSoftwareConflictsByIdentifier` to a switch statement with different logic per platform # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Bug Fixes - Prevented duplicate software installer entries on Linux. - Improved conflict detection for software installers across iOS, macOS, Windows, and Linux platforms to prevent incompatible uploads. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bcb3c39ebb |
🤖 #44198: Add guards on iOS/iPadOS refetch paths (#44205)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44198 Adds checks to the refetch type assertions so we both don't panic and skip writes when we don't have data(which shouldn't really be happening, but is a perf increase if it does). Also adds a warning if expected fields are missing on the checkin so we can still monitor if a customer is reporting missing fields I cannot figure out a reliable repro for this so testing was limited to automated tests added + some basic refetch testing(several times as I tried to figure out how to egt the thing to send it empty) # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Couldn't actually repro the bug but tests do verify the fix For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced Apple Mobile Device Management reliability by improving server response handling. The system now gracefully processes device information queries even when optional fields are missing or have unexpected formats, preventing potential service interruptions and preserving existing device data when updates are incomplete. * **Tests** * Added defensive unit tests covering various edge cases in Apple device information synchronization, including scenarios with missing or malformed data fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
65fcc132ae |
Fixed a race where a host could silently revert to its previous team (#44074)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44071 Verified fix in loadtest. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed a race condition that could cause hosts to silently revert to a previous team after an admin team transfer. * Improved reliability of team-transfer handling to prevent unexpected reversion during certificate/template transfers and device/profile operations. * **Tests** * Added regression tests to ensure team assignments persist correctly across host refreshes and related workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
5b8253173e |
Partial revert of #38785 work-in-progress (#44061)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38785 This feature has been pushed to 4.86 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Setup Experience Updates** * Windows: Automatic cancellation of pending setup steps when required software installation fails has been removed from the device setup experience * macOS: Device setup experience behavior remains unchanged, continuing to enforce software requirements during enrollment * Behavior is now platform-specific to align with individual operating system requirements <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c22954edf2 |
Remove unused windows_updates MySQL table and ingestion (#44128)
**Related issue:** Resolves #44127 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Removed the unused Windows Updates feature: ingestion, parsing, persistence APIs, and detail query; added a migration to drop the related database table. * **Tests** * Removed unit and integration tests for Windows update parsing, ingestion, persistence, and query inclusion. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
6c28db8206 |
Add activities when toggling GitOps exception settings (#44094)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #40171 # Details Adds audit activity when enabling or disabling GitOps exceptions. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a, unreleased ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <img width="714" height="699" alt="image" src="https://github.com/user-attachments/assets/161bd084-347b-4cde-893e-9b385f13872c" /> For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Activity feed now records when GitOps exceptions (labels, software, secrets) are enabled or disabled. * **UI** * Activity messages show which specific exception was enabled or disabled. * **Tests** * Added unit and integration tests verifying generation and rendering of enable/disable exception activities, including single and multiple flips and no-op updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sgress454 <553428+sgress454@users.noreply.github.com> Co-authored-by: Luke Heath <luke@fleetdm.com> |
||
|
|
9b01710a80 |
Don't throw gitops-exceptions-related errors on Free tier (#44118)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44098 # Details We set the "secrets" exception on for all new instances (and the label exception for existing instances), but you can't turn them off in the free tier. That means GitOps runs (including the one we use to initialize new instances) would fail with the "you can't use this key because the exception is on" error. This PR fixes the issue by not enforcing that rule for free tier instances. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a, unreleased ## Testing - [X] Added/updated automated tests - [X] added test verifying that the free tier can run gitops using excepted keys w/out error, and verified that it fails on main and passes on this branch - [X] QA'd all new/changed functionality manually - [X] spun up a new free-tier server successfully For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps exception enforcement no longer blocks free-tier users; enforcement is applied only for premium licenses, allowing GitOps applies on free tiers. * **Tests** * Added an integration test validating free-tier GitOps behavior to prevent regressions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
ee8c50e3b3 |
Fixed broken tests due validation logic not taking into account feature routes (#44112)
Extends API endpoint catalog validation to also look at feature routes. |
||
|
|
1539c6b094 |
Enforce consistent fleet name uniqueness across UI and GitOps (#33557)
Resolves #33557 The tems.name column uses utf8mb4_unicode_ci, so names like "ABC" and "abc" compare as equal at the database level. Before this change name collisions were handled in different ways in the UI and in GitOps. The changes introduced here, consolidates the logic used for detecting name collisions in all code path. All conflicts return 409 with the canonical copy "Fleet names must differ by at least one non-special character (case-insensitive). |
||
|
|
ba0f6b3c72 |
Update GitOps for managed local account fields (#44058)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42948 - Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup) SetDefaultsIfNeeded()` to account for new fields - Updated default creation and editing for team edit/creation paths - Updated `generate-gitops` warning message from `macos_setup` to `setup_experience` - Updated fields types to optjson and updated test files # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Team edit and team creation through GitOps, validated config with ` curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'` - New error message says `setup_experience` instead of `macos_setup` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added macOS MDM settings to control local account behavior: enable managed local accounts (default false) and specify end-user local account type (default "admin") for fleet and team configs. GitOps output now highlights unsupported setup-experience cases. * **Tests** * Updated fixtures and integration tests to assert and persist the new macOS local-account settings across config, team, and GitOps scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
28908e6083 |
Dashboard charts backend (#43910)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #42812 # Details This PR implements a new bounded context, `chart`, with a single endpoint `/charts`. The context encompasses a framework for recording and querying and aggregating historical data for Fleet hosts, and returning that data via the API for the purpose of charting. This initial iteration has a full implementation of a dataset called "uptime" which captures which hosts were online hour-by-hour (online meaning, having been "seen" at some point during that hour). It has a partial implementation of a "cve" dataset which will capture which hosts were vulnerable to which CVEs during a given day. ### Data storage Data is stored in an SCD (slowly-changing dimension) format in the `host_scd_data` table, where the main "value" in a row is stored in the `host_bitmap` column, which is a `mediumblob` where each bit encodes a host ID (bit one represents host ID 1, bit 1444 represents host ID 1444, etc.). The set of bits set on a row represents that hosts for which that dataset is "on" during a given time period represented by the `valid_from` (inclusive) and `valid_to` (exclusive) dates, where a `valid_to` can have the special "sentinel" value 9999-12-31T00:00:00.000 meaning that the row is still "open" (the value represents everything from `valid_from` to the present). Additionally an `entity_id` column can be used for datasets with multiple dimensions, e.g. CVE exposure or software usage which would have entity IDs representing CVEs or software items respectively. ### Data collection Data is collected via a cron job that runs every 10 minutes. Each dataset has its own `Collect` method which will sample the data for the given moment. For example the "uptime" dataset gathers the set of hosts that are online at the moment, and the "cve" dataset will gather the set of hosts that are vulnerable to each CVE at that moment. The sample can then be recorded using one of two strategies: * `accumulate`: bitwise OR the sample with any data already recorded for the current hour, or add a new pre-closed row for that hour. * `snapshot`: if there is no open row, create one with the sample and `valid_to set` to the sentinel. Otherwise: * If the sample has the same value as the current open row, do nothing * If the sample has a different value and the current open row's `valid_from` is within the same hour, update the current row's value * If the sample has a different value and the current open row's `valid_from` is not within the same hour, close the current open row and start a new one with `valid_from` = the start of the current hour ### Data retrieval 1. Gets the set of host IDs to retrieve data for. This starts with the set of host IDs in the requested fleet (or all the hosts a user has access to if no `fleet_id` param was passed to the `/charts` endpoint), and further whittled down by any filter options supplied with the request (labels, platforms, etc.). 2. Finds all `host_scd_data` rows for the requested dataset and date range (i.e. all rows whose `valid_from` is < the date range end and `valid_to` is > the date range start). 3. Calculates the date ranges of the "buckets" to return datapoints for. For the uptime chart we default to 3-hour buckets, so we want 8 buckets per day. 4. Iterates over each bucket and finds the row or rows from host_scd_data that cover that bucket range. For datasets using the "accumulate" strategy, the values for those rows are ORed together. For "snapshot"s, we take the one active at the bucket end time to represent the bucket (e.g. "which hosts had a given CVE at the end of the day") ### Tools This PR includes two dev tools that don't require deep review: * **chart-backfill** - used to backfill data to various datasets for testing * **charts-collect** - used to collect data from a live server via the API and put into a local hosts_scd_data table # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually - With [front-end branch](https://github.com/fleetdm/fleet/pull/43878) <img width="712" height="434" alt="image" src="https://github.com/user-attachments/assets/b2ccce49-b5fd-4076-b47f-0eea6a53260c" /> ## Database migrations - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added charting bounded context: HTTP API for metrics (uptime, CVE), dataset registry, hosted dataset collection, background collection/cleanup with opt-out env. * New utilities: host bitmap operations and string-list/uint-list parsers. * New CLI tools to collect and backfill chart data. * **Database** * Migration and schema to store host time-series SCD chart data. * **Tests** * Extensive unit and integration tests for service, storage, caching, cron, and utilities. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2d687d94b3 |
41676 updated apple crs invalid domain error message (#43396)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41676 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [ x] Added/updated automated tests - [x ] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Apple MDM APNS certificate signing now shows a clear, domain-specific error when an unsupported email domain is supplied (applies to CSR requests and renewal flows), replacing the previous generic "invalid email" message. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
bf3a12a960 |
Updated eula pdf upload size check to default max request body size (#43517)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40856 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * EULA PDF upload size validation now reports the configured maximum upload size (in MiB) instead of a fixed value, improving clarity of rejection messages. * **Tests** * Added tests covering EULA upload size validation, ensuring oversized uploads are rejected with the proper status and that configured request-body limits are respected. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
7fa1341c73 |
Windows wipe failed acivitiy (#43795)
**Related issue:** Resolves #42290 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Failed Windows MDM wipe attempts now create a tracked "Failed wipe" activity showing the affected host and display name for visibility; UI filter and activity feed now surface this type. * **Bug Fixes** * Improved detection and reporting of wipe result statuses so real failures are reliably surfaced. * Duplicate failure responses are suppressed to avoid repeated alerts. * **Tests** * Added tests validating wipe-failure activity creation and related control flows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com> |
||
|
|
9feb9c2be0 | Fix Recovery Lock password desync on MDM re-enrollment (#43827) | ||
|
|
cf11ebefff |
Add VPP verify timeout for device endpoint (#43975)
Fixes #43957 |
||
|
|
2d3a790a10 |
Don't resend pending certificates (#43820)
**Related issue:** Resolves #37556 |
||
|
|
04c9abbdb6 |
Bugfix: fallback the script execution timeout from agent options to global if not set for team (#43911)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40952 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually See https://drive.google.com/file/d/1lot6KmliWmTpJ-paKyT_aI8GYq8PO71L/view?usp=drive_link <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Script execution timeout now falls back to the global agent setting when not explicitly defined at the team level, making timeout behavior more predictable. * **Tests** * Added test coverage validating timeout resolution across various team/global configuration scenarios to prevent regressions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
39e4f616ea |
macOS managed local account foundations (#43381)
Implements both #42942 and #42943 Co-authored-by: jkatz01 <yehonatankatz@gmail.com> |
||
|
|
91d9b25924 |
Allow conditional downloads across fleets (#43679)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43417 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. done in https://github.com/fleetdm/fleet/pull/42216 ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - Using a local fileserver, added the same software to two fleets and ran `fleetctl gitops`. Verified that the first fleet downloaded the file, the second fleet used the cache, and both fleet showed the software installer in the UI. ## Summary by CodeRabbit * **Chores** * Updated software installer lookup mechanism to support optional team-scoped searches, enabling fallback to cross-team installer cache when team-specific installers are unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
1f6e556818 |
Improved Windows MDM performance (#43912)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43875 `POST /api/mdm/microsoft/management` is the hot endpoint for any Windows-MDM-enrolled fleet. Every enrolled host hits it twice per check-in interval. At 40k hosts that's a four-figure sustained queries-per-second rate on the database reader pool, dominated by one expensive query plus a handful of redundant `MDMWindowsGetEnrolledDeviceWithDeviceID` lookups on the same row. This PR cuts that load by: 1. Short-circuiting the pending-commands query when the device's queue is empty (the overwhelming common case). Replaces a 3-table join plus anti-join with a cheap primary-key probe. 2. Loading the enrolled device exactly once in `isTrustedRequest` and threading it through to every downstream consumer instead of re-fetching it three times. No behavior change to the protocol, no schema change. Also filed a related issue: https://github.com/fleetdm/fleet/issues/43897 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Alerted the release DRI if additional load testing is needed <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved Windows MDM server performance at scale by reducing database queries during device check-ins. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
915ba8a45c |
🤖 Ignore DEP premium filters on Fleet Free tier (#43865)
## Summary Fixes #43826. Adds code to silently ignore `dep_profile_error` and `dep_assign_profile_response` query parameters similar to other premium only parameters on the list hosts endpoint. This PR does _NOT_ include filtering out the `dep_profile_error` field from the host details object since it doesn't seem like a common pattern to filter out individual fields from a response object, but that can be changed if needed. # Checklist for submitter ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ### Before fix On premium: - UI: can see "AB issue" card - UI: can see MDM status with Profile assignment error: failed - API: can curl `/fleet/hosts?dep_profile_error=true` and get the host - API: can curl `/fleet/hosts` and see `"dep_profile_error":true` for the failed host - API: can curl `/fleet/hosts/7` and see `"dep_profile_error": true` - API: can curl `/fleet/hosts?dep_assign_profile_response=FAILED` On free: - UI: can't see "AB issue" card - UI: can see MDM status pending but no details, api response has `"dep_profile_error": true` - API: can curl `/fleet/hosts?dep_profile_error=true` and get the host - API: can curl `/fleet/hosts` and see `"dep_profile_error":true` for the failed host - API: can curl `/fleet/hosts?dep_assign_profile_response=FAILED` ### After fix On free: - API: curl `/fleet/hosts?dep_profile_error=true` and the filter gets ignored - API: curl `/fleet/hosts?dep_assign_profile_response=FAILED` or other statuses and the filter gets ignored - API: can curl `/fleet/hosts` to show all hosts and it shows `"dep_profile_error":true` since it is not filtered at the host details level - API: can curl `/fleet/hosts/7` and see dep_profile_error since it is not filtered at the host details level ``` jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_assign_profile_response=FAILED' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8878 0 8878 0 0 144k 0 --:--:-- --:--:-- --:--:-- 146k 5 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_profile_error=true' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8879 0 8879 0 0 298k 0 --:--:-- --:--:-- --:--:-- 298k 5 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8879 0 8879 0 0 283k 0 --:--:-- --:--:-- --:--:-- 289k 5 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_profile_error=true&dep_assign_profile_response=FAILED' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8879 0 8879 0 0 311k 0 --:--:-- --:--:-- --:--:-- 321k 5 ``` On premium: - Everything works as expected, filters work ``` jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_assign_profile_response=FAILED' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1696 0 1696 0 0 58681 0 --:--:-- --:--:-- --:--:-- 60571 1 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_profile_error=true' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1696 0 1696 0 0 59613 0 --:--:-- --:--:-- --:--:-- 60571 1 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9055 0 9055 0 0 355k 0 --:--:-- --:--:-- --:--:-- 368k 5 jonathan@jonathans-macbook-pro:~/fleet$ curl -k -X GET 'https://localhost:8080/api/v1/fleet/hosts?dep_profile_error=true&dep_assign_profile_response=FAILED' -H "Authorization: Bearer $TOKEN" | wc -l % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1696 0 1696 0 0 60845 0 --:--:-- --:--:-- --:--:-- 62814 1 ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Host listing and counts now ignore premium-only filters on non-premium licenses, ensuring consistent results for free-tier users. * **Tests** * Added tests validating that host listing and counting behave differently between free and premium license tiers when premium filters are used. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
2b35eabd5d |
Added middleware for api-only users auth (#43772)
Fixes #42885 Added new middleware (APIOnlyEndpointCheck) that enforces 403 for API-only users whose request either isn't in the API endpoint catalog or falls outside their configured per-user endpoint restrictions. |