275b266ca134ec8a7fadde65d39d1fc52406dcff
4176
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Magnus Jensen
|
275b266ca1 |
produce failed enrollment renewal activity (#44511)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41418 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually To manually QA, I put an early return with `msg.Fail` in the `mdm_scep.go` file under PKIOperation method, and then triggered a SCEP renewal. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Activity logging for Apple MDM enrollment profile renewal failures to improve auditing and diagnostics. * Host display enhancements: include computer name and hardware model to improve host identification in activities and UI. * **Tests** * Integration tests verifying enrollment renewal failure activity creation, association to the correct host, and activity payload contents. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
de86536f42 |
Redis-backed cache for host-by-key lookups (#43936)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43928 This PR adds a Redis-backed cache in front of the two host-by-key lookups on the agent auth paths. Docs: https://github.com/fleetdm/fleet/pull/44504 ## What changes **Read path (osquery/orbit auth):** - `LoadHostByNodeKey` and `LoadHostByOrbitNodeKey` now check Redis before falling through to MySQL. - Successful lookups are cached for 60s ± 10% jitter (configurable via `FLEET_REDIS_HOST_CACHE_TTL`). - `NotFound` results are cached for 5s as a negative entry, dampening repeated probes for keys that do not exist (deleted hosts whose agents are still polling, attacker scans, retry storms). - Concurrent lookups for the same key collapse into one DB query via `singleflight`. The shared query runs under a context detached from any one caller's deadline so the leader giving up does not abort the work for joiners. The shared query is itself bounded by a 30s timeout so a wedged DB call cannot pin the singleflight slot indefinitely. **Write path (invalidations):** - These methods now invalidate the cache after a successful inner call: `UpdateHost`, `SerialUpdateHost`, `UpdateHostOsqueryIntervals`, `UpdateHostRefetchRequested`, `UpdateHostRefetchCriticalQueriesUntil`, `UpdateHostIdentityCertHostIDBySerial`, `EnrollOsquery`, `EnrollOrbit`, `NewHost`, `DeleteHost`, `DeleteHosts`, `CleanupExpiredHosts`, `CleanupIncomingHosts`, `AddHostsToTeam`. - `AddHostsToTeam`, `DeleteHosts`, `CleanupExpiredHosts`, and `CleanupIncomingHosts` use a pipelined batch invalidator so 10k-host operations stay in the millisecond range instead of taking minutes of sequential round-trips. - Inner-call errors are not invalidations: a failing write leaves cached state intact. **Configuration:** - New flags `FLEET_REDIS_HOST_CACHE_ENABLED` (default `true`) and `FLEET_REDIS_HOST_CACHE_TTL` (default `60s`). - Server refuses to start if the cache is enabled with `TTL <= 0`. **Observability:** - Three new OTEL counters under the `fleet` meter: - `fleet.host_cache.lookups{result=hit|negative_hit|miss}` - `fleet.host_cache.errors{op=get|set|del}` - `fleet.host_cache.invalidations{reason=update|enroll|team|delete|cert}` - A pre-built SigNoz dashboard ships in `tools/signoz/host_cache_dashboard.json`. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Optional Redis-backed host lookup cache for osquery and orbit auth, with automatic invalidation and metrics/monitoring dashboard. * **Bug Fixes** * Fixed host-removal batching so cache-related removals use correct chunks. * **Tests** * Added comprehensive host-cache unit tests covering hits, negative cache, invalidation, concurrency, and JSON round-trips. * **Chores** * New config flags to enable the cache and set TTL (default 60s ±10% jitter). <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor LyuboslavskyandCopilot
|
62b716cc4e |
Enable disk encryption when only Windows MDM is configured. (#44462)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44194 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Team-level disk encryption can be toggled when at least one MDM platform (Windows or Apple) is configured, enabling BitLocker control for Windows-only deployments. * **Bug Fixes** * Updates validation to reject disk-encryption changes only when no MDM platforms are configured. * **Tests** * Added coverage for platform combinations and expected behavior, including Apple-specific profile creation when applicable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Lucas Manuel Rodriguez
|
1e4a9f292f |
Add activities for user actions on labels (#44522)
Resolves #36976 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Label operations (create, edit, delete) now generate activities shown in the activity feed with label and optional fleet context. * Host label add/remove operations emit corresponding label edited activities; duplicate label names are deduplicated. * Label activity types are selectable/filterable in the activity dashboard. * **Tests** * Added unit, integration, and UI tests covering label activity emission, rendering, filtering, and GitOps label lifecycle scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
2723c132c2 |
Fixed GET /api/v1/fleet/commands timeout in large Fleet deployments (#44297)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44170 and Resolves #44422 Pagination is now pushed into each branch of the merged query, so per-tick work scales with page size instead of total commands. The Windows side was rewritten to avoid a disjunctive join that forced a nested-loop plan. `per_page` is capped (default 10), `page` is capped, and `order_key` is enforced against a closed allowlist on both code paths. Cursor pagination is fixed and is the recommended way to traverse beyond the page cap. This PR improves but does not fix the use case of fetching commands from all hosts. Deprecate usage without host_identifier: https://github.com/fleetdm/fleet/pull/44392/changes API doc updates: https://github.com/fleetdm/fleet/pull/44292 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enforced pagination on MDM commands list: per_page defaults to 10 (max 1,000) and page is capped at 100; traversal beyond page 100 requires cursor pagination via after. * **Bug Fixes / Performance** * Improved MDM command listing performance and de-duplication for large queries; fixed SQL error when combining host identifier with cursor pagination. * **Validation** * Requests exceeding pagination caps return 400; invalid sort keys return 422. * **Tests** * Added tests for pagination boundaries, cursor behavior, sort-key validation, and error responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Tim Lee
|
2d586cb2ff |
fleetctl vulnerability-data-stream to download OSV data (#44260)
|
||
|
Juan Fernandez
|
38a6129d0a |
Add include_all label scope to policies and reports (#44305)
**Related issue:** Resolves #41564 - Added include_all label scope to policies. - Added include_all and include_any scope to reports. |
||
|
W0lfbaneandCopilot
|
ad7ea0aa7f |
fix(android): remove tautological NCR filter in hostVPPInstalls (#42873)
## Problem The `hostVPPInstalls` function in `server/datastore/mysql/software.go` contains a SQL condition: ```sql (hvsi.platform != 'android' OR ncr.id IS NULL) AND ``` This is a logical tautology — it **never filters any rows**: - **Android rows**: `ncr.id` is always `NULL` because Android installs use Google's Android Management API, not nanoMDM. The condition evaluates to `(FALSE OR TRUE) = TRUE`. - **Apple rows**: The first operand `hvsi.platform != 'android'` is `TRUE`, so the whole expression is `TRUE` regardless of `ncr.id`. The condition was likely added during early Android VPP support to guard against unexpected NCR joins for Android. However, since `nano_command_results` is only written by the nanoMDM Apple MDM storage layer (`server/mdm/nanomdm/storage/mysql/queue.go:168`), the guard can never trigger. Elsewhere in the codebase, the canonical pattern for NCR filtering is: ```sql -- vpp.go:248, software_installers.go:1812 (ncr.id IS NOT NULL OR (:platform = 'android' AND ncr.id IS NULL)) ``` This pattern has *different semantics* — it filters per-app aggregate status counts to only include confirmed installs. The `hostVPPInstalls` function serves the host software list where showing all statuses (including pending) is intentional, so no NCR filter is needed. ## Changes - Removed the dead condition from the `last_vpp_install` UNION branch - Added a clarifying comment explaining why no NCR filter is applied and how this differs from other query sites - Added changelog entry ## Testing - No behavior change — the removed condition was always TRUE - Existing tests pass without modification - `go build ./server/datastore/mysql/...` compiles clean #android #sql #cleanup --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Scott Gress
|
07b6679ead | Build fleetctl MSI packages (#43403) | ||
|
Dante Catalfamo
|
ecc7d2ce02 |
Add macOS 26 Tahoe CIS benchmark v1.0.0 (#44090)
**Related issue:** Resolves #35173 # macOS 26 Tahoe CIS benchmark v1.0.0 (new benchmark) Adds a brand-new policy set covering the **CIS Apple macOS 26 Tahoe Benchmark, v1.0.0** under `ee/cis/macos-26/`. Follows the same layout as `macos-13`/`-14`/`-15` (`cis-policy-queries.yml`, `README.md`, `test/scripts/`, `test/profiles/`). ## Coverage | Section | Title | Status | |---|---|---| | 1 | Install Updates, Patches and Additional Security Software | complete (6/6 automated) | | 2 | System Settings | complete (all automated across §2.1–§2.18) | | 3 | Logging and Auditing | complete (5/5 automated) | | 4 | Network Configurations | complete (3/3 automated) | | 5 | System Access, Authentication and Authorization | complete (19/19 automated) | | 6 | Applications | complete (7/7 automated) | | 7 | Supplemental | skipped (per Fleet convention) | Total automated policies shipped: **89**. Manual-assessment recommendations are documented in `ee/cis/macos-26/README.md` under **Limitations**. ## Notable query/format choices - **Combined-key profiles per CIS instructions.** §2.2.1+§2.2.2 (Firewall + Stealth Mode) are shipped as a single `2.2.1-and-2.2.2.mobileconfig` because CIS explicitly requires both keys in the same profile. §2.6.5 (Gatekeeper) and §2.11.2 (screensaver wake-password + delay) follow the same pattern. - **§2.5.2.1 (Siri)** uses the new `allowAssistant=false` key on `com.apple.applicationaccess`, replacing the deprecated `com.apple.ironwood.support` payload from earlier benchmarks. - **§2.6.3.2** uses the spaced literal key `Siri Data Sharing Opt-In Status` (integer 2) on `com.apple.assistant.support` — the v1.0.0 PayloadType move from `com.apple.applicationaccess`. - **§5.1.6, §5.1.7, §3.1, §5.7** use fleetd-only osquery tables (`find_cmd`, `authdb`, `pwd_policy`, `dscl`, etc.) and are flagged `(Fleetd Required)` in the policy descriptions. - **§2.10.1.2** (Apple Silicon sleep ≤15 min) default-passes on Intel hosts via a `system_info.cpu_type` check. ## Test artifacts added | Type | Count | Location | |---|---|---| | Pass scripts | 48 | `ee/cis/macos-26/test/scripts/CIS_*_pass.sh` | | Fail scripts | 46 | `ee/cis/macos-26/test/scripts/CIS_*_fail.sh` | | Pass-only scripts | 2 | `CIS_1.1.sh`, `CIS_5.1.6.sh` | | MDM profiles | 37 | `ee/cis/macos-26/test/profiles/*.mobileconfig` | Profile-only recommendations (§2.3.1.x AirDrop/AirPlay, §2.5.x Apple Intelligence, §2.6.3.x Analytics, §6.x Safari/Terminal) ship with a `.mobileconfig` only and no script counterpart, since CIS marks them as configurable solely via profile. ## Documentation updates | File | Change | |---|---| | `ee/cis/macos-26/README.md` | New file — coverage table, limitations, per-section notes (query patterns, fleetd dependencies, FDA requirements). | | `ee/cis/CIS-BENCHMARKS.md` | Added `macos-26/` to the directory layout; updated **Query patterns** doc to include the `EXISTS`/`NOT EXISTS` user-vs-system-scope guidance and `username = ''` notes. | | `ee/cis/prompt.md` | Refreshed authoring prompts with macOS-26 conventions (combined-key profiles, fleetd-table flagging). | | `tools/cis/cis-test-runner.py` | Minor adjustments to support the new benchmark directory. | | `changes/35173-cis-macos-26-v1` | User-visible change note. | <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added macOS 26 CIS Benchmark v1.0.0 with comprehensive configuration profiles to enforce recommended system and app settings (updates, firewall/stealth, privacy, backups, FileVault, Safari, Terminal, etc.). * **Tests** * Added extensive pass/fail remediation and validation scripts for CIS controls across macOS subsystems; test runner updated to include macOS 26 support and mark an SSH-related control as manual. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Dante Catalfamo
|
96569a9c1c |
Fix SCEP autorenew failing for offline hosts (#44250)
**Related issue:** Resolves #44111 Customers reported certificates deployed via custom SCEP proxy were silently failing to auto-renew, leaving devices with expired certs. Five compounding bugs were causing this: ### 1. Cert metadata was wiped on every reconcile re-render `BulkUpsertMDMManagedCertificates` unconditionally overwrote `not_valid_before`, `not_valid_after`, and `serial` in `ON DUPLICATE KEY UPDATE`. Since the SCEP-proxy render-time payload has those fields nil (cert details aren't known until the device completes the handshake and osquery reports), every renewal trigger wiped them. Once NULL, the renewal cron's `HAVING validity_period IS NOT NULL` clause excluded the row — silently disabling future renewal attempts. Fixed by switching those columns to `COALESCE(VALUES(col), col)` so a nil incoming value preserves the existing value. DigiCert's flow (which does set the fields) and osquery's separate UPDATE in `updateHostMDMManagedCertDetailsDB` are unaffected. ### 2. 1-hour challenge TTL was too short for offline devices The challenge is generated at profile-render time but consumed when the device makes its SCEP request — which can be hours or days later (laptop asleep, on a plane, etc.). Devices that didn't pick up the InstallProfile push within the hour hit `challenge not found: sql: no rows in result set` and the renewal failed. Bumped `OneTimeChallengeTTL` from 1 hour to 7 days. Once consumed, the challenge is deleted immediately regardless of TTL. ### 3. Renewal cron re-fired on in-flight deliveries `WHERE hp.status IS NOT NULL` matched `'pending'` and `'verifying'` too, so a host whose delivery was still in flight (e.g., offline laptop) would have its profile re-rendered with a fresh challenge every cron tick — generating orphan nano commands and challenge rows hourly. Pre-fix this was masked by bug 1; once the COALESCE preserves cert metadata, the loop becomes visible. Tightened the filter to `WHERE hp.status IN ('verified', 'failed')` — settled states only. ### 4. iOS/iPadOS managed-cert profiles short-circuited to verified before cert metadata synced iOS/iPadOS profiles short-circuit `pending` → `verified` directly on MDM ack (no `verifying` step), since osquery isn't available to drive the standard verification cycle. That's correct for non-cert profiles, but for managed-cert profiles it created a window where the renewal cron saw `status='verified'` paired with stale cert metadata still in the renewal window — and the new `IN ('verified', 'failed')` filter from bug 3 kept matching, re-firing renewal each tick until `CertificateList` ingestion eventually caught up. Fixed by parking iOS/iPadOS managed-cert profiles at `'verifying'` on MDM ack and flipping them to `'verified'` from `updateHostMDMManagedCertDetailsDB` once fresh cert metadata arrives — i.e., reusing the existing state machine instead of inventing a parallel "renewal in flight" tracking column. The `EXISTS(SELECT 1 FROM host_mdm_managed_certificates ...)` check is folded into the existing platform-detection query, so no extra round-trip. macOS is unaffected: the new flip is redundant with `VerifyHostMDMProfiles` but idempotent. **Trade-off worth flagging:** if `CertificateList` ingestion never runs for an iOS managed-cert profile (broken cron, device offline indefinitely), the profile sits at `'verifying'` and the renewal cron's filter excludes it. In practice both run on the same Apple MDM cron loop — if one is broken, much else is too — but it's a sharper failure mode than letting renewals re-fire wastefully. ### 5. Permanent-failure profiles loop hourly through the renewal cron Once `'failed'` was added to the cron's status filter (bug 3), there was no longer any circuit breaker for profiles that fail at render time for non-transient reasons — CA deleted from app config, IDP variables missing from host, premium license downgraded. Each cron tick (1h interval) the cron flips `'failed'` → NULL, reconcile re-renders and immediately re-fails via `fleet.MarkProfilesFailed`, status returns to `'failed'`, repeat. Pre-fix this was masked by bug 1 (metadata wipe acted as accidental circuit breaker); once metadata is preserved (bug 1 fix), the loop becomes real and produces a profile-render attempt + nano command per failed cert per hour. Added a `renewalFailedRetryBackoff` constant (24h) and gated the `'failed'` branch on `hp.updated_at < DATE_SUB(NOW(), INTERVAL renewalFailedRetryBackoff SECOND)`. Transient SCEP-server outages still recover (within at most 24h, well under any cert validity window). Permanent failures still get retried daily (so a customer fixing the underlying issue eventually auto-recovers), but they don't churn nano commands hourly. `'verified'` rows in the renewal window are unaffected — they bypass the gate. ## Tests - `testMDMManagedSCEPCertificates`: three new sub-tests covering (a) cert-metadata preservation across reconcile re-renders, (b) in-flight-status skip behavior, (c) the permanent-failure backoff. Exercised against both NDES and Custom SCEP via the existing table-driven harness. - New `testIOSManagedCertProfileStaysVerifying`: verifies that on iOS, a managed-cert profile stays at `'verifying'` after MDM ack and only flips to `'verified'` once `UpdateHostCertificates` ingests fresh cert metadata. - New `challenges_test.go` covering `NewChallenge`/`ConsumeChallenge` lifecycle and TTL boundaries. - `TestCustomSCEPIntegration`: updated the hardcoded 2-hour challenge backdate to use `fleet.OneTimeChallengeTTL` so it stays correct as the constant evolves. - New `TestCustomSCEPRenewalPreservesCertMetadata` end-to-end test: drives the full reconcile path (rather than calling the bare datastore method) so a future change to the render-time payload structure can't silently regress the COALESCE preservation. |
||
|
Jordan MontgomeryandCopilot Autofix powered by AI
|
62b60fef24 |
Improve filtering on commands endpoints (#44426)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> Provides better errors on invalid/unexpected sort keys passed to `/api/v1/fleet/commands`, `/api/v1/fleet/mdm/commands` and `/api/v1/fleet/mdm/apple/commands` endpoints # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved validation for invalid `order_key` values on MDM command endpoints (`/api/v1/fleet/commands`, `/api/v1/fleet/mdm/commands`, and `/api/v1/fleet/mdm/apple/commands`), ensuring only approved sorting parameters are accepted. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Carlo
|
07e4e7afe6 |
Short-circuit for empty software config in Gitops dry run (#44405)
Fixes #42607 |
||
|
|
3e38592fda |
Fix FD leak in goval_dictionary Analyze (#42741) (#43983)
**Related issue:** Resolves #42741 ## Problem `goval_dictionary.Analyze` opened a `*sql.DB` via `LoadDb` but never closed it. `pkg/download/download.go` atomically renames the goval sqlite on each refresh, unlinking the old inode while the pool still held FDs on it. lsof showed them as `(deleted)`, accumulating over days until Fleet server restart. ## Fix - New `Database.Close()` that delegates to the underlying `*sql.DB`. - `defer func() { _ = db.Close() }()` in `Analyze` right after `LoadDb`. ## How this was tested - New unit test `TestDatabaseCloseReleasesFileHandle` opens a file-backed sqlite, runs a query to force a pool connection, then asserts Close drains the pool and blocks further queries. - `go test ./server/vulnerabilities/goval_dictionary/...` passes. - Standalone Go program reproduced the leak mechanism: `sql.Open` + query + unlink left the FD on the orphaned inode; adding Close released it. ## Confidence and QA ~90% confident. I did not reproduce end-to-end through Fleet's vuln cron locally (the analyzer never entered its query loop; likely `HostIDsByOSVersion` hadn't populated for the Rocky test host). Reviewer: flag anything that drops your confidence. @xpkoala for QA after merge: please exercise in a production-like env with enrolled RHEL hosts and confirm no `(deleted)` FDs after goval refreshes. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/` (`changes/42741-fix-goval-dictionary-fd-leak`). - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (N/A, no new input paths). - [x] Timeouts are implemented and retries are limited to avoid infinite loops (N/A, no new network calls). - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes (N/A, no endpoint changes). ## Testing - [x] Added/updated automated tests - [x] Where appropriate, automated tests simulate multiple hosts and test for host isolation (N/A, package-level unit test). - [ ] QA'd all new/changed functionality manually (pending, post-merge by @xpkoala). ## Database migrations - [x] Checked schema for modified tables for auto-updating timestamp columns (N/A, no schema changes). - [x] Confirmed timestamp updates are acceptable (N/A, no schema changes). - [x] Ensured correct collation is explicitly set for character columns (N/A, no schema changes). ## New Fleet configuration settings - [x] Setting(s) is/are explicitly excluded from GitOps (N/A, no new settings). ## fleetd/orbit/Fleet Desktop - [x] Verified compatibility with the latest released version of Fleet (N/A, server-only change). - [x] If the change applies to only one platform, confirmed `runtime.GOOS` is used (N/A). - [x] Verified fleetd runs on macOS, Linux and Windows (N/A, server-only change). - [x] Verified auto-update works (N/A, server-only change). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed a file-descriptor leak in vulnerability processing so deleted SQLite database files are properly closed without requiring a server restart, improving stability and resource usage. * **Tests** * Added a regression test to ensure database handles are released after close. * **Documentation** * Documented the fix for the file-descriptor leak. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
Jordan MontgomeryandCopilot
|
78c0b0c651 |
43885: MLAPR migration + UUID capture (#44244)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43885 Adds a migration and code to capture the value of the fleet managed admin account if one exists. Changes file added for entire feature # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Automatic password rotation for managed local admin accounts on macOS, triggered after viewing activity. * Provisioning now captures and persists the managed admin account identifier (UUID) to support rotation and prevents that account from being stored as a regular user. * Hosts will request a best-effort recheck when the managed admin identifier is not yet available. * **Chores** * Database schema updated to store rotation scheduling and pending credential state. * **Tests** * Added tests covering UUID capture, conditional updates, migration, and ingest behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Magnus Jensen
|
98cad56716 |
redirect to correct URL, and allow both URLs for MDM SSO SAML validation if set (#44156)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41592 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed SSO failures when a custom Apple MDM URL is configured: callback requests are now redirected to the configured MDM URL when needed, and SAML validation correctly considers the configured MDM/server URLs so authentication succeeds for custom MDM setups. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
4334017b38 |
Add Vulnerabilities exposure dataset (#44124)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #43769 # Details Adds methods to collect data for the `cve` dataset. As with all sets this is collected at hourly granularity, but unlike the `uptime` set, the `cve` set uses the "snapshot" strategy so that we record at most one change (the most recent) per hour. For this first iteration, we are _recording_ data for all CVEs (i.e., which hosts were exposed to which CVEs at a given time), but we are only _reporting_ a subset of CVEs for the dashboard chart. See [this comment](https://github.com/fleetdm/fleet/pull/44124#discussion_r3155554405) for more info. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually - [X] Spot-checked the CVEs chosen by the `trackedCVESoftwareMatchers` and didn't find any outside of the expected - [X] With [front-end PR](https://github.com/fleetdm/fleet/pull/44261), generated chart: <img width="706" height="421" alt="image" src="https://github.com/user-attachments/assets/539d9877-6573-4406-a159-1d2a711a045f" /> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Host vulnerability (CVE) chart added to the dashboard; CVE chart data collection is now active. * Critical CVE tracking surfaces high-severity vulnerabilities. * **Improvements** * CVE chart refreshes every 3 hours (was daily) for more timely insights. * Snapshot collection reconciles and closes prior data during empty runs to keep charts accurate. * CVE queries may produce zero datapoints when no tracked CVEs exist, without affecting other metrics. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
3cadfa1714 |
Fix issue with fleet's docker image in k8s environments (#44373)
Resolves #44298 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [X] QA'd all new/changed functionality manually 1. Running the docker image pushed by this PR with the user 3333 doesn't fail anymore: ```sh docker run --platform linux/amd64 -it --user 3333:3333 fleetdm/fleet@sha256:1a06bcae25e13e37f871378c7c156f5a2cdf67bc3c3e3bcdc95b6afc0c6decbb [...] ts=2026-04-29T13:25:56Z level=warn msg="could not connect to db" err="dial tcp [::1]:3306: connect: connection refused" sleep_interval=0s [...] ``` 4.84.0 fails with: ```sh docker run --platform linux/amd64 -it --user 3333:3333 fleetdm/fleet:v4.84.0@sha256:51b56ad59a840b28e074ff9b06d6d5b232b0ca2f0d999bb164820da69c7cbe15 Failed to fetch user info for home directory: user: unknown userid 33332026/04/29 13:28:08 71 <nil> ``` 2. `strings ./build/fleet | rg github.com/AbGuthrie/goquery/v2` returns nothing in this branch and returns plenty of matches in `main`. 3. Smoke tested `fleetctl goquery` functionality. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Resolved Docker image startup failures in Kubernetes environments caused by a dependency side effect. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
52caba768c |
Fix filtering in /api/v1/fleet/labels/:id/hosts endpoint (#44293)
- [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed filtering in the /api/v1/fleet/labels/:id/hosts endpoint and tightened validation to reject invalid sort/order keys with HTTP 422 responses. * Enforced ordering restrictions tied to feature flags (issues and device-mapping), rejecting unsupported order_key values. * **Tests** * Added extensive integration tests for order_key validation, deterministic sorting across allowed keys, and cursor pagination. * **Documentation** * Added a changelog entry noting the hosts-in-label filtering fix. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
9628f49cb8 |
Improved the performance of Windows MDM profile reconciliation (#44075)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44052 Improve performance by reducing the time for the synchronous API call to update profiles or switch teams. And spreading out the application of profiles by processing 2000 hosts every 30 seconds. 1. **Windows profile reconciliation is no longer synchronous to bulk-set.** Apple, Android, and Apple-declaration paths still write their pending state inside the bulk-set transaction. The Windows path commits the transactional inputs and lets the existing `mdm_windows_profile_manager` cron pick the work up on its next tick. The visible effect is that `host_mdm_windows_profiles` is no longer guaranteed to be populated by the time bulk-set returns; it converges within one cron interval. 2. **The Windows reconciler now processes hosts in bounded batches, with a persisted cursor.** Previous behavior was "scan the universe of pending Windows hosts on every tick." New behavior is a host-window query bounded by batch size and a `host_uuid` cursor, advanced after the batch commits successfully and persisted across ticks. A failed tick leaves the cursor untouched so the same window is retried. 3. **Two replication races are now explicitly handled.** - Admin-delete vs reconcile: the existence check the reconciler uses to avoid touching a just-deleted profile reads from the primary, not a replica. - Insert lag in the reconciler's own listings: hosts that appear in the cursor query but are not yet visible in the scoped listings advance the cursor instead of jamming the loop. 4. **`updates.WindowsConfigProfile` from `BulkSetPendingMDMHostProfiles` is now always false in production.** The only consumer ORs it with the transactional signal from `BatchSetMDMProfiles`, which is the accurate source. The bulk-set call no longer attempts to compute or return that activity signal itself. 5. **Tests opt in to the old synchronous behavior via a named hook.** Default test behavior matches production (deferred). Legacy tests whose assertions require Windows rows immediately after bulk-set call an explicit enable-hook and rely on `t.Cleanup` to restore. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Windows MDM profile reconciliation batching improvements enable large team transfers and bulk profile change operations to complete faster, with profile updates rolling out in the background without blocking host check-ins or other MDM activity. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
RachelElysia
|
7f7f6ac5d0 | Fleet UI: Improved error message when deleting a label that is for targeting a software installation (#44320) | ||
|
Victor Lyuboslavsky
|
a59ffd9288 |
Revert "Partial revert of #38785 work-in-progress (#44061)" (#44285)
This reverts commit
|
||
|
Jonathan Katz
|
51dca83dec |
Fix script-only packages not setting install script file (#44299)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43659 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Preserves install scripts for script-only software installers when using hash-based references in GitOps, preventing self-service installs from silently no‑opping. * **Tests** * Added an integration regression test to verify batch installer resolution by hash preserves uploaded install script contents. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Juan Fernandez
|
384789d178 |
Fixed patch logic on EditUperPage component (#44302)
**Related issue:** Resolves #44252 & #44227 # Checklist for submitter If some of the following don't apply, delete the relevant line. ## Testing - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Bug Fixes * Fixed an issue preventing users from enabling two-factor authentication when editing existing user accounts. * Fixed team assignment handling when editing user accounts to properly process empty team selections. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
5ac50a2dc9 |
Bound orbit retries on missing installer details to 5 mins (#44284)
Fixes #44084 |
||
|
Magnus Jensen
|
a1b4833a82 |
updated default profile, added endpoint for seeing what default is applied (#44236)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43789 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * View and download the default automatic MDM (Apple Setup Assistant) enrollment profile via a new endpoint. * Shows a last-updated timestamp when present; returns the in‑app default with no timestamp if none is stored. * **Access** * Access follows existing team and global permission rules; not available on Free-tier licenses. * **Tests** * Added unit and integration tests covering endpoint behavior and access controls. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Konstantin Sykulev
|
9ec20e60b7 |
Windows MDM improved host profile status performance (#44225)
**Related issue:** Resolves #44189 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance** * Optimized Windows MDM profile removal to skip redundant database writes for terminal removals. * **Bug Fixes** * Ensure terminal remove responses (both verified and failed) delete the corresponding profile records without affecting concurrent installs. * **Tests** * Added coverage for mixed install/remove responses and re-install after a verified removal. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
RachelElysia
|
5ba261d5c6 | Fleet UI: New empty states (#43896) | ||
|
Jonathan Katz
|
899dc5aa57 |
Check for duplicate linux software installers (#44234)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43959 #44038 Refactored `checkSoftwareConflictsByIdentifier` to a switch statement with different logic per platform # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Bug Fixes - Prevented duplicate software installer entries on Linux. - Improved conflict detection for software installers across iOS, macOS, Windows, and Linux platforms to prevent incompatible uploads. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Nico
|
b471e24cd2 |
Fix: Label shows outdated manually selected host after navigating back to the Edit page (#44216)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43135 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed stale "Selected hosts" display on the edit label page by ensuring host data refreshes after successful label edits, so the UI reflects the updated host set immediately. * Fixed stale host selections when navigating between manual labels by scoping and resetting the form to the correct current host set, preventing selections from carrying over between labels. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Nico
|
fb19bd48b6 |
Fix: Navigating to a a new host displays activity feed for a previously opened host details page (#44218)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43591 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] QA'd all new/changed functionality manually ### Before https://github.com/user-attachments/assets/ddd1bfe6-d8f8-426c-8add-71721013e18d ### After https://github.com/user-attachments/assets/86e4f60f-b78e-40cb-835b-a7ea40d54f10 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Bug Fixes * Fixed an issue where the host details activity feed could incorrectly display activities from a previously viewed host when navigating between different hosts. The activity data is now properly scoped to ensure the activity feed shows only activities relevant to the currently selected host, preventing stale data from appearing. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrew Mellor
|
4bb40c99ee |
updated AB tokens renew and delete options to enabled in gitops mode (#43510)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42440 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enabled renewing and deleting Apple Business Manager (AB) tokens in the UI when running in GitOps mode. * Apple Business Manager table actions now adapt to GitOps mode: some actions are disabled and contextual tooltips explain unavailable options (including repository-linked guidance when applicable). <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jordan Montgomery
|
bcb3c39ebb |
🤖 #44198: Add guards on iOS/iPadOS refetch paths (#44205)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44198 Adds checks to the refetch type assertions so we both don't panic and skip writes when we don't have data(which shouldn't really be happening, but is a perf increase if it does). Also adds a warning if expected fields are missing on the checkin so we can still monitor if a customer is reporting missing fields I cannot figure out a reliable repro for this so testing was limited to automated tests added + some basic refetch testing(several times as I tried to figure out how to egt the thing to send it empty) # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Couldn't actually repro the bug but tests do verify the fix For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced Apple Mobile Device Management reliability by improving server response handling. The system now gracefully processes device information queries even when optional fields are missing or have unexpected formats, preventing potential service interruptions and preserving existing device data when updates are incomplete. * **Tests** * Added defensive unit tests covering various edge cases in Apple device information synchronization, including scenarios with missing or malformed data fields. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
bd18bac797 |
Adding gitOpsModeEnabled and gitOpsModeExceptions to anonymous statistics payload (#44161)
**Related issue:** Resolves #42240. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Statistics now include GitOps mode: whether it’s enabled and the ordered list of configured exception categories (serializes as an empty list when none). * **Tests** * Added tests for GitOps-related statistics transitions and made statistics-timing tests deterministic for reliable behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Luke Heath
|
fc7af80db4 | Adding changes for Fleet v4.84.0 (#43092) (#44174) | ||
|
Victor LyuboslavskyandCopilot
|
65fcc132ae |
Fixed a race where a host could silently revert to its previous team (#44074)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44071 Verified fix in loadtest. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed a race condition that could cause hosts to silently revert to a previous team after an admin team transfer. * Improved reliability of team-transfer handling to prevent unexpected reversion during certificate/template transfers and device/profile operations. * **Tests** * Added regression tests to ensure team assignments persist correctly across host refreshes and related workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Victor Lyuboslavsky
|
5b8253173e |
Partial revert of #38785 work-in-progress (#44061)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38785 This feature has been pushed to 4.86 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Setup Experience Updates** * Windows: Automatic cancellation of pending setup steps when required software installation fails has been removed from the device setup experience * macOS: Device setup experience behavior remains unchanged, continuing to enforce software requirements during enrollment * Behavior is now platform-specific to align with individual operating system requirements <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
c22954edf2 |
Remove unused windows_updates MySQL table and ingestion (#44128)
**Related issue:** Resolves #44127 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Removed the unused Windows Updates feature: ingestion, parsing, persistence APIs, and detail query; added a migration to drop the related database table. * **Tests** * Removed unit and integration tests for Windows update parsing, ingestion, persistence, and query inclusion. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
43552b8dac |
Reworked Windows profile delete batching (#44047)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42545 This rework does not significantly improve the worst case performance, but it does improve some cases (like lower number of hosts with a lot of profiles). # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance Improvements** * Improved batch deletion for Windows MDM configuration profiles to handle very large-scale cleanup with fewer database updates. * Replaced per-profile update loops with multi-profile batched updates to reduce update overhead and improve determinism. * **Tests** * Added tests validating multi-profile batch delete behavior and ensuring each queued delete command is correctly targeted. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
4fd9244195 |
Don't update host values that didn't change (reduce writer load) (#43992)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42545 Tested in loadtest. `host_orbit_info` and `host_disks` are seeing the massive skip wins # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance** * Avoids unnecessary database writes when incoming host data already match stored values, reducing steady-state update volume. * **Tests** * Added tests to validate skip-on-no-change behavior, including cases with small numeric precision differences and idempotency checks. * **Documentation** * Added changelog entry describing the reduced-write behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Magnus JensenandCopilot
|
43e32d916e |
don't send AccountConfiguration for non macs (#44085)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43228 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Account configuration commands for SSO and managed local admin setup are no longer sent to iOS and iPadOS devices when end-user authentication is enabled; these commands are restricted to macOS only. * **Tests** * Added an integration test ensuring AccountConfiguration is not enqueued for iOS/iPadOS enrollments when end-user authentication is enabled. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Scott Gress
|
207c3058c5 |
Add dashboard charts frontend (#43878)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #43769 # Details This PR adds "Hosts active" and "Hosts enrolled" charts to the dashboard. New components: * **ChartCard.tsx**: encapsulates a visualization-agnostic chart, for data provided by the new `/charts` endpoint created in https://github.com/fleetdm/fleet/pull/43910 * **ChartFilterModal.tsx**: modal for setting filters on a chart. Currently supports filtering by label, platform and individual host. * **CheckerboardViz.tsx**: a checkerboard visualization for use in ChartCard. Capable of charting 1, 7, 14 or 30 days at a time, although only 30 day charts are used right now. Bespoke rendering using SVG, since recharts scatterchart was harder to wrangle than it was worth. * **LineChartViz.tsx**: a line-chart visualization using Recharts * **HostsEnrolledCard.tsx**: a bar chart of enrolled hosts using Recharts # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - With backend provided by https://github.com/fleetdm/fleet/pull/43910: <img width="1426" height="428" alt="image" src="https://github.com/user-attachments/assets/0f53b9d1-c87b-4225-a175-2d40af5afe41" /> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Dashboard now shows interactive "Hosts active" (line/heatmap) and "Hosts enrolled" (bar) charts with metric selection, filter modal (labels/platforms/hosts), legends, tooltips, and responsive layout. * **Tests** * Added comprehensive tests covering chart rendering, checkerboard heatmap, and no-data states. * **Chores** * Added charting library dependency to support visualizations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Juan Fernandez
|
1539c6b094 |
Enforce consistent fleet name uniqueness across UI and GitOps (#33557)
Resolves #33557 The tems.name column uses utf8mb4_unicode_ci, so names like "ABC" and "abc" compare as equal at the database level. Before this change name collisions were handled in different ways in the UI and in GitOps. The changes introduced here, consolidates the logic used for detecting name collisions in all code path. All conflicts return 409 with the canonical copy "Fleet names must differ by at least one non-special character (case-insensitive). |
||
|
Dante Catalfamo
|
90e1dc311b |
Update macOS 15 to CIS Benchmark 2.0.0 (#43909)
**Related issue:** Resolves #35171 # macOS 15 CIS benchmark v1.1.0 → v2.0.0 change set ## Policy changes | CIS ID | Policy name (new) | Change type | Effect | |---|---|---|---| | 1.1 | CIS - Ensure Apple-provided Software Updates Are Installed (Fleetd Required) | Modified | Renamed from "Ensure All Apple-provided Software Is Current"; resolution expanded with terminal `softwareupdate -i -a` method; `cis_id` added. Query unchanged (still uses fleetd's `software_update` table). | | 1.6 | CIS - Ensure Software Update Deferment Is Less Than or Equal to 30 Days (MDM Required) | Modified | Added Apple-deprecation note to description; `cis_id` added. Query unchanged. | | 1.7 | CIS - Ensure XProtect Is Running and Updated | Removed | Deleted — v2.0.0 removed 1.7 from the numbered benchmark and moved it to Supplemental section 7.4. Per authoring outline, Fleet does not track section 7+ recommendations. | | 2.1.1.1 | CIS - Ensure iCloud Passwords & Keychain is enabled/disabled (MDM Required) | Modified | Renamed from "iCloud Keychain" to match v2.0.0's "Audit iCloud Passwords & Keychain"; Description, Rationale, and Impact Statement updated from the new PDF; `cis_id` added to both enable/disable variants. Query unchanged (`allowCloudKeychainSync` key is the same). | | 2.3.1.1 | CIS - Ensure AirDrop Is Disabled (MDM Required) | Modified | Added note to resolution stating AirDrop can only be toggled via configuration profile; `cis_id` added. Query unchanged. | | 2.3.3.4 | CIS - Ensure Remote Login Is Disabled | Modified | Description rewritten to match v2.0.0 text; terminal remediation (`systemsetup -setremotelogin off`) added to resolution; `cis_id` added. Query unchanged (still checks `disabled.plist`). | | 2.6.3.1, 2.6.3.2, 2.6.3.3, 2.6.3.4 | CIS - Ensure Sending Diagnostic and Usage Data to Apple Is Disabled (MDM Required) | Modified | **Query change**: Siri Data Sharing Opt-In Status check moved from `com.apple.applicationaccess` → `com.apple.assistant.support` domain to track v2.0.0's new PayloadType; `cis_id` added (combined). | | 2.4.1 | CIS - Ensure Show Wi-Fi status in Menu Bar Is Enabled (MDM Required) | Removed | Deleted — recommendation removed in v2.0.0. | | 2.4.2 | CIS - Ensure Show Bluetooth Status in Menu Bar Is Enabled (MDM Required) | Removed | Deleted — recommendation removed in v2.0.0 (replaced by Manual 2.4.1 "Audit Menu Bar and Control Center Icons"). | | 6.1.1 | CIS - Ensure Show All Filename Extensions Setting is Enabled | Removed | Deleted — recommendation downgraded to Manual in v2.0.0. | Policy count: 113 → 109. ## Test artifacts added | Target CIS ID | File | Type | Notes | |---|---|---|---| | 2.3.3.4 | `ee/cis/macos-15/test/scripts/CIS_2.3.3.4_pass.sh` | pass script | `systemsetup -setremotelogin off` | | 2.3.3.4 | `ee/cis/macos-15/test/scripts/CIS_2.3.3.4_fail.sh` | fail script | `systemsetup -setremotelogin on` | | 1.6 | `ee/cis/macos-15/test/profiles/1.6.mobileconfig` | MDM profile | Sets `enforcedSoftwareUpdateDelay=30`, `forceDelayedSoftwareUpdates=true` | | 2.3.1.1 | `ee/cis/macos-15/test/profiles/2.3.1.1.mobileconfig` | MDM profile | Sets `allowAirDrop=false` | | 2.6.3.2 | `ee/cis/macos-15/test/profiles/2.6.3.2.mobileconfig` | MDM profile | Four payloads covering the combined Fleet query: Siri opt-in (new v2.0.0 domain), AutoSubmit, allowDiagnosticSubmission, AXSAudioDonationSiriImprovementEnabled | No test added for 1.1 — query depends on live OS update state (fleetd `software_update` table) and cannot be toggled by a script or profile. No test added for 2.1.1.1 — it is an org-decision Audit recommendation (Fleet ships both enable and disable variants); one of the two policies fails by construction regardless of system state. ## Documentation updates | File | Change | |---|---| | `ee/cis/macos-15/README.md` | Benchmark version bumped v1.1.0 → v2.0.0. Limitations list renumbered to v2.0.0 section numbers; added 2.4.1 (new "Audit Menu Bar and Control Center Icons") and 6.1.1 (now Manual "Audit Show All Filename Extensions"). Org-decision entry renamed from "Audit iCloud Keychain" to "Audit iCloud Passwords & Keychain" to match v2.0.0 terminology. | |
||
|
Victor Lyuboslavsky
|
fb499b9a01 |
Improved Windows MDM profiles summary performance (#44034)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44050 Original query like: ```sql SELECT CASE WHEN EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp WHERE h.uuid = hmwp.host_uuid AND hmwp.status = 'failed' -- fleet.MDMDeliveryFailed AND hmwp.profile_name NOT IN (<reserved_names>) ) THEN 'failed' WHEN EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp WHERE h.uuid = hmwp.host_uuid AND (hmwp.status IS NULL OR hmwp.status = 'pending') -- fleet.MDMDeliveryPending AND hmwp.profile_name NOT IN (<reserved_names>) AND NOT EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp2 WHERE h.uuid = hmwp2.host_uuid AND hmwp2.status = 'failed' AND hmwp2.profile_name NOT IN (<reserved_names>) ) ) THEN 'pending' WHEN EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp WHERE h.uuid = hmwp.host_uuid AND hmwp.operation_type = 'install' -- fleet.MDMOperationTypeInstall AND hmwp.status = 'verifying' -- fleet.MDMDeliveryVerifying AND hmwp.profile_name NOT IN (<reserved_names>) AND NOT EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp2 WHERE h.uuid = hmwp2.host_uuid AND hmwp2.operation_type = 'install' AND hmwp2.profile_name NOT IN (<reserved_names>) AND (hmwp2.status IS NULL OR hmwp2.status NOT IN ('verifying', 'verified')) ) ) THEN 'verifying' WHEN EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp WHERE h.uuid = hmwp.host_uuid AND hmwp.operation_type = 'install' AND hmwp.status = 'verified' -- fleet.MDMDeliveryVerified AND hmwp.profile_name NOT IN (<reserved_names>) AND NOT EXISTS ( SELECT 1 FROM host_mdm_windows_profiles hmwp2 WHERE h.uuid = hmwp2.host_uuid AND hmwp2.operation_type = 'install' AND hmwp2.profile_name NOT IN (<reserved_names>) AND (hmwp2.status IS NULL OR hmwp2.status != 'verified') ) ) THEN 'verified' ELSE '' END AS final_status, SUM(1) AS count FROM hosts h JOIN host_mdm hmdm ON h.id = hmdm.host_id JOIN mdm_windows_enrollments mwe ON h.uuid = mwe.host_uuid WHERE mwe.device_state = 'enrolled' -- microsoft_mdm.MDMDeviceStateEnrolled AND h.platform = 'windows' AND hmdm.is_server = 0 AND hmdm.enrolled = 1 AND <team_filter> GROUP BY final_status; ``` New query like: ```sql SELECT ( SELECT CASE WHEN SUM(CASE WHEN hmwp.status = 'failed' AND hmwp.profile_name NOT IN (<reserved_names>) THEN 1 ELSE 0 END) > 0 THEN 'failed' WHEN SUM(CASE WHEN (hmwp.status IS NULL OR hmwp.status = 'pending') AND hmwp.profile_name NOT IN (<reserved_names>) THEN 1 ELSE 0 END) > 0 THEN 'pending' WHEN SUM(CASE WHEN hmwp.operation_type = 'install' AND hmwp.status = 'verifying' AND hmwp.profile_name NOT IN (<reserved_names>) THEN 1 ELSE 0 END) > 0 THEN 'verifying' WHEN SUM(CASE WHEN hmwp.operation_type = 'install' AND hmwp.status = 'verified' AND hmwp.profile_name NOT IN (<reserved_names>) THEN 1 ELSE 0 END) > 0 THEN 'verified' ELSE '' END FROM host_mdm_windows_profiles hmwp WHERE hmwp.host_uuid = h.uuid ) AS final_status, SUM(1) AS count FROM hosts h JOIN host_mdm hmdm ON h.id = hmdm.host_id JOIN mdm_windows_enrollments mwe ON h.uuid = mwe.host_uuid WHERE mwe.device_state = 'enrolled' AND h.platform = 'windows' AND hmdm.is_server = 0 AND hmdm.enrolled = 1 AND <team_filter> GROUP BY final_status; ``` # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Alerted the release DRI if additional load testing is needed <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Performance Improvements** * Optimized Windows MDM profile summary and host OS settings filtering for faster, lower-cost server queries. * **Tests** * Added an exhaustive verification test covering all Windows MDM profile permutations to ensure correct status bucketing and host membership. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
28908e6083 |
Dashboard charts backend (#43910)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #42812 # Details This PR implements a new bounded context, `chart`, with a single endpoint `/charts`. The context encompasses a framework for recording and querying and aggregating historical data for Fleet hosts, and returning that data via the API for the purpose of charting. This initial iteration has a full implementation of a dataset called "uptime" which captures which hosts were online hour-by-hour (online meaning, having been "seen" at some point during that hour). It has a partial implementation of a "cve" dataset which will capture which hosts were vulnerable to which CVEs during a given day. ### Data storage Data is stored in an SCD (slowly-changing dimension) format in the `host_scd_data` table, where the main "value" in a row is stored in the `host_bitmap` column, which is a `mediumblob` where each bit encodes a host ID (bit one represents host ID 1, bit 1444 represents host ID 1444, etc.). The set of bits set on a row represents that hosts for which that dataset is "on" during a given time period represented by the `valid_from` (inclusive) and `valid_to` (exclusive) dates, where a `valid_to` can have the special "sentinel" value 9999-12-31T00:00:00.000 meaning that the row is still "open" (the value represents everything from `valid_from` to the present). Additionally an `entity_id` column can be used for datasets with multiple dimensions, e.g. CVE exposure or software usage which would have entity IDs representing CVEs or software items respectively. ### Data collection Data is collected via a cron job that runs every 10 minutes. Each dataset has its own `Collect` method which will sample the data for the given moment. For example the "uptime" dataset gathers the set of hosts that are online at the moment, and the "cve" dataset will gather the set of hosts that are vulnerable to each CVE at that moment. The sample can then be recorded using one of two strategies: * `accumulate`: bitwise OR the sample with any data already recorded for the current hour, or add a new pre-closed row for that hour. * `snapshot`: if there is no open row, create one with the sample and `valid_to set` to the sentinel. Otherwise: * If the sample has the same value as the current open row, do nothing * If the sample has a different value and the current open row's `valid_from` is within the same hour, update the current row's value * If the sample has a different value and the current open row's `valid_from` is not within the same hour, close the current open row and start a new one with `valid_from` = the start of the current hour ### Data retrieval 1. Gets the set of host IDs to retrieve data for. This starts with the set of host IDs in the requested fleet (or all the hosts a user has access to if no `fleet_id` param was passed to the `/charts` endpoint), and further whittled down by any filter options supplied with the request (labels, platforms, etc.). 2. Finds all `host_scd_data` rows for the requested dataset and date range (i.e. all rows whose `valid_from` is < the date range end and `valid_to` is > the date range start). 3. Calculates the date ranges of the "buckets" to return datapoints for. For the uptime chart we default to 3-hour buckets, so we want 8 buckets per day. 4. Iterates over each bucket and finds the row or rows from host_scd_data that cover that bucket range. For datasets using the "accumulate" strategy, the values for those rows are ORed together. For "snapshot"s, we take the one active at the bucket end time to represent the bucket (e.g. "which hosts had a given CVE at the end of the day") ### Tools This PR includes two dev tools that don't require deep review: * **chart-backfill** - used to backfill data to various datasets for testing * **charts-collect** - used to collect data from a live server via the API and put into a local hosts_scd_data table # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually - With [front-end branch](https://github.com/fleetdm/fleet/pull/43878) <img width="712" height="434" alt="image" src="https://github.com/user-attachments/assets/b2ccce49-b5fd-4076-b47f-0eea6a53260c" /> ## Database migrations - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added charting bounded context: HTTP API for metrics (uptime, CVE), dataset registry, hosted dataset collection, background collection/cleanup with opt-out env. * New utilities: host bitmap operations and string-list/uint-list parsers. * New CLI tools to collect and backfill chart data. * **Database** * Migration and schema to store host time-series SCD chart data. * **Tests** * Extensive unit and integration tests for service, storage, caching, cron, and utilities. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Dante Catalfamo
|
9ff63eb52e |
macOS 14 CIS benchmark v3.0.0 update (#43797)
**Related issue:** Resolves #35172 Updates the macOS 14 (Sonoma) CIS policy set to benchmark v3.0.0, adds a `cis_id` field to every policy, fixes several broken test scripts, introduces an automated test runner, and ships `CIS-BENCHMARKS.md` as a central guide for authoring and maintaining CIS benchmarks. ## Summary of changes - `ee/cis/macos-14/cis-policy-queries.yml`: v3.0.0 updates + `cis_id` added to every entry - `ee/cis/CIS-BENCHMARKS.md`: new authoring/testing/automation guide for all macOS CIS benchmarks (and the pattern other OS dirs follow) - `tools/cis/cis-test-runner.py`: new 2150-line Python runner that drives end-to-end validation against a real tart VM + Fleet server - `ee/cis/macos-14/test/scripts/`: 10 new pass/fail script pairs, 8 existing scripts fixed (several were silently broken) # How the automated testing works The runner (`tools/cis/cis-test-runner.py`) exercises the full policy lifecycle against a real macOS VM: **Phases** 1. **Parse** `cis-policy-queries.yml` and filter by `--all`, `--cis-ids`, `--match`, and type flags (`--only-scripts`, `--only-mdm`, `--only-manual`). 2. **Classify** each policy into a test type based on available artifacts: | Priority | Type | Artifacts | Behavior | |---|---|---|---| | 1 | `PASS_FAIL` | `CIS_{id}_pass.sh` + `_fail.sh` | Run fail → verify query fails → run pass → verify passes | | 2 | `PASS_ONLY` | `CIS_{id}.sh` | Run script → verify passes | | 3 | `PROFILE` | `.mobileconfig` only | Verify query fails before profile → push profile → verify passes | | 4 | `ORG_DECISION` | paired `-enable`/`-disable` profiles | Toggle between variants | | 5 | `MANUAL` | none | Prompt operator, or skip with `--skip-manual` | 3. **Provision**: create a fresh Fleet team with a unique enroll secret, build a fleetd pkg bound to it, create+boot a tart VM, install the agent, and enroll. 4. **MDM**: prompt the operator for MDM enrollment if any tests need it. Clear team profiles, baseline the VM, push all required profiles in one batch, wait for delivery. 5. **Execute**: for each plan, SCP the script, run it over SSH, then run the policy via `fleetctl query --hosts <hostname>`. A query that returns rows = pass. 6. **Report** summary with PASS/FAIL/SKIP/ERROR counts. 7. **Cleanup** (with `--cleanup`) deletes the team, host record, and VM. **Special-case handling** (keyed by OS version because CIS IDs aren't stable across releases): - `SSH_BREAKING_CIS_IDS`: tests that disable sshd (2.3.3.4, 2.3.3.5) are forced to `MANUAL` so the runner doesn't lock itself out. - `PASSWORD_POLICY_CIS_IDS`: 5.2.x profiles invalidate the VM's `admin`/`admin` login — forced to `MANUAL`. - `NON_AUTOMATABLE_CIS_IDS`: tests that can't run reliably in a VM (Location Services, Touch ID, shared Siri profile state) forced to `MANUAL` with a per-entry reason. - `--keep-vm`: reuses the VM across runs, skipping agent install/enrollment if the host is already in Fleet. Falls back to fresh creation if SSH is unreachable. **Credential resolution order**: CLI flag → `FLEET_URL`/`FLEET_API_TOKEN` env → `~/.fleet/config` (from `fleetctl login`). ## How to use `CIS-BENCHMARKS.md` going forward The doc is the single reference for authoring and maintaining CIS benchmark policies across all macOS (and Windows) versions. For each new benchmark release, the workflow is: 1. **Read "Updating benchmarks when a new CIS version is released"** — directs you to the PDF's *Appendix: Change History* to enumerate Added/Modified/Removed recommendations. 2. **Use the field reference and query patterns** to write or update policies: direct table check, `managed_policies` EXISTS/NOT EXISTS, or plist negation check. Name qualifiers `(MDM Required)` / `(Fleetd Required)` / `(FDA Required)` are documented. 3. **Create matching test artifacts** — pass/fail scripts for togglable settings, `.mobileconfig` profiles for MDM-only settings. Script conventions (full paths, sudo pattern, `not_always_working_` prefix) are standardized. 4. **Update the per-OS README** with limitations, org-decision policies, and optional policies. 5. **Run the test runner** to validate. The doc also contains an **end-to-end AI agent prompt** (section at the bottom) designed to be handed a new CIS PDF plus the previous version's PDF, to automatically generate the diff, write policies, produce test artifacts, update docs, and run validation. This lets future benchmark updates start from a consistent, repeatable baseline rather than being hand-authored from scratch. ## Query changes All entries in `cis-policy-queries.yml` received a `cis_id` field so the runner (and humans) can map policies → scripts → profiles → the benchmark document without parsing the display name. | CIS ID | Change | |---|---| | 1.1 | Renamed "Ensure All Apple-provided Software Is Current" → "Ensure Apple-provided Software Updates Are Installed"; added terminal remediation to `resolution` | | 1.6 | Expanded description with v3.0.0 language about rapid security response updates | | 1.x (deferment) | **Removed** — "Ensure Software Update Deferment Is Less Than or Equal to 30 Days" dropped from v3.0.0 | | 2.3.1.1 | Renamed "Ensure AirDrop Is Disabled" → "Ensure AirDrop Is Disabled When Not Actively Transferring Files"; expanded description | | 2.3.3.1 (DVD/CD Sharing) | **Removed** — dropped from v3.0.0 | | 2.3.3.4 (Remote Login) | Query now checks BOTH `disabled.plist` AND that `com.openssh.sshd` is not in the `launchd` table; resolution updated to terminal method | | 5.1.7 | Query rewritten: sticky-bit dirs now properly excluded (first-char-of-mode check instead of bit-AND on full mode string); SIP-protected dirs excluded via `com.apple.rootless` xattr check | | 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.7 | Dropped `username = ''` filter — Safari profiles deliver at user scope, so the system-scope filter guaranteed zero rows | | 6.3.3 | Fixed `NOT EXISTS` domain typo: `com.apple.loginwindow` → `com.apple.Safari` (the check was previously meaningless) | | Wi-Fi/Bluetooth menu bar | **Removed** — "Show Wi-Fi status in Menu Bar" and "Show Bluetooth status in Menu Bar" dropped from v3.0.0 | | Show All Filename Extensions | **Removed** — dropped from v3.0.0 | ## Script changes ### New scripts | Script | Purpose | |---|---| | `CIS_1.1_pass.sh` / `_fail.sh` | Install updates (pass); clear `LastFullSuccessfulDate` (fail — caveat: only works when real updates are pending) | | `CIS_1.6_pass.sh` / `_fail.sh` | Open/remove the `1.6.mobileconfig` profile | | `CIS_2.3.1.1_pass.sh` / `_fail.sh` | Open/remove the AirDrop profile | | `CIS_2.3.2.2_pass.sh` / `_fail.sh` | `launchctl load/unload -w` of `com.apple.timed.plist` | | `CIS_2.3.3.4_pass.sh` / `_fail.sh` | `systemsetup -setremotelogin off/on` (runs as MANUAL via the runner's SSH-breaking safeguard) | ### Existing scripts fixed | Script | Bug | Fix | |---|---|---| | `CIS_2.3.3.1.sh` | Disabled `com.apple.ODSAgent` (DVD/CD sharing), not Screen Sharing | Now disables `com.apple.screensharing` | | `CIS_2.9.2.sh` | `pmset -a womp 0` sets Wake-on-Network, not Power Nap | Now `pmset -a/-b/-c powernap 0` | | `CIS_3.2.sh` | `sed` pipeline into root-owned file via user redirection silently failed; did nothing if the flags line was missing | `awk` with `tee`/`mv`; appends a flags line when absent; enforces 0400 root:wheel | | `CIS_3.3.sh` | Only stripped `all_max=`; never added `ttl=365` when missing, so the query could never pass from a fresh system | `awk` now both strips `all_max=` and inserts/updates `ttl=365` on the `* file` line | | `CIS_3.4.sh` | `sudo sed … > /etc/security/audit_control` — redirect runs as caller, not root; write silently failed | Rewrites via `tee`/`mv` with proper perms; appends when line is absent | | `CIS_3.5.sh` | `chmod -R o-rw` doesn't produce the exact `0400`/`0440` modes the query requires | Explicit `chmod 0400` on `audit_control`, `find … -exec chmod 0440 {}` under `/var/audit/` | | `CIS_5.1.7.sh` | `sudo IFS=$'\n'` runs IFS in a subshell that exits immediately; searched `/System/Volumes/Data/Library` but the query looks at `/Library/%` | IFS set in parent shell; searches `/Library`; skips SIP-protected dirs via xattr | | `CIS_5.7.sh` | Wrote `use-login-window-ui` which the query doesn't accept | Writes `authenticate-session-owner` | | `CIS_6.3.6.sh` | Contained literal `<username>` placeholders that were never substituted | Iterates non-system users (`UniqueID >= 500`) and runs `defaults write` as each | <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * macOS 14 (Sonoma) CIS policies updated to v3.0.0 with refreshed policy names and CIS IDs. * New CLI test runner to automate CIS validation against macOS VMs. * **Bug Fixes / Improvements** * Updated remediations and audit/query logic; safer, atomic config updates; several policies revised or removed. * **Tests** * Many new and improved pass/fail helper scripts for validating CIS checks and profiles. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
5da912a33e |
Bugfix: escape characters not supported in JSON when resolving variables (#43955)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38013 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually See https://drive.google.com/file/d/1zeFNLuf_rT5FWzDiYyL2_hbIBW2neba-/view?usp=drive_link <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps variables in JSON configuration profiles (Apple DDM declarations and Android profiles) are now automatically escaped for JSON special characters, ensuring proper handling of sensitive values. * **Tests** * Added JSON configuration profile escaping validation to the enterprise GitOps integration test suite. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
fe66e3fd08 |
Don't refetch "change management" settings page on focus (#44054)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43977 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [ ] Added/updated automated tests - [X] QA'd all new/changed functionality manually - Repro'd issue on main - Verified on this branch that dirtying the form and then focusing elsewhere / refocusing on page doesn't lose changes. For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Bug Fixes * Fixed an issue where the Change Management form would reset upon losing and regaining page focus. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrew Mellor
|
2d687d94b3 |
41676 updated apple crs invalid domain error message (#43396)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41676 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [ x] Added/updated automated tests - [x ] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Apple MDM APNS certificate signing now shows a clear, domain-specific error when an unsupported email domain is supplied (applies to CSR requests and renewal flows), replacing the previous generic "invalid email" message. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |