imgbot
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nico
|
5a1365dc41 |
40493 webhooks for host activities (#50595)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40493 Changes already reviewed in the PRs merged to this feature branch. Only additive change was https://github.com/fleetdm/fleet/pull/50595/commits/c0934e1fee46a734f9499a4c782563d4fcc345c4 to address CodeRabbit's comments. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually https://github.com/user-attachments/assets/ea7f5157-a67a-4d83-842d-62197bd1546d ## New Fleet configuration settings - [ ] Setting(s) is/are explicitly excluded from GitOps If you didn't check the box above, follow this checklist for GitOps-enabled settings: - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary by CodeRabbit * **New Features** * Added host activity automations with configurable webhook destinations. * Manage automations from the Hosts page with validation, permissions, and enable/disable controls. * Added GitOps support for team and unassigned-host webhook settings. * Activity webhooks now include fleet-scoped host IDs where applicable. * Added profile UUIDs to MDM profile resend activity details. * **Bug Fixes** * Improved Windows MDM enrollment activity details by including the linked host ID when available. * Preserved existing webhook settings when omitted during updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
a4af4d896c |
Add default fleet for new Windows MDM enrollments (#41787) (#49922)
Demo: https://www.youtube.com/watch?v=cWxZlu9WuwA Guide updates: https://github.com/fleetdm/fleet/pull/49603/changes IT admins can configure the fleet that hosts enrolling through user-driven Windows MDM enrollment (Windows Autopilot, Entra join) are automatically assigned to, via the Windows MDM settings page, the mdm.windows_enrollment.default_fleet config setting, or GitOps. - New windows_enrollment_config row stores the default team; the config API surfaces it by fleet name and hydrates reads from the row so team renames and deletions never serve a stale name. Deleting the fleet clears the setting. - New edited_windows_enrollment_default_fleet activity, emitted only when the value changes. - The OMA-DM session persists the device-reported SMBIOS serial on still-unlinked enrollments, and orbit enrollment reverse-links by that serial and assigns the default fleet before orbit's one-shot setup-experience init, so the default fleet's software, scripts, and profiles apply during the Autopilot ESP. The DevDetail and osquery link paths keep the same assignment as fallbacks, and the EUA-token link path now shares the same post-link bookkeeping. - Hosts are only assigned when new to Fleet in this enrollment cycle: existing hosts, including ones parked in Unassigned, keep their fleet on re-enrollment, matching macOS ABM behavior. - GitOps defers applying the setting until teams declared in the same run are created, and fleetctl generate-gitops exports it. - Windows MDM settings page redesign per Figma: programmatic enrollment toggle, User driven enrollment section with the Entra-gated Default fleet dropdown, and a Migration section. <!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41787 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for assigning a default Fleet Premium fleet to new Windows MDM enrollments, including Autopilot and Entra join. * Default-fleet settings can be configured, cleared, and managed through Windows MDM settings and GitOps. * Assigned fleet software, scripts, and profiles can apply during out-of-box setup. * Added activity-feed visibility for default-fleet changes. * Improved Windows enrollment matching using hardware serial numbers. * **Documentation** * Documented default-fleet assignment for Windows enrollment. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
c83ecc2231 |
Match Windows software with version in name to FMA software title
Resolves #44406 Windows programs report a version in their name (e.g. `Granola 7.373.2`), so each version created its own `software_title` and never linked to the Fleet-maintained app installer's title (`Granola`), hiding the uninstall action. macOS handles this via `bundle_identifier`; Windows had no join key. - Give matching Windows programs the canonical FMA name at ingestion (name-prefix match), so all versions collapse onto the title the installer owns. `software.name` is unchanged. - Merge already-mismatched versioned titles onto the canonical title in `ReconcileMaintainedAppSoftwareNames` (runs on FMA sync; no migration needed). --------- Co-authored-by: Tim Lee <timlee@fleetdm.com> Co-authored-by: Juan Fernandez <juan@fleetdm.com> |
||
|
Sharon Katzandcopilot-swe-agent[bot]
|
5c127e5fe4 |
Fix software ingestion lock convoys and unbatched deletes (#49894)
**Related issue:** Resolves #49805, Resolves #48719 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually --- ## Context A customer (~2,500 hosts, v4.89.1) had their DB writer slammed with `DELETE FROM host_software_installed_paths` statements carrying 30,000+ IDs each. These never completed, required repeated manual intervention, and the table grew from 14.5M to 14.8M rows in 2 days. This is #49805. While investigating, Victor linked #48719, a related `software_titles` INSERT lock convoy issue seen in load tests. Both are in the same software ingestion code path (`server/datastore/mysql/software.go`), so this PR fixes both. ## Root cause ### #49805: Unbatched DELETEs on `host_software_installed_paths` When a host's software changes, Fleet computes a delta and deletes stale rows from `host_software_installed_paths`. The function `deleteHostSoftwareInstalledPaths()` issued a **single** `DELETE FROM host_software_installed_paths WHERE id IN (?)` with all IDs expanded by `sqlx.In()`. With 30,000+ IDs and 14.8M rows in the table, these massive statements held row locks for minutes, timed out, and never completed. On the next agent check-in, the same (or larger) DELETE was retried, creating a feedback loop where the table grew unboundedly. Notably, the INSERT function for the same table (`insertHostSoftwareInstalledPaths`) already batched at 500 rows. The DELETE simply lacked the same treatment. ### #48719: INSERT IGNORE lock convoys on `software_titles` (related) When a host reports software that Fleet hasn't seen before, `preInsertSoftwareInventory()` runs `INSERT IGNORE INTO software_titles (...)` inside a `withRetryTxx` transaction. For homogeneous fleets (many hosts sharing the same software catalog, typical for imaged corporate Windows machines), hundreds of concurrent goroutines try to INSERT IGNORE the same title rows simultaneously. Even though `INSERT IGNORE` is a no-op when the row already exists, InnoDB still acquires row/gap locks on the unique index for the duration of the enclosing transaction. With many goroutines holding or waiting on the same index locks, the DB enters a "lock convoy" where sessions serialize on locks they don't actually need. In load tests (40 Fleet instances, 100K hosts, 141 identical Windows software items), this produced 690 average active sessions on the writer and 85s fleet-wide p99. The existing read-first check (`getIncomingSoftwareChecksumsToExistingTitles`) prevents the convoy at steady state. But on cold start (empty `software_titles`, e.g. after cleanup purges orphaned titles), the check finds nothing and all goroutines race to INSERT the same titles. ## How I reproduced it Started MySQL via `docker compose up -d mysql_test`, created a git worktree. ### #49805 `TestHostSoftwareInstalledPathsDeleteExplosion`: Created a host with 500 software items and installed paths, then replaced all software with an entirely new set. This triggers `deleteHostSoftwareInstalledPaths()` with all 500 old IDs in a single unbatched DELETE statement. At 500 IDs the local test completes quickly, but the structure confirms the problem: at 30K+ IDs on production Aurora with 14M rows, these never finish. ### #48719 `TestSoftwareTitlesInsertIgnoreLockConvoy`: Created 50 hosts, each reporting 100 identical software items (simulating a homogeneous fleet). Used a barrier to release all 50 goroutines simultaneously, then measured two phases: 1. **Cold start** (empty `software_titles`): All 50 hosts concurrently call `ds.UpdateHostSoftware()`. 2. **Steady state** (titles exist): Same 50 hosts re-ingest. **Before fix:** | Metric | Cold start | Steady state | |--------|-----------|-------------| | Wall time | 3.0s | 38ms | | Avg per-host | 1,981ms | 29ms | | **Convoy factor** | **79x** | | The 79x slowdown confirms the lock convoy. ## How I fixed it ### #49805: Batch the DELETE at 500 Changed `deleteHostSoftwareInstalledPaths()` from a single `DELETE ... WHERE id IN (all IDs)` to a loop that processes 500 IDs per batch, matching the existing INSERT batching pattern in the same file. ### #48719: Three-layer defense against lock convoys **Layer 1 - Move title INSERT IGNORE outside the transaction.** Previously, `INSERT IGNORE INTO software_titles` ran inside `withRetryTxx`, so locks were held for the full transaction duration. Now each title INSERT is executed via `ds.writer(ctx).ExecContext()` outside any transaction, auto-committing independently and holding locks for microseconds. **Layer 2 - singleflight per title key.** Added a `singleflight.Group` on the `Datastore` struct. For each title, only one goroutine actually executes the INSERT; concurrent goroutines wait and share the result. **Layer 3 - In-process cache (`sync.Map`).** After a title is inserted, its key is stored in `knownSoftwareTitleKeys`. Subsequent ingestions check the cache first and skip the INSERT entirely. `CleanupSoftwareTitles` clears the cache when it deletes orphaned titles. The three layers work together: the cache handles the common case (title already known), singleflight handles the cold-start race (only one INSERT per title), and auto-commit ensures even the winning INSERT holds locks for microseconds. ## How I tested that it works ### New reproduction tests - `TestSoftwareTitlesInsertIgnoreLockConvoy`: 50 concurrent hosts, 100 identical software items. Measures cold-start convoy factor and verifies all 100 titles are created. - `TestHostSoftwareInstalledPathsDeleteExplosion`: Full software replacement path with 500 items per host, including concurrent hosts. ### Existing test suite Ran all existing software tests including: - `UpdateHostSoftware`, `UpdateHostSoftwareDeadlock`, `PreInsertSoftwareInventory` - `SoftwareTitleUpgradeCodeDriftMatch`, `UpdateHostSoftwareSameBundleIDDifferentNames` - `CleanupSoftwareTitles` (validates cache invalidation works correctly) - `SaveHost`, `SyncHostsSoftware`, and ~80 other subtests All pass. ### After-fix measurements | Metric | Before fix | After fix | |--------|-----------|-----------| | Cold-start wall (50 hosts) | ~3.0s | ~1.4s | | Cold-start avg per-host | ~1,981ms | ~594ms | | Steady-state wall | ~38ms | ~7ms | | Titles created correctly | 100/100 | 100/100 | The remaining cold-start time is from other pipeline operations (`INSERT IGNORE INTO software`, host_software linking), not from `software_titles`. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Performance Improvements** - Improved software inventory ingestion under large, concurrent workloads, including more efficient handling of repeated software-title inserts. - Reduced lock contention when many devices report the same titles at the same time. - Batched deletions of installed software-path records to speed up large updates. - **Bug Fixes** - Ensured deterministic, collation-safe software-title deduplication to prevent incorrect or stale title mapping. - Strengthened orphan cleanup behavior so caches are cleared when orphan titles are removed. - **Tests** - Added stress/regression tests for software-title insert contention and large installed-path delete workloads. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
Lucas Manuel Rodriguez
|
1ceca6ad8e |
Cleanup policy_membership stale entries in distributed/write (#48674)
Resolves #47241. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Better host policy results by automatically cleaning up out-of-scope `policy_membership` records. * Refreshes host failing-policy counts after cleanup, including when distributed writes report “no policies in scope.” * Preserves existing safeguards by skipping this cleanup during setup/initial configuration to prevent premature updates. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
abd38218fa |
Self service categories endpoints + activities (#46593)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #46391 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually |
||
|
Lucas Manuel Rodriguez
|
a1d91464ea |
Fix issue with permissions in host activity list for fleet-users (#46362)
**Related issue:** Resolves #46009. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Resolved an authorization issue preventing users from viewing past host activities on hosts that contained user-initiated operations such as lock, wipe, run script, or install software. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/46362?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
057e1615b4 |
Move mysql/testing_utils.go to a separate mysql/mysqltest package (#45406)
Resolves #45220 (one of several PRs to achieve removing "testing" package as dependency in production binary) ## Testing - [x] QA'd all new/changed functionality manually. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Switched many tests to use a dedicated MySQL test helper package and consolidated test-only utilities for datastore setup, cleanup, ad‑hoc SQL, certificate generation, and activity/aggregation helpers. * Added expanded test utilities for replication, DB connections and test data seeding to improve integration-test reliability. * **Chores** * No production behavior or user-facing APIs were changed. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45406) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
