imgbot
73
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jonathan Katz
|
6ef4ba3910 | Merge remote-tracking branch 'origin/main' into feat/39962-patch-when-closed | ||
|
Jonathan Katz
|
45abf8c9ad |
Add software installer upload/download progress to GitOps runs (#50250)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #45728 Changes: - Adds a new redis key to keep track of downloaded packages. It starts out with an empty list and gets filled with each download. Each update writes the entire struct at once to the key. - Adds logging in the fleetctl gitops client to show which packages were downloaded - Fixes the categories key potentially expiring # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - ❌ Timeouts are implemented and retries are limited to avoid infinite loops - Right now the batch will write the whole slice of all packages to a single redis key for every package in the loop. Looks like performance is acceptable for now (500 packages), but maybe this will need to be limited. - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## New Features - Added per-package software download progress in fleetctl GitOps. - Progress now reports downloading, completed, skipped, and failed packages during real and dry runs. - Installation output now distinguishes applying and applied stages. ## Bug Fixes - Improved download error messages and cached-package handling. - Prevented duplicate progress messages and ensured tracking issues do not interrupt successful software batches. ## Tests - Expanded coverage for progress reporting, failures, dry runs, package types, and authorization scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
0dc8c382c5 |
Add 'linux' as platform for labels (#50270)
Resolves #44088. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Linux as a label platform option. * Linux labels now apply across supported distributions, including Ubuntu, Debian, RHEL, CentOS, and generic Linux hosts. * Updated platform names for improved clarity and consistency. * **Bug Fixes** * Improved platform matching so Linux labels apply consistently to compatible hosts. * Removed the obsolete Zorin platform option. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
CarloandJonathan Katz
|
31a096e06c |
Add patch_when_closed to GitOps and generate-gitops (#49844)
**Related issue:** Resolves #49418 Adds `patch_when_closed` support to GitOps for patch policies and round-trips it through `fleetctl generate-gitops`. - Validate `patch_when_closed` in the patch-policy YAML: reject an explicit `continuous_automations_enabled: false` alongside it (GitOps is declarative — the datastore would otherwise silently force it on), auto-set it when omitted, and reject a `pre_install_query` on the referenced Fleet-maintained app (Fleet manages that query). - Emit `patch_when_closed` from `fleetctl generate-gitops`. # Checklist for submitter - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually ## New Fleet configuration settings Follow this checklist for GitOps-enabled settings: - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md) (#49022) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file --------- Co-authored-by: Jonathan Katz <yehonatankatz@gmail.com> |
||
|
Carlo DiCelico
|
06863c4b37 |
Merge remote-tracking branch 'origin/main' into feat/28108-multiple-custom-packages
# Conflicts: # frontend/pages/SoftwarePage/SoftwareTitleDetailsPage/LibraryItemAccordion/LibraryItemAccordion.tsx # frontend/pages/SoftwarePage/SoftwareTitleDetailsPage/SoftwareTitleDetailsPage.tsx # pkg/spec/gitops.go # server/datastore/mysql/schema.sql # server/fleet/software_installer.go |
||
|
|
078fbc0f40 |
Add Targeted platforms column and platform filter to Policies page (#44125)
- @noahtalerman: For the following quick win: - https://github.com/fleetdm/fleet/issues/23737 ## Summary Adds a "Targeted platforms" column and a platform filter dropdown to the Policies page (`/policies/manage`), matching the pattern already used on the Reports page (`/queries/manage`, `ManageQueriesPage`). ### Frontend - New non-sortable **Targeted platforms** column rendered via `PlatformCell`, sourced from each policy's comma-separated `platform` field. - New platform filter dropdown (All / macOS / Windows / Linux / ChromeOS) wired as a `customControl` on the Policies table, alongside the existing automation filter. Selecting a value pushes a new URL (not a replace), resets `page` to 0, and updates the `platform` query param. - `ManagePoliciesPage` reads `location.query.platform` and threads it through to both `globalPoliciesAPI.loadAll` / `teamPoliciesAPI.loadAll` and the react-query keys, plus the count endpoints. The automation-filter and count "hide" conditions now include the platform filter so they remain visible when only a platform filter is active. - `frontend/services/entities/global_policies.ts` and `team_policies.ts` accept an optional `platform` param (with `"all"` normalized to `undefined`). - Added tests for the new column and dropdown in `PoliciesTable.tests.tsx`. ### Backend - Added `Platform string ` + `` `query:"platform,optional"` `` to `ListGlobalPoliciesRequest`, `CountGlobalPoliciesRequest`, `ListTeamPoliciesRequest`, `CountTeamPoliciesRequest`. - Extended datastore and service signatures (`ListGlobalPolicies`, `ListTeamPolicies`, `ListMergedTeamPolicies`, `CountPolicies`, `CountMergedTeamPolicies`, `ListGlobalPolicies`/`ListTeamPolicies` on the service) to accept a `platform string` arg. Mocks and all call sites updated. - Platform filtering in SQL uses a new helper `platformFilterClause`: ```sql AND (p.platforms = '' OR FIND_IN_SET(?, p.platforms)) ``` so policies targeting "all platforms" (empty `platforms` field) always match regardless of the selected filter. `FIND_IN_SET` uses a bound parameter (no injection risk). - Added a new MySQL integration test `testPoliciesPlatformFilter` covering empty-platform (match-all), per-platform filter, and team/merged paths. ### Docs - REST API docs for `GET /api/v1/fleet/global/policies`, `GET /api/v1/fleet/fleets/:id/policies`, and the corresponding `/count` endpoints now document the `platform` query param. - Added `changes/policies-targeted-platforms-filter`. ## Behavior - `platform=all` (or missing) returns all policies. - Selecting a specific platform returns policies whose `platforms` column is empty OR contains the selected token. - The dropdown only renders when the table is searchable (results exist OR any filter is active). - Changing the filter pushes a new URL and resets the page. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] QA'd all new/changed functionality manually Local verification: - `go build ./...` — clean - `go vet ./server/... ./cmd/... ./ee/...` — clean - `make lint-go-incremental` — 0 issues - Go service-level policy tests pass. MySQL integration tests compile but could not be run locally (no Docker); CI will exercise the new `testPoliciesPlatformFilter` test. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a "Targeted platforms" column with platform icons and an "All platforms" option. * Added a platform filter dropdown to scope policy lists; counts, last-updated, and controls adapt when a platform filter is active. Backend now honors an optional platform query parameter so filtering returns matching policies. * **Tests** * Added and updated unit and integration tests covering the new column, filter UI, and platform-filtered policy listings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: nulmete <nicoulmete1@gmail.com> |
||
|
Jonathan Katz
|
7124b27187 |
Allow multiple packages - GitOps (#48710)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #48399 Summary: - Allows multiple installers for the same title to be defined in a yaml file - `generate-gitops` generates a file like this if multiple installers are available per title - Allows labels, self_service, categories keys to be defined per package - Inherits fleet-level keys only if they are not set at the package-level - Repoints policies.software_installer_id for a deleted installer to either the first added installer for that title, or NULL if none are available # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * GitOps output now supports software titles that contain multiple packages, generating a dedicated package file plus related assets. * Software imports and updates now preserve package order and handle multi-package titles more consistently. * **Bug Fixes** * Improved inheritance and validation for software fields so package-level settings are respected and conflicting settings are flagged. * Fixed installer batch updates to better handle added, removed, and reordered packages without disrupting related policies or pending installs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Nico
|
5e27628266 |
GitOps: combined include/exclude policy label targeting + labels_exclude_all (#33441) (#47505)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #46584 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. (Already added in main.) ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually `generate-gitops`: https://github.com/user-attachments/assets/d32e89c3-2ce7-4c57-9492-66deb0a3dfe8 `gitops`: https://github.com/user-attachments/assets/ac0e3935-bc8d-4541-b3e5-f992109630d9 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added `labels_exclude_all` field support for refining policy label scopes (available with Fleet Premium license). * **Bug Fixes** * Enhanced validation of policy label scope configurations to prevent invalid field combinations and enforce license requirements. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
5d59b0e627 |
Skip VPP label validation in dry runs (#46106)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #45844 Skips label validation against the database in dry runs, because if new ones are being applied in the same run then they wouldnt be in the db ahead of time. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Added a new label and vpp reference to it, both dry run and real run worked. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes **Bug Fixes** * GitOps dry runs no longer fail when a VPP app references a label that is introduced within the same run. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/46106?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
e447a685f0 |
Rename fleetctl's testing_utils.go to testing_utils_test.go and create separate test package (#45585)
Resolves #45220 (one of several PRs, we are very close) ## Testing - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved test infrastructure for the CLI: consolidated and renamed test helpers, added a dedicated in-process CLI test helper, and updated many test cases to use the new helpers. * Tightened several test assertions and standardized output/error validation across unit and integration tests to improve reliability. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45585) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
057e1615b4 |
Move mysql/testing_utils.go to a separate mysql/mysqltest package (#45406)
Resolves #45220 (one of several PRs to achieve removing "testing" package as dependency in production binary) ## Testing - [x] QA'd all new/changed functionality manually. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Switched many tests to use a dedicated MySQL test helper package and consolidated test-only utilities for datastore setup, cleanup, ad‑hoc SQL, certificate generation, and activity/aggregation helpers. * Added expanded test utilities for replication, DB connections and test data seeding to improve integration-test reliability. * **Chores** * No production behavior or user-facing APIs were changed. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45406) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
393007ede1 |
First move of server/service to server/service/svctest to prevent testing code in Fleet's production binary (#45221)
**Related issue:** Resolves #45220 (one of many small PRs) ## Testing - [x] QA'd all new/changed functionality manually. (Ran basic smoke testing.) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a consolidated test utilities package to boot full test servers, seed users, retrieve auth tokens, and provide common mocks for integration/service tests. * Updated integration and service tests to use the new helpers for more consistent, reliable test setup and wiring. * **Chores** * Centralized and reorganized test configuration types and options, and standardized test wiring for clearer, maintainable test setup. [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45221) <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
d439cb1690 |
Fix missing deleted_policy activity for auto-cleaned patch policies (#45045)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44286 Unset `patch_software_title_id` rather than deleting the policy in `BatchSetSoftwareInstallers`, so the orphaned policy gets picked up by the `policiesToDelete` loop in `server/service/client.go:3121`. As a result, the `deleted_policy` activity is now created properly, and gitops dry/real runs also report the deletion: ``` dry run: [-] would've deleted policy macOS - 010 Editor up to date [-] would've deleted 1 policy real run: [-] deleting policy macOS - 010 Editor up to date [-] deleting 1 policy [-] deleted 1 policy ``` # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed missing deletion activity logs when patch policies are removed via GitOps so policy deletion events are now recorded. * **Behavior Changes** * Batch-updating installers now retains obsolete patch policies but clears their patch installer reference instead of deleting the policy. * **Tests** * Added integration coverage to verify deletion activities are emitted and installer batch behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
b9933f45a2 |
Fix gitops 500 when software title icon bytes are missing (#44735)
Fixes #43511 |
||
|
Sharon Katz
|
beca71e674 |
Fix gitops dry-run to catch manual_agent_install + macos_script conflict (#44432)
**Related issue:** Resolves #34464 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually --- ## What GitOps `--dry-run` was succeeding when `macos_manual_agent_install` was set to `true` and a `macos_script` was configured under `setup_experience`, but the actual GitOps run would fail with: ``` Couldn't add setup experience script. To add script, first disable macos_manual_agent_install. ``` ## Why The `manual_agent_install` conflict validation only existed server-side in `ee/server/service/setup_experience.go:SetSetupExperienceScript()`. The script upload call (`uploadMacOSSetupScript()`) was gated by `!opts.DryRun` in `server/service/client.go`, so during dry-run the upload was skipped entirely and the validation never fired. ## Fix Added client-side validation in `server/service/client.go` at the point where the YAML-parsed `MacOSSetup` struct is processed — before the script file is validated and loaded. This check runs for **both dry-run and real runs**, catching the conflict early. Two code paths were fixed: 1. **Team path** (~line 803): Checks `setup.ManualAgentInstall.Value` when `setup.Script.Value` is set 2. **No-team path** (~line 2603): Checks `macOSSetup.ManualAgentInstall.Value` when `macOSSetup.Script.Value` is set ## How I reproduced the issue locally ### Prerequisites - MySQL and Redis running via Docker: `docker compose up -d mysql_test redis` ### Steps 1. Wrote an integration test (`TestDryRunMacOSSetupScriptWithManualAgentInstallConflict`) that: - Creates a GitOps user and fleetctl config - Creates a bootstrap package server serving `testdata/signed.pkg` - Creates a `.sh` script file with `echo "setup script"` - Creates a **global config** YAML (minimal server settings) - Creates a **team config** YAML with `macos_manual_agent_install: true`, `macos_script: <path>`, and `macos_bootstrap_package: <url>` - Runs `fleetctl gitops --dry-run` and asserts it fails - Runs `fleetctl gitops` (no dry-run) and asserts it fails 2. Ran the test **before the fix** — confirmed the bug: ``` Dry-run error: <nil> ← BUG: should have failed Real run error: ...status 422...first disable macos_manual_agent_install ← correctly fails ``` 3. Applied the fix and re-ran — **both dry-run and real run now fail** with the `macos_manual_agent_install` conflict error. ### Test command ```bash MYSQL_TEST=1 REDIS_TEST=1 go test -v \ -run TestIntegrationsEnterpriseGitops/TestDryRunMacOSSetupScriptWithManualAgentInstallConflict \ ./cmd/fleetctl/integrationtest/gitops/... -count=1 -timeout 600s ``` Both sub-tests (team and no-team paths) pass. All related existing tests continue to pass: - `TestMacOSSetup`, `TestMacOSSetupScriptWithFleetSecret`, `TestDeletingNoTeamYAML`, `TestDisallowSoftwareSetupExperience` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps dry-run now correctly fails when a macOS setup configuration combines manual agent installation with a provided setup script, preventing false-positive dry-run success. * **Tests** * Added unit and integration regression tests to verify dry-run and real-run rejection of conflicting macOS setup configurations for both team-scoped and unassigned host scopes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
ce5640c99e | Prevent silent corruption of software title icons (#44540) | ||
|
Lucas Manuel Rodriguez
|
1e4a9f292f |
Add activities for user actions on labels (#44522)
Resolves #36976 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Label operations (create, edit, delete) now generate activities shown in the activity feed with label and optional fleet context. * Host label add/remove operations emit corresponding label edited activities; duplicate label names are deduplicated. * Label activity types are selectable/filterable in the activity dashboard. * **Tests** * Added unit, integration, and UI tests covering label activity emission, rendering, filtering, and GitOps label lifecycle scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
ba0f6b3c72 |
Update GitOps for managed local account fields (#44058)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42948 - Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup) SetDefaultsIfNeeded()` to account for new fields - Updated default creation and editing for team edit/creation paths - Updated `generate-gitops` warning message from `macos_setup` to `setup_experience` - Updated fields types to optjson and updated test files # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Team edit and team creation through GitOps, validated config with ` curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'` - New error message says `setup_experience` instead of `macos_setup` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added macOS MDM settings to control local account behavior: enable managed local accounts (default false) and specify end-user local account type (default "admin") for fleet and team configs. GitOps output now highlights unsupported setup-experience cases. * **Tests** * Updated fixtures and integration tests to assert and persist the new macOS local-account settings across config, team, and GitOps scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
5da912a33e |
Bugfix: escape characters not supported in JSON when resolving variables (#43955)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38013 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually See https://drive.google.com/file/d/1zeFNLuf_rT5FWzDiYyL2_hbIBW2neba-/view?usp=drive_link <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps variables in JSON configuration profiles (Apple DDM declarations and Android profiles) are now automatically escaped for JSON special characters, ensuring proper handling of sensitive values. * **Tests** * Added JSON configuration profile escaping validation to the enterprise GitOps integration test suite. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
2a8803884b |
DDMV: Support Fleet variables in DDM (#43222)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43047 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually See https://github.com/fleetdm/fleet/issues/42960#issuecomment-4244206563 and subsequent comments. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Apple DDM declarations support a vetted subset of Fleet variables with per-host substitution; premium license required. Declaration tokens and resend behavior now reflect variable changes; unresolved host substitutions mark that host’s declaration as failed. * **Bug Fixes** * Clearer errors for unsupported or license-restricted Fleet variables and more consistent DDM resend/update semantics when variables change. * **Tests** * Added extensive unit and integration tests covering Fleet variable validation, substitution, token changes, resends, and failure states. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
3ae98ee01d |
Clean up Gitops tests and add deprecation tests (#43039)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #40015 * Moves repeated empty mocks into a new `setupEmptyGitOpsMocks` method * Adds new "deprecation" tests: * In TestGitOpsFullGlobal, TestGitOpsFullTeam and TestGitOpsFullGlobalAndTeam tests "kitchen sink" with both new and deprecated keys * Added keys and checks to verify `setup_experience`, `apple_business_manager` and `volume_purchasing_program` configs * Consolidated map of deprecated -> new GitOps keys in one place |
||
|
Scott Gress
|
1eabb85a5a |
Activate deprecation warnings (#41449)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40015 # Details Activates deprecation warnings for old API params and CLI args, updates tests that would generate warnings (except for tests explicitly designed to generate warnings). The expectation from here on is that Fleet UI usage should not generate any deprecation warnings in the server logs, nor should the output from `generate-gitops` generate any warnings when fed into `gitops`. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually - [X] clicked around in an mdm-enabled instance, turned setup experience features on and off, saw no server warnings - [X] did `fleetctl generate-gitops` on mdm-enabled instance, saw no server or cli warnings - [X] did `fleetctl gitops` on mdm-enabled instance, saw no server or cli warnings |
||
|
Scott Gress
|
6598b608b7 |
Enforce GitOps exceptions (#42191)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42180 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enhanced GitOps exception handling for labels, secrets, and software with clearer enforcement and omission semantics. * Server-side prefetch of team software so omitted team software can preserve existing installers during validation. * Presence flags track whether top-level keys (labels, secrets, software) were provided versus omitted. * **Behavior Changes** * Omitted vs empty sections are now distinguished: omission can mean “no-op” or “delete-all” depending on exception settings. * GitOps YAML can define and manage labels directly; validations now reject YAML that includes keys marked as excepted. <!-- end of auto-generated comment: release notes by coderabbit.ai --> # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually * **Labels** - [ ] Validated that with label exceptions off, omitting `labels:` key from default.yml clears all global labels - [ ] Validated that with label exceptions off, omitting `labels:` key from a fleet .yml clears all labels for that fleet - [ ] Validated that with label exceptions off, setting empty `labels:` key from default.yml clears all global labels - [ ] Validated that with label exceptions off, setting empty `labels:` key from a fleet .yml clears all labels for that fleet - [ ] Validated that with label exceptions on, omitting `labels:` key from default .yml leaves existing global labels as-is - [ ] Validated that with label exceptions on, omitting `labels:` key from a fleet .yml leaves existing labels as-is - [ ] Validated that with label exceptions on, setting `labels:` key on default .yml generates an error - [ ] Validated that with label exceptions on, setting `labels:` key on a fleet .yml generates an error - [ ] Validated that with label exceptions on, a policy using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, a query using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, an MDM profile using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, a software package using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) - [ ] Validated that with label exceptions on, an app store app using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) - [ ] Validated that with label exceptions on, a fleet maintained app using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) * **Secrets** - [ ] Validated that with secrets exceptions off, omitting `secrets:` key from default.yml clears all global secrets - [ ] Validated that with secrets exceptions off, omitting `secrets:` key from a fleet .yml clears all secrets for that fleet - [ ] Validated that with secrets exceptions on, omitting `secrets:` key from default .yml leaves existing global secrets as-is - [ ] Validated that with secrets exceptions on, omitting `secrets:` key from a fleet .yml leaves existing secrets as-is - [ ] Validated that with secrets exceptions on, setting `secrets:` key on default .yml generates an error - [ ] Validated that with secrets exceptions on, setting `secrets:` key on a fleet .yml generates an error * **Software** - [ ] Validated that with software exceptions off, omitting `software:` key from no-team.yml/unassigned.yml clears all software for "no team" - [ ] Validated that with software exceptions off, omitting `software:` key from a fleet .yml clears all software for that fleet - [ ] Validated that with software exceptions off, setting empty `software:` key on a fleet .yml clears all software for that fleet - [ ] Validated that with software exceptions off, setting empty `software:` key on no-team.yml/unassigned.yml clears all software for "no team - [ ] Validated that with software exceptions on, omitting `software:` key from a fleet .yml leaves existing software as-is - [ ] Validated that with software exceptions on, setting `software:` key on a fleet .yml generates an error - [ ] Validated that with software exceptions on, omitting `software:` key from no-team.yml/unassigned.yml leaves existing software as-is for "no team" - [ ] Validated that with software exceptions on, setting `software:` key on no-team.yml/unassigned.yml generates an error - [ ] Validated that with software exceptions on, a policy using `install_software.hash_sha256` referencing an existing package succeeds without `software:` key - [ ] Validated that with software exceptions on, a policy using `install_software.app_store_id` referencing an existing VPP app succeeds without `software:` key - [ ] Validated that with software exceptions on, a patch policy using `fleet_maintained_app_slug` referencing an existing FMA succeeds without `software:` key - [ ] Validated that with software exceptions on, `setup_experience.software` referencing existing software succeeds without `software:` key (server-side validation fallback) - [ ] Validated that with software exceptions on, omitting `software:` from no-team.yml/unassigned.yml preserves existing no-team software - [ ] Validated that with software exceptions on, a policy in no-team.yml/unassigned.yml using `install_software.hash_sha256` referencing existing no-team software succeeds without `software:` key For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results I don't think so. There is a bit of overhead when this feature is used since we have to fetch software from the server, but it would be done in a specific test, so even if there is an impact it should affect existing load testing, only new, specific tests. |
||
|
Scott GressandCopilot Autofix powered by AI
|
40e91c0ece |
Allow hosts key to be empty for manual labels (#42022)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41672 # Details Updates GitOps label functionality so that omitting the `hosts:` key under a manual label will _not_ clear hosts from that label, but will instead preserve the existing membership. This allows users to manage manual hosts with an external system (via the labels API), while still managing the labels themselves in GitOps. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - [X] verified that you can still add a manual label with `hosts:` - [X] verified that leaving `hosts:` off a manual label doesn't change the host assignment - [X] verified that putting `hosts:` with no value on a manual label clears the hosts - [X] verified that you can still add a dynamic label - [X] verified that generate-gitops still exports manual hosts --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
George Karr
|
99f8639484 |
feat installers labels include all (#42061)
- **DB migration: Add `require_all` column to installers labels tables (#41279)** - **Backend: Support labels_include_all for installers/apps (#41324)** - **ui impl for labels include all (#41836)** <!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #39916 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled |
||
|
Magnus Jensen
|
a8c9e261d7 |
speed up macOS profile delivery for initial enrollments (#41960)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #34433 It speeds up the cron, meaning fleetd, bootstrap and now profiles should be sent within 10 seconds of being known to fleet, compared to the previous 1 minute. It's heavily based on my last PR, so the structure and changes are close to identical, with some small differences. **I did not do the redis key part in this PR, as I think that should come in it's own PR, to avoid overlooking logic bugs with that code, and since this one is already quite sized since we're moving core pieces of code around.** # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Faster macOS onboarding: device profiles are delivered and installed as part of DEP enrollment, shortening initial setup. * Improved profile handling: per-host profile preprocessing, secret detection, and clearer failure marking. * **Improvements** * Consolidated SCEP/NDES error messaging for clearer diagnostics. * Cron/work scheduling tuned to prioritize Apple MDM profile delivery. * **Tests** * Expanded MDM unit and integration tests, including DeclarativeManagement handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jahziel Villasana-Espinoza
|
02a9eb8769 | merge main | ||
|
Martin AngersandJahziel Villasana-Espinoza
|
ba04887100 |
Backend: Support labels_include_all for installers/apps (#41324)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40721 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually I (Martin) did test `labels_include_all` for FMA, custom installer, IPA and VPP apps, and it seemed to all work great for gitops apply and gitops generate, **except for VPP apps** which seem to have 2 important pre-existing bugs, see https://github.com/fleetdm/fleet/issues/40723#issuecomment-4041780707 ## New Fleet configuration settings - [ ] Verified that the setting is exported via `fleetctl generate-gitops` - [ ] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [ ] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [ ] Verified that any relevant UI is disabled when GitOps mode is enabled --------- Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com> |
||
|
Victor Lyuboslavsky
|
902b4af289 |
Fixed GitOps failing to delete a certificate authority (#41693)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38036 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps now correctly orders operations so certificate authorities can be removed only after referencing certificate templates are handled, preventing failed deletions during config updates. * Improved user-facing error when a CA cannot be deleted because certificate templates still reference it, with guidance to remove templates first. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
01d13f5080 |
add keymap for new renames, and shallow duplication (#41682)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #41091 # Details This PR finishes the work of aliasing multi-platform keys by: * Added the renames to the list maintained by generate-gitops so that `fleetctl get` can use the new names * Updated the code that adds the new names to API and `fleetctl get` output to only add new nested keys under new parents, e.g. add `apple_settings.configuration_profiles`, but not `macos_settings.configuration_profiles`. The API key duplicator now runs through `RewriteDeprecatedKeys` which is a little heavier per-token, but for old keys we're doing less work so I think this ends up being slightly more performant than before, at least for large payloads. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a, changelog for new keys added in previous PR ## Testing - [X] Added/updated automated tests updated tests for the duplicators - [X] QA'd all new/changed functionality manually - [X] `/config` and `/fleets` APIs now only return new keys under new parents - [X] `fleetctl get fleets` now returns new multiplatform keys |
||
|
|
2abacc577e |
Feat/31914 patch policy (#41518)
Implements patch policies #31914 - https://github.com/fleetdm/fleet/pull/40816 - https://github.com/fleetdm/fleet/pull/41248 - https://github.com/fleetdm/fleet/pull/41276 - https://github.com/fleetdm/fleet/pull/40948 - https://github.com/fleetdm/fleet/pull/40837 - https://github.com/fleetdm/fleet/pull/40956 - https://github.com/fleetdm/fleet/pull/41168 - https://github.com/fleetdm/fleet/pull/41171 - https://github.com/fleetdm/fleet/pull/40691 - https://github.com/fleetdm/fleet/pull/41524 - https://github.com/fleetdm/fleet/pull/41674 --------- Co-authored-by: Jonathan Katz <44128041+jkatz01@users.noreply.github.com> Co-authored-by: jkatz01 <yehonatankatz@gmail.com> Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com> Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com> |
||
|
Scott Gress
|
056e567bab |
Implement webhooks_and_tickets_enabled flag for policies in GitOps (#41183)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40627 # Details This PR updates the way we enable failed policy reporting (via webhook or ticket integration) for individual policies in GitOps. The existing method is to declare a `policy_ids` key underneath `failing_policies_webhook:` in either the global or a fleet .yml file, and specify a list of policy IDs to enable the automation for. This PR maintains this feature for backwards compatibility, and adds a new feature where you can set `webhook_and_tickets_enabled: true` key in the policy declaration itself. If _both_ these methods are used, the GitOps run will fail. **Implementation note:** Because we're keeping the old way of doing this until Fleet 5, I took the easy route and just translated the new way into the old way; that is, we gather up the list of policies with `webhook_and_tickets_enabled: true`, get their IDs and send that list to the server under the same config we did previously. This works fine and there's nothing _wrong_ with it but ideally this flag would work the same as other per-policy flags like `calendar_events_enabled` that are stored on the policy record. That requires a migration and more new code that we'd have to maintain alongside the existing code (or translate the old strategy to the new one). I'm taking the lower-touch path here. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - [x] Verified that `generate-gitops` outputs the new `webhooks_and_tickets_enabled` flag instead of outputting `policy_ids` under `failing_policies_webhook` - [X] Verified that using the new flag in a fleet .yml file results in the specified policies being enabled in the "other" automations for policies (whether the webhook automation is enabled or not) - [X] Verified the same for a global .default.yml file - [X] Verified that using the old `failing_policies_webhook.policy_ids` a fleet .yml file results in the specified policies being enabled in the "other" automations for policies (whether the webhook automation is enabled or not) - [X] Verified the same for a global .default.yml file - [X] Verified that trying to use both `webhooks_and_tickets_enabled` and `failing_policies_webhook.policy_ids` at the same time results in an error. ## New Fleet configuration settings - [ ] Setting(s) is/are explicitly excluded from GitOps If you didn't check the box above, follow this checklist for GitOps-enabled settings: - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [ ] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) see https://github.com/fleetdm/fleet/issues/40627#issuecomment-4024988552 - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added configuration flag to enable webhooks and tickets for policies in GitOps settings. * System automatically resolves and assigns policy IDs when using the new flag. * **Tests** * Added comprehensive test coverage for webhook and ticket enablement in GitOps workflows, including conflict detection and policy ID assignment validation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
63be71fd72 |
require controls on either global or no-team (#41350)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41307 # Details * Fixes a potential issue where running `fleetctl gitops` with only the global file, with no controls provided, could wipe out global controls that are provided in the "no team" file. * Fixes error message when controls are missing. # Checklist for submitter ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - [x] `fleetctl gitops -f /path/to/default.yml` without controls, gives `error: 'controls' must be set on global config, no-team.yml or unassigned.yml` - [x] `fleetctl gitops -f /path/to/default.yml` with empty controls works - [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/no-team.yml` without controls, gives `error: 'controls' must be set on global config or no-team.yml` - [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/unassigned.yml` without controls, gives `error: 'controls' must be set on global config or unassigned.yml` - [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/no-team.yml` with empty controls in no-team.yml works - [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/unassigned.yml` with empty controls in unassigned.yml works - [x] `fleetctl gitops -f /path/to/no-team.yml` gives error `global config must be provided alongside no-team.yml` - [x] `fleetctl gitops -f /path/to/no-team.yml` gives error `global config must be provided alongside unassigned.yml` - [x] `fleetctl gitops -f /path/to/some-real-team.yml` with no controls works For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results |
||
|
Scott Gress
|
9c4d5ce97e |
Make most GitOps top-level optional (#41138)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41012 # Details This PR makes it allowable to leave out almost all top-level keys from GitOps files. The only required keys are _either_ `name:` (for a fleet settings file) or `org_settings:` (for a global settings file). Omitting a key is identical to supplying it with no value. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests Updated the "missing all global keys test", and added some new tests to verify that omitting the key was the same as supplying it with an empty value. - [X] QA'd all new/changed functionality manually 1. Ran `fleetctl generate-gitops` to get a clean set of GitOps yml files 2. Removed all removable keys from default.yml and ran `fleetctl gitops` 3. Ran `fleetctl generate-gitops` again into a different dir 4. Ran `fleetctl gitops` with the original files to get back to original state 5. Cleared out all now-removable keys and replaced them with empty value (e.g. `reports:` with nothing under it) 6. Ran `fleetctl generate-gitops` again into a third dir 7. Compared the files from the second and third generate-gitops runs to verify that omitting the key had the same result as supplying it with an empty value 8. Did the above steps with a fleet (i.e. non-global) .yml file. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * GitOps files now support omitting top-level configuration keys instead of requiring them to be explicitly set to empty values. * org_settings is now required when team name is not specified. * **Tests** * Added integration tests validating behavior when omitting top-level keys in global and team-level GitOps configurations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
1fa339298b | Bugfix: gitops policy linked to software package with env var fails to apply (#40944) | ||
|
Magnus Jensen
|
b4b27d0d6f |
avoid double encoding $FLEET_SECRET in GitOps (#40866)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40108 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed double encoding of secret environment variables when configured through GitOps, ensuring secrets are stored with proper escaping. * **Tests** * Added test coverage for configuration profile escaping to verify proper handling of secret variables and API keys during GitOps operations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Magnus Jensen
|
861d0ee122 |
don't allow manual_agent_install without bootstrap package (#40812)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38432 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually |
||
|
Victor Lyuboslavsky
|
62186cb6bd |
Final slog migration PR: test infrastructure + tools + remaining standalone files (#40727)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40540 go-kit/log is no longer a direct dependency; moved kitlog adapter required for some 3rd party libraries into its own package # Checklist for submitter - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - Present in previous PR ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Modernized logging across the codebase: switched from legacy logging wrappers to Go's standard slog, updated adapters, tests, tools, and server components. * Threaded the new slog logger through test utilities and tooling; adjusted a small number of logging-related function/constructor signatures to accept the new logger type (minor compatibility updates). <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Victor Lyuboslavsky
|
4dfdc870bd |
slog migration: service layer + subsystem libraries (#40661)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40540 # Checklist for submitter - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - Changes present in previous PR ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Refactor** * Updated internal logging infrastructure to improve consistency and maintainability across the application. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
4305ca1840 |
Fix output from gitops to use "fleets" instead of "teams", "queries" instead of "reports" (#40610)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40621 # Details This PR updates the output from `fleetctl gitops` and `fleetctl apply` to use the correct terminology: * "fleet" instead of "team" * "report" instead of "query" (where appropriate) * "for unassigned hosts" in place of "No Team" where possible, and "for fleet Unassigned" otherwise. All changes other than tests are in `client.go` and are text-only; no functional code is changed (and no code relies on the strings besides tests). # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually I did a `fleetctl generate-gitops` and a `fleetctl gitops` run and saw the updated logs: ``` [+] would've applied EULA [+] would've applied certificate authorities [+] would've applied fleet config [+] would've applied MDM profiles [+] would've applied enroll secrets [+] would've applied 1 report [+] would've applied 1 software package for fleet 'Unassigned' [+] would've applied 0 app store apps for fleet 'Unassigned' [+] would've applied webhook settings for unassigned hosts [+] would've applied 1 policy [!] gitops dry run succeeded ``` but I did not go through and try and replicate every log or error message. I think the best we can do on this one is eyeball the code changes for mistakes. |
||
|
Jonathan Katz
|
60152bac45 |
Followup for enforcing manual agent install restrictions in gitops (#40503)
**Related issue:** Resolves #40412 ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually - Individually or together setting vpp, software installers, or fleet maintained apps will send the correct error now when applied to no team with manual_agent_install enabled. |
||
|
Victor Lyuboslavsky
|
c14bea44de |
Replaced all kitlog.Logger instances with the intermediate *logging.Logger (#40425)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40054 # Checklist for submitter - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - Changes included in previous PR ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Refactor** * Consolidated and standardized internal logging infrastructure across the application by adopting a unified logging package throughout the codebase, replacing previous external logging dependencies. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Konstantin Sykulev
|
8757d365bc |
Revert "Added deny list for checking external user submitted urls"
This reverts commit
|
||
|
Scott Gress
|
772fb12cf5 |
Add more deprecation logs and mute by default (#40305)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40122 # Details * Adds deprecation warnings to `fleetctl apply` * Adds alias conflict errors (i.e. using both new and deprecated keys in the same spec) to `fleetctl apply` * Adds logic around all deprecated field warnings to check the topic first * Disables deprecation warnings by default for `fleet serve`, `fleetctl gitops` and `fleetctl apply` * Enables deprecation warnings for dogfood via env var To turn on warnings: * In `fleet serve`, use either `--logging_enable_topics=deprecated-field-names` or the `FLEET_LOGGING_ENABLE_TOPICS=deprecated-field-names` env var * In `fleetctl gitops` / `fleetctl apply` use either `--enable-log-topics=deprecated-field-names` or `FLEET_ENABLE_LOG_TOPICS=deprecated-field-names` # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually tested in `fleetctl apply`, `fleet serve` and `fleet gitops` that warnings are suppressed by default and added when the appropriate env var or CLI option is used |
||
|
Scott Gressandkiloconnect[bot]
|
34e7b5c358 |
Deprecate "team" and "query" API params (#39873)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #39344 # Details This PR builds on the previous PR (https://github.com/fleetdm/fleet/pull/39847) which added `renameto` tags to certain API parameters to mark them as deprecated. How this is used: ### In requests * When decoding requests, log a warning if a `json` or `query` param is used that has a `renameto` tag, e.g. if a `team_id` param is sent but the related struct has `renameto:"fleet_id"` in it. * If the `renamedto` version (e.g. `fleet_id`) is sent in the request, rewrite it to the deprecated name so that it can be unmarshalled into the struct * If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an error and quit * URLs with deprecated terms have new aliases using `WithAltPaths` -- warning on using old URLSs a TODO that will be handled in a subsequent PR. ### In responses * Output _both_ the deprecated and new names for fields that have `renameto` tags, so that we don't break existing workflows expecting the old keys. Uses a shared `DuplicateJSONKeys` to do the duplication. * Most API responses are handled in `EncodeCommonResponse`. Exceptions are activities, failing policy webhooks and the streaming "list hosts" endpoints which call the function directly. ### In fleetctl * Similar to requests, log warnings when deprecated keys are used and rewrite the new keys internally so that they can be unmarshalled. * For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the new names * The set of keys to replace is hardcoded in `fleetctl` rather than being dynamically generated as it is for API endpoints. Given the mixture of typed and untyped data and the level of nesting, dynamic map generation was very fragile and error-prone. ### Performance considerations * The biggest performance hit is the addition of the JSON key rewriter to the request pipeline. The rewriter buffers the entire request into memory before eventually passing it to the decoder than unmarshals the data into structs. I tried implementing this as a true streaming rewriter but encountered issues where the request would hang if the downstream reader (the decoder) encountered any errors. It's possible we could implement this in a streaming fashion if we replace our [current request decoder](https://github.com/fleetdm/fleet/blob/da43bf8371695382c4af0972d5da456c6b94bdaf/server/service/endpoint_utils.go#L108) with the v2 version, which is a bigger change requiring more thoughtful discussion in the engineering team. As it stands, memory usage for requests with deprecated fields will double while the request is being decoded. * The "alias rules" used to determine the old and new key names are cached per struct type and for most endpoints are generated on server start, so no performance impact is expected. * Some `fleetctl` commands may have an extra unmarshal/marshal step but as these are user-initiated and not performed in tight loops, the impact should be minimal. ### TODO * Log deprecation warnings when old URLs like "/fleet/teams" are used * Update API fields that the front-end uses to avoid deprecation warnings * Update `fleetctl apply` to accept/return `kind: fleet` rather than `kind: team` * Find/update any fleet server config vars with old language * Update all error messages that use old language # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually * Clicking around the front-end, no broken pages due to request ingestion errors or bad responses * Looking in network tab to verify that responses have both the old and new keys * Running `fleetctl generate-gitops` and verifying that the output looks correct and can be ingested by `fleetctl gitops` * Running `fleetctl get` and `fleetctl apply` --------- Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com> |
||
|
Konstantin Sykulev
|
3d4a3e1b87 |
Added deny list for checking external user submitted urls (#39947)
This PR changes 3 things. 1. Validate `admin_url` + all URLs for HTTPS/non-private 2. Add custom `DialContext` hook in fleethttp.NewClient(), this is needed for DNS-rebinding protection at connection time 3. Validate Smallstep SCEP challenge endpoint # **IMPORTANT** There are two validations occurring. 1. `CheckURLForSSRF` 2. `SSRFDialContext` ## Why? `CheckURLForSSRF` checks the hostname. It resolves DNS, validates the ip, and then returns an error to the user. It protects certificate authority create/update API endpoints. But then `GetSmallstepSCEPChallenge` calls `http.NewRequest(http.MethodPost, ca.ChallengeURL, ...)` with the original hostname This is where `SSRFDialContext` comes into play. It fires when an actual HTTP request is attempted. Meaning Fleet would first build the request, encode the body, set up TLS, etc., before being blocked at the dial. `CheckURLForSSRF` stops the operation before any of that work happens. `SSRFDialContext` protects the actual challenge fetch that happens later at enrollment time. They're not always called together. The dial-time check is the only thing protecting the enrollment request and DNS rebinding. ## Should we remove `CheckURLForSSRF` This is debatable and I don't have a strong opinion. Removing `CheckURLForSSRF` would still provide the same protection. However, it would return a generic connection error from the HTTP client which would make it slightly hard to diagnose why it is broken. ## What's next I implemented this for certificate authorities. I am sure there are other places in the code base that take user submitted urls and could also use this check. That is outside the scope of this particular PR. But worthy to investigate in the near future. If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Security** * Added SSRF protections for validating external URLs and blocking private/IP-metadata ranges; dev mode can bypass checks for local testing * **New Features** * Introduced an SSRF-protected HTTP transport and an option to supply a custom transport per client * **Tests** * Added comprehensive tests covering SSRF validation, dialing behavior, and resolution edge cases <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
544ba92114 |
Enforce GitOps setup experience when manual agent install is enabled (#39775)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38431 Attempts to get the ManualAgentInstall config from `ds.TeamLite` in the case of software installers. In the VPP case, it attempts to get it from either `ds.TeamByName` or from global `ds.AppConfig` if no team name is provided. I had to add some mock functions for TeamLite that were missing, or missing for the team 0 case. Also added FLEET_INTEGRATION_TESTS_DISABLE_LOG to disable logs in gitops integration tests. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually |
||
|
Noah Talerman
|
52f8ba3c72 |
fleet-gitops tests: Remove Workstations (canary) enroll secret (#39550)
- Workstations (canary) team no longer exists: https://github.com/fleetdm/fleet-gitops/tree/main/teams |
||
|
Nico
|
e5849ee720 |
Show Manage Automations disabled button with tooltip on Queries page (#39302)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #39303 (child of #25080). - Added `inherited_query_count` to `ListQueriesResponse` (thought of adding a brand new endpoint just for counting, but felt like extending the current one was good enough). In the parent task, [it was suggested](https://github.com/fleetdm/fleet/issues/25080#issuecomment-3326071574) to `"Depend on team list entity endpoint's count field / team entity count endpoint for whether or not to disable the manage automations button"`, which Rachael approved, so I went for this approach. - The `ManageQueryAutomationsModal` now fetches its own data with `merge_inherited = false` (meaning it only fetches non-inherited queries only). Previously, queries were passed down as props to it, which would not show the queries available to automate if the first page of queries were all inherited and the second page contained queries for that team (the user would have to navigate to the second page for the button to be enabled). ^ The fact that the modal fetches its own data is similar behavior to what is currently done in `Policies`. For queries, I noticed that we would need to add pagination within the `Manage Automations` modal, but that can be a follow-up. <img width="2480" height="1309" alt="Screenshot 2026-02-04 at 11 48 42 AM" src="https://github.com/user-attachments/assets/ebac79a5-a793-4708-9313-d9a697dfd7de" /> # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] QA'd all new/changed functionality manually https://github.com/user-attachments/assets/119f03b9-dde1-4bb9-9fee-6204b1a58879 |
||
|
Jordan Montgomery
|
a251e8e766 |
Fix broken test (#39498)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves # no ticket however More context here: https://fleetdm.slack.com/archives/C019WG4GH0A/p1770295767905349 And here https://fleetdm.slack.com/archives/C019WG4GH0A/p1770389045550669 # Checklist for submitter If some of the following don't apply, delete the relevant line. ## Testing - [x] Added/updated automated tests |
