imgbot
107
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jonathan Katz
|
6ef4ba3910 | Merge remote-tracking branch 'origin/main' into feat/39962-patch-when-closed | ||
|
Jonathan Katz
|
45abf8c9ad |
Add software installer upload/download progress to GitOps runs (#50250)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #45728 Changes: - Adds a new redis key to keep track of downloaded packages. It starts out with an empty list and gets filled with each download. Each update writes the entire struct at once to the key. - Adds logging in the fleetctl gitops client to show which packages were downloaded - Fixes the categories key potentially expiring # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - ❌ Timeouts are implemented and retries are limited to avoid infinite loops - Right now the batch will write the whole slice of all packages to a single redis key for every package in the loop. Looks like performance is acceptable for now (500 packages), but maybe this will need to be limited. - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## New Features - Added per-package software download progress in fleetctl GitOps. - Progress now reports downloading, completed, skipped, and failed packages during real and dry runs. - Installation output now distinguishes applying and applied stages. ## Bug Fixes - Improved download error messages and cached-package handling. - Prevented duplicate progress messages and ensured tracking issues do not interrupt successful software batches. ## Tests - Expanded coverage for progress reporting, failures, dry runs, package types, and authorization scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
0dc8c382c5 |
Add 'linux' as platform for labels (#50270)
Resolves #44088. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Linux as a label platform option. * Linux labels now apply across supported distributions, including Ubuntu, Debian, RHEL, CentOS, and generic Linux hosts. * Updated platform names for improved clarity and consistency. * **Bug Fixes** * Improved platform matching so Linux labels apply consistently to compatible hosts. * Removed the obsolete Zorin platform option. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
a442d7af3a |
Python script-only packages: follow-on QA fixes (#50143)
**Related issues:** Resolves #50068, Resolves #50106, Resolves #50107, Resolves #50108, Resolves #50110, Resolves #50114 Follow-on fixes from QA of #41470 (Python script-only packages): - Software-installer validation errors are action-neutral, so the Add and Edit flows each show the correct single verb, and the unsupported-file error names a content/format mismatch instead of blaming the extension (#50068, #50107). - `.py` packages accept `setup_experience_platform` (`darwin`/`linux`), matching `.sh` (#50106). - A failed-to-run install script (exit code `-1`) now renders a diagnostic instead of empty output, and orbit surfaces the underlying execve error (#50108). - The install-rejection message for `.sh`/`.py` packages says "macOS and Linux hosts" instead of "linux" (#50110). - Orbit writes each script's temp file with an extension matching its shebang (`.py`/`.sh`/`.ps1`), so tracebacks reference the right file type (#50114). # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually ## fleetd/orbit/Fleet Desktop - [x] If the change applies to only one platform, confirmed that `runtime.GOOS` is used as needed to isolate changes. - [x] Verified compatibility with the latest released version of Fleet (orbit-only change; the server↔agent `SoftwareInstallDetails` contract is unchanged). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved installer validation and rejection messaging for unsupported/invalid package contents (including correcting “add” vs “edit” wording and avoiding duplicated phrasing). * Added clearer diagnostics when install scripts fail to start (including empty output cases). * Corrected handling of script-only packages so Python scripts use the proper script type/extension, reducing misleading tracebacks. * Updated platform availability messaging so `.sh`/`.py` packages display macOS+Linux support. * **New Features** * Python script-only packages can now specify macOS and Linux setup experience platforms. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
CarloandJonathan Katz
|
31a096e06c |
Add patch_when_closed to GitOps and generate-gitops (#49844)
**Related issue:** Resolves #49418 Adds `patch_when_closed` support to GitOps for patch policies and round-trips it through `fleetctl generate-gitops`. - Validate `patch_when_closed` in the patch-policy YAML: reject an explicit `continuous_automations_enabled: false` alongside it (GitOps is declarative — the datastore would otherwise silently force it on), auto-set it when omitted, and reject a `pre_install_query` on the referenced Fleet-maintained app (Fleet manages that query). - Emit `patch_when_closed` from `fleetctl generate-gitops`. # Checklist for submitter - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually ## New Fleet configuration settings Follow this checklist for GitOps-enabled settings: - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md) (#49022) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file --------- Co-authored-by: Jonathan Katz <yehonatankatz@gmail.com> |
||
|
Carlo
|
57bab9e5ec |
Allow Python script-only packages (#49070)
**Related issue:** Resolves #41470 Adds support for uploading Python (`.py`) script-only software packages — accepted as script-only (the file contents become the install script; advanced options and automatic install follow `.sh`/`.ps1`), assigned the new `py_packages` source, and installable on macOS and Linux hosts across the UI, REST API, and GitOps. Feature branch combining the backend (#48942) and frontend (#48946) sub-PRs. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for Python (`.py`) script-only software packages across UI uploads, API/self-service installs, and GitOps parsing. * Python installers now derive metadata correctly and render the proper Python icon, with install eligibility for macOS & Linux. * **Bug Fixes** * Improved installer-script validation and “supported file types” error messages to include `.py` (and consistent handling of related script fields/options). * **Tests** * Expanded unit, integration, and GitOps tests to cover Python package parsing, metadata derivation, platform/host eligibility, and UI rendering. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo DiCelico
|
4ce133cb26 | merge main | ||
|
Nico
|
b10642be3c |
44954 custom host vitals (#49334)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44954 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Custom Host Vitals management (create/edit/delete, search) with copyable variable tokens and per-host values. * Show Custom Host Vitals on host details, including role-based editing. * Enabled Custom Host Vitals in host-vitals labels and the activity feed. * Extended GitOps to manage global Custom Host Vitals declaratively (including dry-run behavior). * Split Controls → Variables into Global Variables and Custom Host Vitals, including routing updates. * **Bug Fixes** * Improved validation and expansion of Custom Host Vital references across scripts, profiles, installers, and deployments, with clearer failures when values are missing or invalid. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo DiCelico
|
06863c4b37 |
Merge remote-tracking branch 'origin/main' into feat/28108-multiple-custom-packages
# Conflicts: # frontend/pages/SoftwarePage/SoftwareTitleDetailsPage/LibraryItemAccordion/LibraryItemAccordion.tsx # frontend/pages/SoftwarePage/SoftwareTitleDetailsPage/SoftwareTitleDetailsPage.tsx # pkg/spec/gitops.go # server/datastore/mysql/schema.sql # server/fleet/software_installer.go |
||
|
|
078fbc0f40 |
Add Targeted platforms column and platform filter to Policies page (#44125)
- @noahtalerman: For the following quick win: - https://github.com/fleetdm/fleet/issues/23737 ## Summary Adds a "Targeted platforms" column and a platform filter dropdown to the Policies page (`/policies/manage`), matching the pattern already used on the Reports page (`/queries/manage`, `ManageQueriesPage`). ### Frontend - New non-sortable **Targeted platforms** column rendered via `PlatformCell`, sourced from each policy's comma-separated `platform` field. - New platform filter dropdown (All / macOS / Windows / Linux / ChromeOS) wired as a `customControl` on the Policies table, alongside the existing automation filter. Selecting a value pushes a new URL (not a replace), resets `page` to 0, and updates the `platform` query param. - `ManagePoliciesPage` reads `location.query.platform` and threads it through to both `globalPoliciesAPI.loadAll` / `teamPoliciesAPI.loadAll` and the react-query keys, plus the count endpoints. The automation-filter and count "hide" conditions now include the platform filter so they remain visible when only a platform filter is active. - `frontend/services/entities/global_policies.ts` and `team_policies.ts` accept an optional `platform` param (with `"all"` normalized to `undefined`). - Added tests for the new column and dropdown in `PoliciesTable.tests.tsx`. ### Backend - Added `Platform string ` + `` `query:"platform,optional"` `` to `ListGlobalPoliciesRequest`, `CountGlobalPoliciesRequest`, `ListTeamPoliciesRequest`, `CountTeamPoliciesRequest`. - Extended datastore and service signatures (`ListGlobalPolicies`, `ListTeamPolicies`, `ListMergedTeamPolicies`, `CountPolicies`, `CountMergedTeamPolicies`, `ListGlobalPolicies`/`ListTeamPolicies` on the service) to accept a `platform string` arg. Mocks and all call sites updated. - Platform filtering in SQL uses a new helper `platformFilterClause`: ```sql AND (p.platforms = '' OR FIND_IN_SET(?, p.platforms)) ``` so policies targeting "all platforms" (empty `platforms` field) always match regardless of the selected filter. `FIND_IN_SET` uses a bound parameter (no injection risk). - Added a new MySQL integration test `testPoliciesPlatformFilter` covering empty-platform (match-all), per-platform filter, and team/merged paths. ### Docs - REST API docs for `GET /api/v1/fleet/global/policies`, `GET /api/v1/fleet/fleets/:id/policies`, and the corresponding `/count` endpoints now document the `platform` query param. - Added `changes/policies-targeted-platforms-filter`. ## Behavior - `platform=all` (or missing) returns all policies. - Selecting a specific platform returns policies whose `platforms` column is empty OR contains the selected token. - The dropdown only renders when the table is searchable (results exist OR any filter is active). - Changing the filter pushes a new URL and resets the page. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] QA'd all new/changed functionality manually Local verification: - `go build ./...` — clean - `go vet ./server/... ./cmd/... ./ee/...` — clean - `make lint-go-incremental` — 0 issues - Go service-level policy tests pass. MySQL integration tests compile but could not be run locally (no Docker); CI will exercise the new `testPoliciesPlatformFilter` test. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a "Targeted platforms" column with platform icons and an "All platforms" option. * Added a platform filter dropdown to scope policy lists; counts, last-updated, and controls adapt when a platform filter is active. Backend now honors an optional platform query parameter so filtering returns matching policies. * **Tests** * Added and updated unit and integration tests covering the new column, filter UI, and platform-filtered policy listings. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: nulmete <nicoulmete1@gmail.com> |
||
|
Jonathan Katz
|
7124b27187 |
Allow multiple packages - GitOps (#48710)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #48399 Summary: - Allows multiple installers for the same title to be defined in a yaml file - `generate-gitops` generates a file like this if multiple installers are available per title - Allows labels, self_service, categories keys to be defined per package - Inherits fleet-level keys only if they are not set at the package-level - Repoints policies.software_installer_id for a deleted installer to either the first added installer for that title, or NULL if none are available # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * GitOps output now supports software titles that contain multiple packages, generating a dedicated package file plus related assets. * Software imports and updates now preserve package order and handle multi-package titles more consistently. * **Bug Fixes** * Improved inheritance and validation for software fields so package-level settings are respected and conflicting settings are flagged. * Fixed installer batch updates to better handle added, removed, and reordered packages without disrupting related policies or pending installs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Nico
|
5e27628266 |
GitOps: combined include/exclude policy label targeting + labels_exclude_all (#33441) (#47505)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #46584 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. (Already added in main.) ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually `generate-gitops`: https://github.com/user-attachments/assets/d32e89c3-2ce7-4c57-9492-66deb0a3dfe8 `gitops`: https://github.com/user-attachments/assets/ac0e3935-bc8d-4541-b3e5-f992109630d9 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added `labels_exclude_all` field support for refining policy label scopes (available with Fleet Premium license). * **Bug Fixes** * Enhanced validation of policy label scope configurations to prevent invalid field combinations and enforce license requirements. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Steven Palmesano
|
20c0963331 |
Fix GitOps when using All fleets in VPP settings (#46855)
**Related issue:** Resolves #46824 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed an issue where volume purchasing program assignments failed when "All fleets" was selected; validation and token assignment now treat the "All fleets" label consistently, preventing errors during configuration application. * **Tests** * Added an end-to-end GitOps test case verifying "All fleets" is supported for volume purchasing program entries. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
e34126ab3a |
Merge branch 'main' of github.com:fleetdm/fleet into feat/39018-self-service-categories
Bump migration, fix failing test and nilaway check |
||
|
Scott Gress
|
07df7c5cfd |
Track software deletions in GitOps (#46764)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43729 # Details Adds output to GitOps runs indicating which custom/FMA software packages would be deleted. This involves adding a `deleted_packages` key to the `/software/batch/:request_uuid` ("Get status of software batch-apply request") API, which will be documented separately. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - [X] verified that a GitOps dry run produces one "would've deleted" line per custom package / fma that would be deleted - [X] verified that a GitOps real run produces one "deleted" line per custom package / fma that was deleted - [X] verified that adding software is unaffected <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * GitOps batch software operations now report packages pending deletion: dry-runs show "would've deleted" warnings and real runs show deletions; apply flows surface per-package deletion messages. * Empty payload dry-run now still reports pending deletions when applicable. * **Tests** * Added integration and datastore tests validating deletion-warning output, pending-deletion detection, and related result handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
a3338d032e |
Self service categories - GitOps support (#46671)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #46392 A few things in this PR: - updated the conversion from old default category to the new ones with the emoji included that was introduced in the feature branch. It takes into account what exists in the database now so if an admin wants to add for example "Productivity" without the emoji as a category it won't get overwritten. - updated a few places to ignore missing categories rather than error (what we do for adding a single FMA currently) - updated permissions for "gitops" users - added everything needed for gitops, generate-gitops support using the existing endpoints from the last PR. Didn't add logs like "[+] applied X self service categories" since it wasn't mentioned in the docs, but wouldn't be too hard to add. # Checklist for submitter ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [ ] Verified that any relevant UI is disabled when GitOps mode is enabled - Currently missing, at least on this branch |
||
|
Scott Gress
|
2bd7fec8a7 |
Handle edge case of adding new fleet + vpp at the same time (#46533)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44444 # Details This PR fixes the following edge case when running `fleetctl gitops`: 1. A new fleet is added 2. That fleet is declared as an ABM default fleet and/or a fleet in a VPP token location 3. The _other_ fleets declared as ABM defaults or VPP fleets are _not_ all provided in the GitOps run In that case, the GitOps run would fail with an error that one of the previously-existing fleets could not be found. This PR fixes the bug by making sure that GitOps looks at both the currently-persisted fleets (via the API) and any fleets that are being created in the current run. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - Reproduced the issue on `main` by attempting to create a new fleet _and_ add it both as an ABM default fleet and to the set of VPP token users in a single run, and getting an error about one of the existing fleets not being found - Verified that I was able to complete a gitops run successfully on this branch with a new fleet as a VPP token user and a default ABM fleet <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Bug Fixes** * Improved validation for Apple Business Manager and Volume Purchasing Program token team assignments with clearer error messages when referenced teams aren't found in Fleet. * Enhanced team name matching to properly handle Unicode characters, ensuring consistent team identification across GitOps configurations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
abd38218fa |
Self service categories endpoints + activities (#46593)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #46391 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually |
||
|
Jonathan Katz
|
5d59b0e627 |
Skip VPP label validation in dry runs (#46106)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #45844 Skips label validation against the database in dry runs, because if new ones are being applied in the same run then they wouldnt be in the db ahead of time. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Added a new label and vpp reference to it, both dry run and real run worked. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes **Bug Fixes** * GitOps dry runs no longer fail when a VPP app references a label that is introduced within the same run. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/46106?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
e447a685f0 |
Rename fleetctl's testing_utils.go to testing_utils_test.go and create separate test package (#45585)
Resolves #45220 (one of several PRs, we are very close) ## Testing - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved test infrastructure for the CLI: consolidated and renamed test helpers, added a dedicated in-process CLI test helper, and updated many test cases to use the new helpers. * Tightened several test assertions and standardized output/error validation across unit and integration tests to improve reliability. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45585) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
057e1615b4 |
Move mysql/testing_utils.go to a separate mysql/mysqltest package (#45406)
Resolves #45220 (one of several PRs to achieve removing "testing" package as dependency in production binary) ## Testing - [x] QA'd all new/changed functionality manually. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Switched many tests to use a dedicated MySQL test helper package and consolidated test-only utilities for datastore setup, cleanup, ad‑hoc SQL, certificate generation, and activity/aggregation helpers. * Added expanded test utilities for replication, DB connections and test data seeding to improve integration-test reliability. * **Chores** * No production behavior or user-facing APIs were changed. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45406) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jonathan Katz
|
fd7bf3118b |
Fix TestGitOpsTeamVPPAppleConfiguration failing (#45305)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves # # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Updated test fixtures for Apple VPP configuration in GitOps workflows to better reflect real-world scenarios. [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45305) <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Lucas Manuel Rodriguez
|
393007ede1 |
First move of server/service to server/service/svctest to prevent testing code in Fleet's production binary (#45221)
**Related issue:** Resolves #45220 (one of many small PRs) ## Testing - [x] QA'd all new/changed functionality manually. (Ran basic smoke testing.) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a consolidated test utilities package to boot full test servers, seed users, retrieve auth tokens, and provide common mocks for integration/service tests. * Updated integration and service tests to use the new helpers for more consistent, reliable test setup and wiring. * **Chores** * Centralized and reorganized test configuration types and options, and standardized test wiring for clearer, maintainable test setup. [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45221) <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carloandjkatz01
|
55636b3ef6 |
iOS/iPadOS managed config: GitOps (#43969) (#44933)
Part of #38790. Stacked on top of #44932. Closes #43969. `fleetctl gitops` reads `configuration.path` for iOS / iPadOS VPP apps and in-house `.ipa` apps, validates the file via `fleet.ValidateAppleAppConfiguration`, and stores the bytes through the service layer. `fleetctl generate-gitops` emits `configuration.path` for apps that have a stored configuration, writing the XML to `lib/<team>/software/<app-slug>-config.xml`. Apps without configuration omit the block entirely. Round-trip integration tests (`cmd/fleetctl/integrationtest/gitops/software_test.go`) cover apply + generate for both VPP and in-house, plus a negative case for invalid configuration content. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added managed app configuration support for enterprise software deployments through GitOps * Configurations now support platform-specific formats: XML for iOS/iPadOS, JSON for Android * Configuration validation ensures compatibility and prevents invalid variable references * **Tests** * Added integration tests for configuration handling in software deployment workflows [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/44933) <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: jkatz01 <yehonatankatz@gmail.com> |
||
|
|
fc907c4634 |
Rename Apple Business 'Location' to 'Organization Unit' in UI and docs (#44833)
## Summary - Renames Apple Business "Location" / "Locations" to "Organization Unit" / "Organization Units" across user-facing UI strings, documentation, and code comments - Updates the VPP table column header, modal text, error messages, audit log descriptions, YAML configuration docs, and article guides - Underlying API field names (`location` in JSON) and YAML config keys remain unchanged for backward compatibility --- Built for [Mel Pike](https://fleetdm.slack.com/archives/D0AKX7DJFCN/p1778077861693279?thread_ts=1777299502.461149&cid=D0AKX7DJFCN) by [Kilo for Slack](https://kilo.ai/slack) --------- Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com> Co-authored-by: melpike <melpike.dev@gmail.com> Co-authored-by: Copilot <copilot@github.com> Co-authored-by: melpike <79950145+melpike@users.noreply.github.com> |
||
|
Jonathan Katz
|
d439cb1690 |
Fix missing deleted_policy activity for auto-cleaned patch policies (#45045)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44286 Unset `patch_software_title_id` rather than deleting the policy in `BatchSetSoftwareInstallers`, so the orphaned policy gets picked up by the `policiesToDelete` loop in `server/service/client.go:3121`. As a result, the `deleted_policy` activity is now created properly, and gitops dry/real runs also report the deletion: ``` dry run: [-] would've deleted policy macOS - 010 Editor up to date [-] would've deleted 1 policy real run: [-] deleting policy macOS - 010 Editor up to date [-] deleting 1 policy [-] deleted 1 policy ``` # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Fixed missing deletion activity logs when patch policies are removed via GitOps so policy deletion events are now recorded. * **Behavior Changes** * Batch-updating installers now retains obsolete patch policies but clears their patch installer reference instead of deleting the policy. * **Tests** * Added integration coverage to verify deletion activities are emitted and installer batch behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Marko LisicaandCarlo
|
e988dd4756 |
Support VPP apps from non-US App Store regions (#44368)
**Related issue:** Resolves #43846 --------- Co-authored-by: Carlo <1778532+cdcme@users.noreply.github.com> |
||
|
Carlo
|
b9933f45a2 |
Fix gitops 500 when software title icon bytes are missing (#44735)
Fixes #43511 |
||
|
Sharon Katz
|
beca71e674 |
Fix gitops dry-run to catch manual_agent_install + macos_script conflict (#44432)
**Related issue:** Resolves #34464 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually --- ## What GitOps `--dry-run` was succeeding when `macos_manual_agent_install` was set to `true` and a `macos_script` was configured under `setup_experience`, but the actual GitOps run would fail with: ``` Couldn't add setup experience script. To add script, first disable macos_manual_agent_install. ``` ## Why The `manual_agent_install` conflict validation only existed server-side in `ee/server/service/setup_experience.go:SetSetupExperienceScript()`. The script upload call (`uploadMacOSSetupScript()`) was gated by `!opts.DryRun` in `server/service/client.go`, so during dry-run the upload was skipped entirely and the validation never fired. ## Fix Added client-side validation in `server/service/client.go` at the point where the YAML-parsed `MacOSSetup` struct is processed — before the script file is validated and loaded. This check runs for **both dry-run and real runs**, catching the conflict early. Two code paths were fixed: 1. **Team path** (~line 803): Checks `setup.ManualAgentInstall.Value` when `setup.Script.Value` is set 2. **No-team path** (~line 2603): Checks `macOSSetup.ManualAgentInstall.Value` when `macOSSetup.Script.Value` is set ## How I reproduced the issue locally ### Prerequisites - MySQL and Redis running via Docker: `docker compose up -d mysql_test redis` ### Steps 1. Wrote an integration test (`TestDryRunMacOSSetupScriptWithManualAgentInstallConflict`) that: - Creates a GitOps user and fleetctl config - Creates a bootstrap package server serving `testdata/signed.pkg` - Creates a `.sh` script file with `echo "setup script"` - Creates a **global config** YAML (minimal server settings) - Creates a **team config** YAML with `macos_manual_agent_install: true`, `macos_script: <path>`, and `macos_bootstrap_package: <url>` - Runs `fleetctl gitops --dry-run` and asserts it fails - Runs `fleetctl gitops` (no dry-run) and asserts it fails 2. Ran the test **before the fix** — confirmed the bug: ``` Dry-run error: <nil> ← BUG: should have failed Real run error: ...status 422...first disable macos_manual_agent_install ← correctly fails ``` 3. Applied the fix and re-ran — **both dry-run and real run now fail** with the `macos_manual_agent_install` conflict error. ### Test command ```bash MYSQL_TEST=1 REDIS_TEST=1 go test -v \ -run TestIntegrationsEnterpriseGitops/TestDryRunMacOSSetupScriptWithManualAgentInstallConflict \ ./cmd/fleetctl/integrationtest/gitops/... -count=1 -timeout 600s ``` Both sub-tests (team and no-team paths) pass. All related existing tests continue to pass: - `TestMacOSSetup`, `TestMacOSSetupScriptWithFleetSecret`, `TestDeletingNoTeamYAML`, `TestDisallowSoftwareSetupExperience` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps dry-run now correctly fails when a macOS setup configuration combines manual agent installation with a provided setup script, preventing false-positive dry-run success. * **Tests** * Added unit and integration regression tests to verify dry-run and real-run rejection of conflicting macOS setup configurations for both team-scoped and unassigned host scopes. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
ce5640c99e | Prevent silent corruption of software title icons (#44540) | ||
|
Lucas Manuel Rodriguez
|
1e4a9f292f |
Add activities for user actions on labels (#44522)
Resolves #36976 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Label operations (create, edit, delete) now generate activities shown in the activity feed with label and optional fleet context. * Host label add/remove operations emit corresponding label edited activities; duplicate label names are deduplicated. * Label activity types are selectable/filterable in the activity dashboard. * **Tests** * Added unit, integration, and UI tests covering label activity emission, rendering, filtering, and GitOps label lifecycle scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Carlo
|
07e4e7afe6 |
Short-circuit for empty software config in Gitops dry run (#44405)
Fixes #42607 |
||
|
Jonathan Katz
|
ba0f6b3c72 |
Update GitOps for managed local account fields (#44058)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42948 - Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup) SetDefaultsIfNeeded()` to account for new fields - Updated default creation and editing for team edit/creation paths - Updated `generate-gitops` warning message from `macos_setup` to `setup_experience` - Updated fields types to optjson and updated test files # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [ ] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually - Team edit and team creation through GitOps, validated config with ` curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'` - New error message says `setup_experience` instead of `macos_setup` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added macOS MDM settings to control local account behavior: enable managed local accounts (default false) and specify end-user local account type (default "admin") for fleet and team configs. GitOps output now highlights unsupported setup-experience cases. * **Tests** * Updated fixtures and integration tests to assert and persist the new macOS local-account settings across config, team, and GitOps scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
5da912a33e |
Bugfix: escape characters not supported in JSON when resolving variables (#43955)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38013 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually See https://drive.google.com/file/d/1zeFNLuf_rT5FWzDiYyL2_hbIBW2neba-/view?usp=drive_link <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps variables in JSON configuration profiles (Apple DDM declarations and Android profiles) are now automatically escaped for JSON special characters, ensuring proper handling of sensitive values. * **Tests** * Added JSON configuration profile escaping validation to the enterprise GitOps integration test suite. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
91d9b25924 |
Allow conditional downloads across fleets (#43679)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43417 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. done in https://github.com/fleetdm/fleet/pull/42216 ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - Using a local fileserver, added the same software to two fleets and ran `fleetctl gitops`. Verified that the first fleet downloaded the file, the second fleet used the cache, and both fleet showed the software installer in the UI. ## Summary by CodeRabbit * **Chores** * Updated software installer lookup mechanism to support optional team-scoped searches, enabling fallback to cross-team installer cache when team-specific installers are unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Martin Angers
|
2a8803884b |
DDMV: Support Fleet variables in DDM (#43222)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43047 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually See https://github.com/fleetdm/fleet/issues/42960#issuecomment-4244206563 and subsequent comments. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Apple DDM declarations support a vetted subset of Fleet variables with per-host substitution; premium license required. Declaration tokens and resend behavior now reflect variable changes; unresolved host substitutions mark that host’s declaration as failed. * **Bug Fixes** * Clearer errors for unsupported or license-restricted Fleet variables and more consistent DDM resend/update semantics when variables change. * **Tests** * Added extensive unit and integration tests covering Fleet variable validation, substitution, token changes, resends, and failure states. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
aff440236e |
Add cache option for software packages to skip re-downloading unchanged content (#42216)
**Related issue:** Ref #34797 Ref #42675 ## Problem When a software installer spec has no `hash_sha256`, Fleet re-downloads the package, re-extracts metadata, and re-upserts the DB on every GitOps run, even if the upstream file hasn't changed. For deployments with 50+ URL-only packages across multiple teams, this wastes bandwidth and processing time on every run. ## Solution By default, use etags to avoid unnecessary downloads: 1. First run: Fleet downloads the package normally and stores the server's ETag header 2. Subsequent runs: Fleet sends a conditional GET with `If-None-Match`. If the server returns 304 Not Modified, Fleet skips the download, metadata extraction, S3 upload, and DB upsert entirely Opt-out with `always_download:true`, meaning packages continue to be downloaded and re-processed on every run, same as today. No UI changes needed. ```yaml url: https://nvidia.gpcloudservice.com/global-protect/getmsi.esp?version=64&platform=windows always_download: true install_script: path: install.ps1 ``` ### Why conditional GET instead of HEAD Fleet team [analysis of 276 maintained apps](https://github.com/fleetdm/fleet/pull/42216#issuecomment-4105430061) showed 7 apps where HEAD requests fail (405, 403, timeout) but GET works for all. Conditional GET eliminates that failure class: if the server doesn't support conditional requests, it returns 200 with the full body, same as today. ### Why opt-in 5 of 276 apps (1.8%) have stale ETags (content changes but ETag stays the same), caused by CDN caching artifacts (CloudFront, Cloudflare, nginx inode-based ETags). The `cache` key lets users opt in per package for URLs where they've verified ETag behavior is correct. Validation rejects `always_download: true` when hash_sha256` is set ## Changes - New YAML field: `cache` (bool, package-level) - New migration: `http_etag` VARCHAR(512) column (explicit `utf8mb4_unicode_ci` collation) + composite index `(global_or_team_id, url(255))` on `software_installers` - New datastore method: `GetInstallerByTeamAndURL` - `downloadURLFn` accepts optional `If-None-Match` header, returns 304 as `(resp, nil, nil)` with `http.NoBody` - ETag validated per RFC 7232 (ASCII printable only, no control chars, max 512 bytes) at both write and read time - Cache skipped for `.ipa` packages (multi-platform extraInstallers) - TempFileReader and HTTP response leak prevention on download retry - Docs updated in `yaml-files.md` ## What doesn't change - Packages with `hash_sha256`: existing hash-based skip, untouched - FMA packages: FMA version cache, untouched - Packages with `always_download: true`: identical to current behavior - Fleet UI: no changes ## Test plan Automated testing: - [x] 16 unit tests for `validETag` - [x] 8 unit tests for conditional GET behavior (304, 200, 403, 500, weak ETag, S3 multipart, no ETag) - [x] MySQL integration test for `GetInstallerByTeamAndURL` - [x] All 23 existing `TestSoftwareInstallers` datastore tests pass - [x] All existing service tests pass Manual testing: - [x] E2E: 86 packages across 6 CDN patterns, second apply shows 51 conditional hits (304) - [x] @sgress454 used a local fileserver tool to test w/ a new instance and dummy packages <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * ETag-based conditional downloads to skip unchanged remote installer files. * New always_download flag to force full re-downloads. * **Tests** * Added integration and unit tests covering conditional GETs, ETag validation, retries, edge cases, and payload behavior. * **Chores** * Persist HTTP ETag and related metadata; DB migration and index to speed installer lookups. * Added installer lookup by team+URL to support conditional download flow. * **Bug Fix** * Rejects using always_download together with an explicit SHA256 in uploads. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Scott Gress <scott@fleetdm.com> Co-authored-by: Scott Gress <scott@pigandcow.com> Co-authored-by: Ian Littman <iansltx@gmail.com> |
||
|
Scott Gress
|
3ae98ee01d |
Clean up Gitops tests and add deprecation tests (#43039)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #40015 * Moves repeated empty mocks into a new `setupEmptyGitOpsMocks` method * Adds new "deprecation" tests: * In TestGitOpsFullGlobal, TestGitOpsFullTeam and TestGitOpsFullGlobalAndTeam tests "kitchen sink" with both new and deprecated keys * Added keys and checks to verify `setup_experience`, `apple_business_manager` and `volume_purchasing_program` configs * Consolidated map of deprecated -> new GitOps keys in one place |
||
|
Scott Gress
|
1eabb85a5a |
Activate deprecation warnings (#41449)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40015 # Details Activates deprecation warnings for old API params and CLI args, updates tests that would generate warnings (except for tests explicitly designed to generate warnings). The expectation from here on is that Fleet UI usage should not generate any deprecation warnings in the server logs, nor should the output from `generate-gitops` generate any warnings when fed into `gitops`. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually - [X] clicked around in an mdm-enabled instance, turned setup experience features on and off, saw no server warnings - [X] did `fleetctl generate-gitops` on mdm-enabled instance, saw no server or cli warnings - [X] did `fleetctl gitops` on mdm-enabled instance, saw no server or cli warnings |
||
|
Scott Gress
|
c4aa6f5529 |
Use fleetctl new templates for new instances (#42768)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41409 # Details This PR updates the `ApplyStarterLibrary` method and functionality to rely on the same templates and mechanisms as `fleetctl new`. The end result is that running `fleetctl new` and `fleetctl gitops` on a new instance should be a no-op; no changes should be made. Similarly, changing the templates in a Fleet release will automatically affect `fleetctl new` and `ApplyStarterLibrary` in the same exact way for that release. > Note that this moves the template files out of `fleetctl` and into their own shared package. This move comprises the majority of the file changes in the PR. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests Note that <img width="668" height="44" alt="image" src="https://github.com/user-attachments/assets/066cd566-f91d-4661-84fc-2aabbfce2ef9" /> will fail until the 4.83 Fleet docker image is published, since it's trying to push 4.83 config (including `exceptions`) to a 4.82 server. - [X] QA'd all new/changed functionality manually - [X] Created a new instance and validated that the fleets, policies and labels created matched the ones created by `fleetctl new` - [X] Ran `fleetctl new` and verified that it created the expected folders and files - [X] Ran `fleetctl gitops` with the files created by `fleetctl new` and verified that the instance was unchanged. - [X] Ran `fleetctl preview` successfully using a dev build of the Fleet server image (since it won't work against the latest published build, which doesn't support `exceptions`). Verified it shows the expected teams, policies and labels |
||
|
Carlo
|
aa0c0674a8 |
Defer all VPP apps when there are missing teams (#42862)
Fixes #40785 ## Summary When a GitOps run includes a `volume_purchasing_program` config that references a team that doesn't exist yet, the code temporarily removes the entire VPP config from the global AppConfig, clearing ALL VPP token-to-team assignments on the server. However, the code only deferred `app_store_apps` for the missing teams, not for existing teams that also lost their VPP assignments. Those existing teams then failed with "No available VPP Token" when their `app_store_apps` were applied. The fix widens the deferral scope to match the clearing scope. When VPP assignments are temporarily cleared, `app_store_apps` are now deferred for all teams in the VPP config, not just the missing ones. |
||
|
Scott Gress
|
6598b608b7 |
Enforce GitOps exceptions (#42191)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #42180 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Enhanced GitOps exception handling for labels, secrets, and software with clearer enforcement and omission semantics. * Server-side prefetch of team software so omitted team software can preserve existing installers during validation. * Presence flags track whether top-level keys (labels, secrets, software) were provided versus omitted. * **Behavior Changes** * Omitted vs empty sections are now distinguished: omission can mean “no-op” or “delete-all” depending on exception settings. * GitOps YAML can define and manage labels directly; validations now reject YAML that includes keys marked as excepted. <!-- end of auto-generated comment: release notes by coderabbit.ai --> # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually * **Labels** - [ ] Validated that with label exceptions off, omitting `labels:` key from default.yml clears all global labels - [ ] Validated that with label exceptions off, omitting `labels:` key from a fleet .yml clears all labels for that fleet - [ ] Validated that with label exceptions off, setting empty `labels:` key from default.yml clears all global labels - [ ] Validated that with label exceptions off, setting empty `labels:` key from a fleet .yml clears all labels for that fleet - [ ] Validated that with label exceptions on, omitting `labels:` key from default .yml leaves existing global labels as-is - [ ] Validated that with label exceptions on, omitting `labels:` key from a fleet .yml leaves existing labels as-is - [ ] Validated that with label exceptions on, setting `labels:` key on default .yml generates an error - [ ] Validated that with label exceptions on, setting `labels:` key on a fleet .yml generates an error - [ ] Validated that with label exceptions on, a policy using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, a query using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, an MDM profile using `labels_include_any` referencing an existing label succeeds without `labels:` key - [ ] Validated that with label exceptions on, a software package using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) - [ ] Validated that with label exceptions on, an app store app using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) - [ ] Validated that with label exceptions on, a fleet maintained app using `labels_include_any` referencing an existing label succeeds without `labels:` key (requires software exceptions off) * **Secrets** - [ ] Validated that with secrets exceptions off, omitting `secrets:` key from default.yml clears all global secrets - [ ] Validated that with secrets exceptions off, omitting `secrets:` key from a fleet .yml clears all secrets for that fleet - [ ] Validated that with secrets exceptions on, omitting `secrets:` key from default .yml leaves existing global secrets as-is - [ ] Validated that with secrets exceptions on, omitting `secrets:` key from a fleet .yml leaves existing secrets as-is - [ ] Validated that with secrets exceptions on, setting `secrets:` key on default .yml generates an error - [ ] Validated that with secrets exceptions on, setting `secrets:` key on a fleet .yml generates an error * **Software** - [ ] Validated that with software exceptions off, omitting `software:` key from no-team.yml/unassigned.yml clears all software for "no team" - [ ] Validated that with software exceptions off, omitting `software:` key from a fleet .yml clears all software for that fleet - [ ] Validated that with software exceptions off, setting empty `software:` key on a fleet .yml clears all software for that fleet - [ ] Validated that with software exceptions off, setting empty `software:` key on no-team.yml/unassigned.yml clears all software for "no team - [ ] Validated that with software exceptions on, omitting `software:` key from a fleet .yml leaves existing software as-is - [ ] Validated that with software exceptions on, setting `software:` key on a fleet .yml generates an error - [ ] Validated that with software exceptions on, omitting `software:` key from no-team.yml/unassigned.yml leaves existing software as-is for "no team" - [ ] Validated that with software exceptions on, setting `software:` key on no-team.yml/unassigned.yml generates an error - [ ] Validated that with software exceptions on, a policy using `install_software.hash_sha256` referencing an existing package succeeds without `software:` key - [ ] Validated that with software exceptions on, a policy using `install_software.app_store_id` referencing an existing VPP app succeeds without `software:` key - [ ] Validated that with software exceptions on, a patch policy using `fleet_maintained_app_slug` referencing an existing FMA succeeds without `software:` key - [ ] Validated that with software exceptions on, `setup_experience.software` referencing existing software succeeds without `software:` key (server-side validation fallback) - [ ] Validated that with software exceptions on, omitting `software:` from no-team.yml/unassigned.yml preserves existing no-team software - [ ] Validated that with software exceptions on, a policy in no-team.yml/unassigned.yml using `install_software.hash_sha256` referencing existing no-team software succeeds without `software:` key For unreleased bug fixes in a release candidate, one of: - [X] Confirmed that the fix is not expected to adversely impact load test results I don't think so. There is a bit of overhead when this feature is used since we have to fetch software from the server, but it would be done in a specific test, so even if there is an impact it should affect existing load testing, only new, specific tests. |
||
|
Scott GressandCopilot Autofix powered by AI
|
40e91c0ece |
Allow hosts key to be empty for manual labels (#42022)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #41672 # Details Updates GitOps label functionality so that omitting the `hosts:` key under a manual label will _not_ clear hosts from that label, but will instead preserve the existing membership. This allows users to manage manual hosts with an external system (via the labels API), while still managing the labels themselves in GitOps. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually - [X] verified that you can still add a manual label with `hosts:` - [X] verified that leaving `hosts:` off a manual label doesn't change the host assignment - [X] verified that putting `hosts:` with no value on a manual label clears the hosts - [X] verified that you can still add a dynamic label - [X] verified that generate-gitops still exports manual hosts --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
George Karr
|
99f8639484 |
feat installers labels include all (#42061)
- **DB migration: Add `require_all` column to installers labels tables (#41279)** - **Backend: Support labels_include_all for installers/apps (#41324)** - **ui impl for labels include all (#41836)** <!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #39916 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled |
||
|
Magnus Jensen
|
a8c9e261d7 |
speed up macOS profile delivery for initial enrollments (#41960)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #34433 It speeds up the cron, meaning fleetd, bootstrap and now profiles should be sent within 10 seconds of being known to fleet, compared to the previous 1 minute. It's heavily based on my last PR, so the structure and changes are close to identical, with some small differences. **I did not do the redis key part in this PR, as I think that should come in it's own PR, to avoid overlooking logic bugs with that code, and since this one is already quite sized since we're moving core pieces of code around.** # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Faster macOS onboarding: device profiles are delivered and installed as part of DEP enrollment, shortening initial setup. * Improved profile handling: per-host profile preprocessing, secret detection, and clearer failure marking. * **Improvements** * Consolidated SCEP/NDES error messaging for clearer diagnostics. * Cron/work scheduling tuned to prioritize Apple MDM profile delivery. * **Tests** * Expanded MDM unit and integration tests, including DeclarativeManagement handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Jahziel Villasana-Espinoza
|
02a9eb8769 | merge main | ||
|
Martin AngersandJahziel Villasana-Espinoza
|
ba04887100 |
Backend: Support labels_include_all for installers/apps (#41324)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #40721 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually I (Martin) did test `labels_include_all` for FMA, custom installer, IPA and VPP apps, and it seemed to all work great for gitops apply and gitops generate, **except for VPP apps** which seem to have 2 important pre-existing bugs, see https://github.com/fleetdm/fleet/issues/40723#issuecomment-4041780707 ## New Fleet configuration settings - [ ] Verified that the setting is exported via `fleetctl generate-gitops` - [ ] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [ ] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [ ] Verified that any relevant UI is disabled when GitOps mode is enabled --------- Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com> |
||
|
Victor Lyuboslavsky
|
902b4af289 |
Fixed GitOps failing to delete a certificate authority (#41693)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #38036 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * GitOps now correctly orders operations so certificate authorities can be removed only after referencing certificate templates are handled, preventing failed deletions during config updates. * Improved user-facing error when a CA cannot be deleted because certificate templates still reference it, with guidance to remove templates first. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Scott Gress
|
01d13f5080 |
add keymap for new renames, and shallow duplication (#41682)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #41091 # Details This PR finishes the work of aliasing multi-platform keys by: * Added the renames to the list maintained by generate-gitops so that `fleetctl get` can use the new names * Updated the code that adds the new names to API and `fleetctl get` output to only add new nested keys under new parents, e.g. add `apple_settings.configuration_profiles`, but not `macos_settings.configuration_profiles`. The API key duplicator now runs through `RewriteDeprecatedKeys` which is a little heavier per-token, but for old keys we're doing less work so I think this ends up being slightly more performant than before, at least for large payloads. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. n/a, changelog for new keys added in previous PR ## Testing - [X] Added/updated automated tests updated tests for the duplicators - [X] QA'd all new/changed functionality manually - [X] `/config` and `/fleets` APIs now only return new keys under new parents - [X] `fleetctl get fleets` now returns new multiplatform keys |
||
|
|
2abacc577e |
Feat/31914 patch policy (#41518)
Implements patch policies #31914 - https://github.com/fleetdm/fleet/pull/40816 - https://github.com/fleetdm/fleet/pull/41248 - https://github.com/fleetdm/fleet/pull/41276 - https://github.com/fleetdm/fleet/pull/40948 - https://github.com/fleetdm/fleet/pull/40837 - https://github.com/fleetdm/fleet/pull/40956 - https://github.com/fleetdm/fleet/pull/41168 - https://github.com/fleetdm/fleet/pull/41171 - https://github.com/fleetdm/fleet/pull/40691 - https://github.com/fleetdm/fleet/pull/41524 - https://github.com/fleetdm/fleet/pull/41674 --------- Co-authored-by: Jonathan Katz <44128041+jkatz01@users.noreply.github.com> Co-authored-by: jkatz01 <yehonatankatz@gmail.com> Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com> Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com> |
