**Related issue:** Resolves#46824
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed an issue where volume purchasing program assignments failed when
"All fleets" was selected; validation and token assignment now treat the
"All fleets" label consistently, preventing errors during configuration
application.
* **Tests**
* Added an end-to-end GitOps test case verifying "All fleets" is
supported for volume purchasing program entries.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43729
# Details
Adds output to GitOps runs indicating which custom/FMA software packages
would be deleted. This involves adding a `deleted_packages` key to the
`/software/batch/:request_uuid` ("Get status of software batch-apply
request") API, which will be documented separately.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [X] verified that a GitOps dry run produces one "would've deleted"
line per custom package / fma that would be deleted
- [X] verified that a GitOps real run produces one "deleted" line per
custom package / fma that was deleted
- [X] verified that adding software is unaffected
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* GitOps batch software operations now report packages pending deletion:
dry-runs show "would've deleted" warnings and real runs show deletions;
apply flows surface per-package deletion messages.
* Empty payload dry-run now still reports pending deletions when
applicable.
* **Tests**
* Added integration and datastore tests validating deletion-warning
output, pending-deletion detection, and related result handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#46392
A few things in this PR:
- updated the conversion from old default category to the new ones with
the emoji included that was introduced in the feature branch. It takes
into account what exists in the database now so if an admin wants to add
for example "Productivity" without the emoji as a category it won't get
overwritten.
- updated a few places to ignore missing categories rather than error
(what we do for adding a single FMA currently)
- updated permissions for "gitops" users
- added everything needed for gitops, generate-gitops support using the
existing endpoints from the last PR.
Didn't add logs like "[+] applied X self service categories" since it
wasn't mentioned in the docs, but wouldn't be too hard to add.
# Checklist for submitter
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
- Currently missing, at least on this branch
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44444
# Details
This PR fixes the following edge case when running `fleetctl gitops`:
1. A new fleet is added
2. That fleet is declared as an ABM default fleet and/or a fleet in a
VPP token location
3. The _other_ fleets declared as ABM defaults or VPP fleets are _not_
all provided in the GitOps run
In that case, the GitOps run would fail with an error that one of the
previously-existing fleets could not be found. This PR fixes the bug by
making sure that GitOps looks at both the currently-persisted fleets
(via the API) and any fleets that are being created in the current run.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- Reproduced the issue on `main` by attempting to create a new fleet
_and_ add it both as an ABM default fleet and to the set of VPP token
users in a single run, and getting an error about one of the existing
fleets not being found
- Verified that I was able to complete a gitops run successfully on this
branch with a new fleet as a VPP token user and a default ABM fleet
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Bug Fixes**
* Improved validation for Apple Business Manager and Volume Purchasing
Program token team assignments with clearer error messages when
referenced teams aren't found in Fleet.
* Enhanced team name matching to properly handle Unicode characters,
ensuring consistent team identification across GitOps configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#46391
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
Resolves#45220 (one of several PRs, we are very close)
## Testing
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Improved test infrastructure for the CLI: consolidated and renamed
test helpers, added a dedicated in-process CLI test helper, and updated
many test cases to use the new helpers.
* Tightened several test assertions and standardized output/error
validation across unit and integration tests to improve reliability.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45585)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#45220 (one of several PRs to achieve removing "testing"
package as dependency in production binary)
## Testing
- [x] QA'd all new/changed functionality manually.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Switched many tests to use a dedicated MySQL test helper package and
consolidated test-only utilities for datastore setup, cleanup, ad‑hoc
SQL, certificate generation, and activity/aggregation helpers.
* Added expanded test utilities for replication, DB connections and test
data seeding to improve integration-test reliability.
* **Chores**
* No production behavior or user-facing APIs were changed.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45406)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves #
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Updated test fixtures for Apple VPP configuration in GitOps workflows
to better reflect real-world scenarios.
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45305)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#45220 (one of many small PRs)
## Testing
- [x] QA'd all new/changed functionality manually. (Ran basic smoke
testing.)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added a consolidated test utilities package to boot full test servers,
seed users, retrieve auth tokens, and provide common mocks for
integration/service tests.
* Updated integration and service tests to use the new helpers for more
consistent, reliable test setup and wiring.
* **Chores**
* Centralized and reorganized test configuration types and options, and
standardized test wiring for clearer, maintainable test setup.
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45221)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Part of #38790. Stacked on top of #44932.
Closes#43969.
`fleetctl gitops` reads `configuration.path` for iOS / iPadOS VPP apps
and in-house `.ipa` apps, validates the file via
`fleet.ValidateAppleAppConfiguration`, and stores the bytes through the
service layer.
`fleetctl generate-gitops` emits `configuration.path` for apps that have
a stored configuration, writing the XML to
`lib/<team>/software/<app-slug>-config.xml`. Apps without configuration
omit the block entirely.
Round-trip integration tests
(`cmd/fleetctl/integrationtest/gitops/software_test.go`) cover apply +
generate for both VPP and in-house, plus a negative case for invalid
configuration content.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added managed app configuration support for enterprise software
deployments through GitOps
* Configurations now support platform-specific formats: XML for
iOS/iPadOS, JSON for Android
* Configuration validation ensures compatibility and prevents invalid
variable references
* **Tests**
* Added integration tests for configuration handling in software
deployment workflows
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/44933)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: jkatz01 <yehonatankatz@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44286
Unset `patch_software_title_id` rather than deleting the policy in
`BatchSetSoftwareInstallers`, so the orphaned policy gets picked up by
the `policiesToDelete` loop in `server/service/client.go:3121`. As a
result, the `deleted_policy` activity is now created properly, and
gitops dry/real runs also report the deletion:
```
dry run:
[-] would've deleted policy macOS - 010 Editor up to date
[-] would've deleted 1 policy
real run:
[-] deleting policy macOS - 010 Editor up to date
[-] deleting 1 policy
[-] deleted 1 policy
```
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed missing deletion activity logs when patch policies are removed
via GitOps so policy deletion events are now recorded.
* **Behavior Changes**
* Batch-updating installers now retains obsolete patch policies but
clears their patch installer reference instead of deleting the policy.
* **Tests**
* Added integration coverage to verify deletion activities are emitted
and installer batch behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#34464
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
---
## What
GitOps `--dry-run` was succeeding when `macos_manual_agent_install` was
set to `true` and a `macos_script` was configured under
`setup_experience`, but the actual GitOps run would fail with:
```
Couldn't add setup experience script. To add script, first disable macos_manual_agent_install.
```
## Why
The `manual_agent_install` conflict validation only existed server-side
in `ee/server/service/setup_experience.go:SetSetupExperienceScript()`.
The script upload call (`uploadMacOSSetupScript()`) was gated by
`!opts.DryRun` in `server/service/client.go`, so during dry-run the
upload was skipped entirely and the validation never fired.
## Fix
Added client-side validation in `server/service/client.go` at the point
where the YAML-parsed `MacOSSetup` struct is processed — before the
script file is validated and loaded. This check runs for **both dry-run
and real runs**, catching the conflict early. Two code paths were fixed:
1. **Team path** (~line 803): Checks `setup.ManualAgentInstall.Value`
when `setup.Script.Value` is set
2. **No-team path** (~line 2603): Checks
`macOSSetup.ManualAgentInstall.Value` when `macOSSetup.Script.Value` is
set
## How I reproduced the issue locally
### Prerequisites
- MySQL and Redis running via Docker: `docker compose up -d mysql_test
redis`
### Steps
1. Wrote an integration test
(`TestDryRunMacOSSetupScriptWithManualAgentInstallConflict`) that:
- Creates a GitOps user and fleetctl config
- Creates a bootstrap package server serving `testdata/signed.pkg`
- Creates a `.sh` script file with `echo "setup script"`
- Creates a **global config** YAML (minimal server settings)
- Creates a **team config** YAML with `macos_manual_agent_install:
true`, `macos_script: <path>`, and `macos_bootstrap_package: <url>`
- Runs `fleetctl gitops --dry-run` and asserts it fails
- Runs `fleetctl gitops` (no dry-run) and asserts it fails
2. Ran the test **before the fix** — confirmed the bug:
```
Dry-run error: <nil> ← BUG: should have failed
Real run error: ...status 422...first disable macos_manual_agent_install
← correctly fails
```
3. Applied the fix and re-ran — **both dry-run and real run now fail**
with the `macos_manual_agent_install` conflict error.
### Test command
```bash
MYSQL_TEST=1 REDIS_TEST=1 go test -v \
-run TestIntegrationsEnterpriseGitops/TestDryRunMacOSSetupScriptWithManualAgentInstallConflict \
./cmd/fleetctl/integrationtest/gitops/... -count=1 -timeout 600s
```
Both sub-tests (team and no-team paths) pass. All related existing tests
continue to pass:
- `TestMacOSSetup`, `TestMacOSSetupScriptWithFleetSecret`,
`TestDeletingNoTeamYAML`, `TestDisallowSoftwareSetupExperience`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* GitOps dry-run now correctly fails when a macOS setup configuration
combines manual agent installation with a provided setup script,
preventing false-positive dry-run success.
* **Tests**
* Added unit and integration regression tests to verify dry-run and
real-run rejection of conflicting macOS setup configurations for both
team-scoped and unassigned host scopes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#36976
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Label operations (create, edit, delete) now generate activities shown
in the activity feed with label and optional fleet context.
* Host label add/remove operations emit corresponding label edited
activities; duplicate label names are deduplicated.
* Label activity types are selectable/filterable in the activity
dashboard.
* **Tests**
* Added unit, integration, and UI tests covering label activity
emission, rendering, filtering, and GitOps label lifecycle scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42948
- Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup)
SetDefaultsIfNeeded()` to account for new fields
- Updated default creation and editing for team edit/creation paths
- Updated `generate-gitops` warning message from `macos_setup` to
`setup_experience`
- Updated fields types to optjson and updated test files
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
- Team edit and team creation through GitOps, validated config with `
curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'`
- New error message says `setup_experience` instead of `macos_setup`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added macOS MDM settings to control local account behavior: enable
managed local accounts (default false) and specify end-user local
account type (default "admin") for fleet and team configs. GitOps output
now highlights unsupported setup-experience cases.
* **Tests**
* Updated fixtures and integration tests to assert and persist the new
macOS local-account settings across config, team, and GitOps scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38013
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
See
https://drive.google.com/file/d/1zeFNLuf_rT5FWzDiYyL2_hbIBW2neba-/view?usp=drive_link
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* GitOps variables in JSON configuration profiles (Apple DDM
declarations and Android profiles) are now automatically escaped for
JSON special characters, ensuring proper handling of sensitive values.
* **Tests**
* Added JSON configuration profile escaping validation to the enterprise
GitOps integration test suite.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43417
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
done in https://github.com/fleetdm/fleet/pull/42216
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- Using a local fileserver, added the same software to two fleets and
ran `fleetctl gitops`. Verified that the first fleet downloaded the
file, the second fleet used the cache, and both fleet showed the
software installer in the UI.
## Summary by CodeRabbit
* **Chores**
* Updated software installer lookup mechanism to support optional
team-scoped searches, enabling fallback to cross-team installer cache
when team-specific installers are unavailable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43047
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
See
https://github.com/fleetdm/fleet/issues/42960#issuecomment-4244206563
and subsequent comments.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Apple DDM declarations support a vetted subset of Fleet variables with
per-host substitution; premium license required. Declaration tokens and
resend behavior now reflect variable changes; unresolved host
substitutions mark that host’s declaration as failed.
* **Bug Fixes**
* Clearer errors for unsupported or license-restricted Fleet variables
and more consistent DDM resend/update semantics when variables change.
* **Tests**
* Added extensive unit and integration tests covering Fleet variable
validation, substitution, token changes, resends, and failure states.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:**
Ref #34797
Ref #42675
## Problem
When a software installer spec has no `hash_sha256`, Fleet re-downloads
the package, re-extracts metadata, and re-upserts the DB on every GitOps
run, even if the upstream file hasn't changed. For deployments with 50+
URL-only packages across multiple teams, this wastes bandwidth and
processing time on every run.
## Solution
By default, use etags to avoid unnecessary downloads:
1. First run: Fleet downloads the package normally and stores the
server's ETag header
2. Subsequent runs: Fleet sends a conditional GET with `If-None-Match`.
If the server returns 304 Not Modified, Fleet skips the download,
metadata extraction, S3 upload, and DB upsert entirely
Opt-out with `always_download:true`, meaning packages continue to be
downloaded and re-processed on every run, same as today. No UI changes
needed.
```yaml
url: https://nvidia.gpcloudservice.com/global-protect/getmsi.esp?version=64&platform=windows
always_download: true
install_script:
path: install.ps1
```
### Why conditional GET instead of HEAD
Fleet team [analysis of 276 maintained
apps](https://github.com/fleetdm/fleet/pull/42216#issuecomment-4105430061)
showed 7 apps where HEAD requests fail (405, 403, timeout) but GET works
for all. Conditional GET eliminates that failure class: if the server
doesn't support conditional requests, it returns 200 with the full body,
same as today.
### Why opt-in
5 of 276 apps (1.8%) have stale ETags (content changes but ETag stays
the same), caused by CDN caching artifacts (CloudFront, Cloudflare,
nginx inode-based ETags). The `cache` key lets users opt in per package
for URLs where they've verified ETag behavior is correct.
Validation rejects `always_download: true` when hash_sha256` is set
## Changes
- New YAML field: `cache` (bool, package-level)
- New migration: `http_etag` VARCHAR(512) column (explicit
`utf8mb4_unicode_ci` collation) + composite index `(global_or_team_id,
url(255))` on `software_installers`
- New datastore method: `GetInstallerByTeamAndURL`
- `downloadURLFn` accepts optional `If-None-Match` header, returns 304
as `(resp, nil, nil)` with `http.NoBody`
- ETag validated per RFC 7232 (ASCII printable only, no control chars,
max 512 bytes) at both write and read time
- Cache skipped for `.ipa` packages (multi-platform extraInstallers)
- TempFileReader and HTTP response leak prevention on download retry
- Docs updated in `yaml-files.md`
## What doesn't change
- Packages with `hash_sha256`: existing hash-based skip, untouched
- FMA packages: FMA version cache, untouched
- Packages with `always_download: true`: identical to current behavior
- Fleet UI: no changes
## Test plan
Automated testing:
- [x] 16 unit tests for `validETag`
- [x] 8 unit tests for conditional GET behavior (304, 200, 403, 500,
weak ETag, S3 multipart, no ETag)
- [x] MySQL integration test for `GetInstallerByTeamAndURL`
- [x] All 23 existing `TestSoftwareInstallers` datastore tests pass
- [x] All existing service tests pass
Manual testing:
- [x] E2E: 86 packages across 6 CDN patterns, second apply shows 51
conditional hits (304)
- [x] @sgress454 used a local fileserver tool to test w/ a new instance
and dummy packages
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* ETag-based conditional downloads to skip unchanged remote installer
files.
* New always_download flag to force full re-downloads.
* **Tests**
* Added integration and unit tests covering conditional GETs, ETag
validation, retries, edge cases, and payload behavior.
* **Chores**
* Persist HTTP ETag and related metadata; DB migration and index to
speed installer lookups.
* Added installer lookup by team+URL to support conditional download
flow.
* **Bug Fix**
* Rejects using always_download together with an explicit SHA256 in
uploads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Scott Gress <scott@fleetdm.com>
Co-authored-by: Scott Gress <scott@pigandcow.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #40015
* Moves repeated empty mocks into a new `setupEmptyGitOpsMocks` method
* Adds new "deprecation" tests:
* In TestGitOpsFullGlobal, TestGitOpsFullTeam and
TestGitOpsFullGlobalAndTeam tests "kitchen sink" with both new and
deprecated keys
* Added keys and checks to verify `setup_experience`,
`apple_business_manager` and `volume_purchasing_program` configs
* Consolidated map of deprecated -> new GitOps keys in one place
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40015
# Details
Activates deprecation warnings for old API params and CLI args, updates
tests that would generate warnings (except for tests explicitly designed
to generate warnings).
The expectation from here on is that Fleet UI usage should not generate
any deprecation warnings in the server logs, nor should the output from
`generate-gitops` generate any warnings when fed into `gitops`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
- [X] clicked around in an mdm-enabled instance, turned setup experience
features on and off, saw no server warnings
- [X] did `fleetctl generate-gitops` on mdm-enabled instance, saw no
server or cli warnings
- [X] did `fleetctl gitops` on mdm-enabled instance, saw no server or
cli warnings
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41409
# Details
This PR updates the `ApplyStarterLibrary` method and functionality to
rely on the same templates and mechanisms as `fleetctl new`. The end
result is that running `fleetctl new` and `fleetctl gitops` on a new
instance should be a no-op; no changes should be made. Similarly,
changing the templates in a Fleet release will automatically affect
`fleetctl new` and `ApplyStarterLibrary` in the same exact way for that
release.
> Note that this moves the template files out of `fleetctl` and into
their own shared package. This move comprises the majority of the file
changes in the PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
Note that
<img width="668" height="44" alt="image"
src="https://github.com/user-attachments/assets/066cd566-f91d-4661-84fc-2aabbfce2ef9"
/>
will fail until the 4.83 Fleet docker image is published, since it's
trying to push 4.83 config (including `exceptions`) to a 4.82 server.
- [X] QA'd all new/changed functionality manually
- [X] Created a new instance and validated that the fleets, policies and
labels created matched the ones created by `fleetctl new`
- [X] Ran `fleetctl new` and verified that it created the expected
folders and files
- [X] Ran `fleetctl gitops` with the files created by `fleetctl new` and
verified that the instance was unchanged.
- [X] Ran `fleetctl preview` successfully using a dev build of the Fleet
server image (since it won't work against the latest published build,
which doesn't support `exceptions`). Verified it shows the expected
teams, policies and labels
Fixes#40785
## Summary
When a GitOps run includes a `volume_purchasing_program` config that
references a team that doesn't exist yet, the code temporarily removes
the entire VPP config from the global AppConfig, clearing ALL VPP
token-to-team assignments on the server. However, the code only deferred
`app_store_apps` for the missing teams, not for existing teams that also
lost their VPP assignments. Those existing teams then failed with "No
available VPP Token" when their `app_store_apps` were applied.
The fix widens the deferral scope to match the clearing scope. When VPP
assignments are temporarily cleared, `app_store_apps` are now deferred
for all teams in the VPP config, not just the missing ones.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42180
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced GitOps exception handling for labels, secrets, and software
with clearer enforcement and omission semantics.
* Server-side prefetch of team software so omitted team software can
preserve existing installers during validation.
* Presence flags track whether top-level keys (labels, secrets,
software) were provided versus omitted.
* **Behavior Changes**
* Omitted vs empty sections are now distinguished: omission can mean
“no-op” or “delete-all” depending on exception settings.
* GitOps YAML can define and manage labels directly; validations now
reject YAML that includes keys marked as excepted.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
* **Labels**
- [ ] Validated that with label exceptions off, omitting `labels:` key
from default.yml clears all global labels
- [ ] Validated that with label exceptions off, omitting `labels:` key
from a fleet .yml clears all labels for that fleet
- [ ] Validated that with label exceptions off, setting empty `labels:`
key from default.yml clears all global labels
- [ ] Validated that with label exceptions off, setting empty `labels:`
key from a fleet .yml clears all labels for that fleet
- [ ] Validated that with label exceptions on, omitting `labels:` key
from default .yml leaves existing global labels as-is
- [ ] Validated that with label exceptions on, omitting `labels:` key
from a fleet .yml leaves existing labels as-is
- [ ] Validated that with label exceptions on, setting `labels:` key on
default .yml generates an error
- [ ] Validated that with label exceptions on, setting `labels:` key on
a fleet .yml generates an error
- [ ] Validated that with label exceptions on, a policy using
`labels_include_any` referencing an existing label succeeds without
`labels:` key
- [ ] Validated that with label exceptions on, a query using
`labels_include_any` referencing an existing label succeeds without
`labels:` key
- [ ] Validated that with label exceptions on, an MDM profile using
`labels_include_any` referencing an existing label succeeds without
`labels:` key
- [ ] Validated that with label exceptions on, a software package using
`labels_include_any` referencing an existing label succeeds without
`labels:` key (requires software exceptions off)
- [ ] Validated that with label exceptions on, an app store app using
`labels_include_any` referencing an existing label succeeds without
`labels:` key (requires software exceptions off)
- [ ] Validated that with label exceptions on, a fleet maintained app
using `labels_include_any` referencing an existing label succeeds
without `labels:` key (requires software exceptions off)
* **Secrets**
- [ ] Validated that with secrets exceptions off, omitting `secrets:`
key from default.yml clears all global secrets
- [ ] Validated that with secrets exceptions off, omitting `secrets:`
key from a fleet .yml clears all secrets for that fleet
- [ ] Validated that with secrets exceptions on, omitting `secrets:` key
from default .yml leaves existing global secrets as-is
- [ ] Validated that with secrets exceptions on, omitting `secrets:` key
from a fleet .yml leaves existing secrets as-is
- [ ] Validated that with secrets exceptions on, setting `secrets:` key
on default .yml generates an error
- [ ] Validated that with secrets exceptions on, setting `secrets:` key
on a fleet .yml generates an error
* **Software**
- [ ] Validated that with software exceptions off, omitting `software:`
key from no-team.yml/unassigned.yml clears all software for "no team"
- [ ] Validated that with software exceptions off, omitting `software:`
key from a fleet .yml clears all software for that fleet
- [ ] Validated that with software exceptions off, setting empty
`software:` key on a fleet .yml clears all software for that fleet
- [ ] Validated that with software exceptions off, setting empty
`software:` key on no-team.yml/unassigned.yml clears all software for
"no team
- [ ] Validated that with software exceptions on, omitting `software:`
key from a fleet .yml leaves existing software as-is
- [ ] Validated that with software exceptions on, setting `software:`
key on a fleet .yml generates an error
- [ ] Validated that with software exceptions on, omitting `software:`
key from no-team.yml/unassigned.yml leaves existing software as-is for
"no team"
- [ ] Validated that with software exceptions on, setting `software:`
key on no-team.yml/unassigned.yml generates an error
- [ ] Validated that with software exceptions on, a policy using
`install_software.hash_sha256` referencing an existing package succeeds
without `software:` key
- [ ] Validated that with software exceptions on, a policy using
`install_software.app_store_id` referencing an existing VPP app succeeds
without `software:` key
- [ ] Validated that with software exceptions on, a patch policy using
`fleet_maintained_app_slug` referencing an existing FMA succeeds without
`software:` key
- [ ] Validated that with software exceptions on,
`setup_experience.software` referencing existing software succeeds
without `software:` key (server-side validation fallback)
- [ ] Validated that with software exceptions on, omitting `software:`
from no-team.yml/unassigned.yml preserves existing no-team software
- [ ] Validated that with software exceptions on, a policy in
no-team.yml/unassigned.yml using `install_software.hash_sha256`
referencing existing no-team software succeeds without `software:` key
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
I don't think so. There is a bit of overhead when this feature is used
since we have to fetch software from the server, but it would be done in
a specific test, so even if there is an impact it should affect existing
load testing, only new, specific tests.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41672
# Details
Updates GitOps label functionality so that omitting the `hosts:` key
under a manual label will _not_ clear hosts from that label, but will
instead preserve the existing membership. This allows users to manage
manual hosts with an external system (via the labels API), while still
managing the labels themselves in GitOps.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [X] verified that you can still add a manual label with `hosts:`
- [X] verified that leaving `hosts:` off a manual label doesn't change
the host assignment
- [X] verified that putting `hosts:` with no value on a manual label
clears the hosts
- [X] verified that you can still add a dynamic label
- [X] verified that generate-gitops still exports manual hosts
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- **DB migration: Add `require_all` column to installers labels tables
(#41279)**
- **Backend: Support labels_include_all for installers/apps (#41324)**
- **ui impl for labels include all (#41836)**
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39916
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34433
It speeds up the cron, meaning fleetd, bootstrap and now profiles should
be sent within 10 seconds of being known to fleet, compared to the
previous 1 minute.
It's heavily based on my last PR, so the structure and changes are close
to identical, with some small differences.
**I did not do the redis key part in this PR, as I think that should
come in it's own PR, to avoid overlooking logic bugs with that code, and
since this one is already quite sized since we're moving core pieces of
code around.**
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Faster macOS onboarding: device profiles are delivered and installed
as part of DEP enrollment, shortening initial setup.
* Improved profile handling: per-host profile preprocessing, secret
detection, and clearer failure marking.
* **Improvements**
* Consolidated SCEP/NDES error messaging for clearer diagnostics.
* Cron/work scheduling tuned to prioritize Apple MDM profile delivery.
* **Tests**
* Expanded MDM unit and integration tests, including
DeclarativeManagement handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40721
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
I (Martin) did test `labels_include_all` for FMA, custom installer, IPA
and VPP apps, and it seemed to all work great for gitops apply and
gitops generate, **except for VPP apps** which seem to have 2 important
pre-existing bugs, see
https://github.com/fleetdm/fleet/issues/40723#issuecomment-4041780707
## New Fleet configuration settings
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
---------
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38036
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* GitOps now correctly orders operations so certificate authorities can
be removed only after referencing certificate templates are handled,
preventing failed deletions during config updates.
* Improved user-facing error when a CA cannot be deleted because
certificate templates still reference it, with guidance to remove
templates first.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #41091
# Details
This PR finishes the work of aliasing multi-platform keys by:
* Added the renames to the list maintained by generate-gitops so that
`fleetctl get` can use the new names
* Updated the code that adds the new names to API and `fleetctl get`
output to only add new nested keys under new parents, e.g. add
`apple_settings.configuration_profiles`, but not
`macos_settings.configuration_profiles`.
The API key duplicator now runs through `RewriteDeprecatedKeys` which is
a little heavier per-token, but for old keys we're doing less work so I
think this ends up being slightly more performant than before, at least
for large payloads.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, changelog for new keys added in previous PR
## Testing
- [X] Added/updated automated tests
updated tests for the duplicators
- [X] QA'd all new/changed functionality manually
- [X] `/config` and `/fleets` APIs now only return new keys under new
parents
- [X] `fleetctl get fleets` now returns new multiplatform keys
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40627
# Details
This PR updates the way we enable failed policy reporting (via webhook
or ticket integration) for individual policies in GitOps. The existing
method is to declare a `policy_ids` key underneath
`failing_policies_webhook:` in either the global or a fleet .yml file,
and specify a list of policy IDs to enable the automation for. This PR
maintains this feature for backwards compatibility, and adds a new
feature where you can set `webhook_and_tickets_enabled: true` key in the
policy declaration itself. If _both_ these methods are used, the GitOps
run will fail.
**Implementation note:**
Because we're keeping the old way of doing this until Fleet 5, I took
the easy route and just translated the new way into the old way; that
is, we gather up the list of policies with `webhook_and_tickets_enabled:
true`, get their IDs and send that list to the server under the same
config we did previously. This works fine and there's nothing _wrong_
with it but ideally this flag would work the same as other per-policy
flags like `calendar_events_enabled` that are stored on the policy
record. That requires a migration and more new code that we'd have to
maintain alongside the existing code (or translate the old strategy to
the new one). I'm taking the lower-touch path here.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [x] Verified that `generate-gitops` outputs the new
`webhooks_and_tickets_enabled` flag instead of outputting `policy_ids`
under `failing_policies_webhook`
- [X] Verified that using the new flag in a fleet .yml file results in
the specified policies being enabled in the "other" automations for
policies (whether the webhook automation is enabled or not)
- [X] Verified the same for a global .default.yml file
- [X] Verified that using the old `failing_policies_webhook.policy_ids`
a fleet .yml file results in the specified policies being enabled in the
"other" automations for policies (whether the webhook automation is
enabled or not)
- [X] Verified the same for a global .default.yml file
- [X] Verified that trying to use both `webhooks_and_tickets_enabled`
and `failing_policies_webhook.policy_ids` at the same time results in an
error.
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
see
https://github.com/fleetdm/fleet/issues/40627#issuecomment-4024988552
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added configuration flag to enable webhooks and tickets for policies
in GitOps settings.
* System automatically resolves and assigns policy IDs when using the
new flag.
* **Tests**
* Added comprehensive test coverage for webhook and ticket enablement in
GitOps workflows, including conflict detection and policy ID assignment
validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41307
# Details
* Fixes a potential issue where running `fleetctl gitops` with only the
global file, with no controls provided, could wipe out global controls
that are provided in the "no team" file.
* Fixes error message when controls are missing.
# Checklist for submitter
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [x] `fleetctl gitops -f /path/to/default.yml` without controls, gives
`error: 'controls' must be set on global config, no-team.yml or
unassigned.yml`
- [x] `fleetctl gitops -f /path/to/default.yml` with empty controls
works
- [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/no-team.yml`
without controls, gives `error: 'controls' must be set on global config
or no-team.yml`
- [x] `fleetctl gitops -f /path/to/default.yml -f
/path/to/unassigned.yml` without controls, gives `error: 'controls' must
be set on global config or unassigned.yml`
- [x] `fleetctl gitops -f /path/to/default.yml -f /path/to/no-team.yml`
with empty controls in no-team.yml works
- [x] `fleetctl gitops -f /path/to/default.yml -f
/path/to/unassigned.yml` with empty controls in unassigned.yml works
- [x] `fleetctl gitops -f /path/to/no-team.yml` gives error `global
config must be provided alongside no-team.yml`
- [x] `fleetctl gitops -f /path/to/no-team.yml` gives error `global
config must be provided alongside unassigned.yml`
- [x] `fleetctl gitops -f /path/to/some-real-team.yml` with no controls
works
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41012
# Details
This PR makes it allowable to leave out almost all top-level keys from
GitOps files. The only required keys are _either_ `name:` (for a fleet
settings file) or `org_settings:` (for a global settings file). Omitting
a key is identical to supplying it with no value.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
Updated the "missing all global keys test", and added some new tests to
verify that omitting the key was the same as supplying it with an empty
value.
- [X] QA'd all new/changed functionality manually
1. Ran `fleetctl generate-gitops` to get a clean set of GitOps yml files
2. Removed all removable keys from default.yml and ran `fleetctl gitops`
3. Ran `fleetctl generate-gitops` again into a different dir
4. Ran `fleetctl gitops` with the original files to get back to original
state
5. Cleared out all now-removable keys and replaced them with empty value
(e.g. `reports:` with nothing under it)
6. Ran `fleetctl generate-gitops` again into a third dir
7. Compared the files from the second and third generate-gitops runs to
verify that omitting the key had the same result as supplying it with an
empty value
8. Did the above steps with a fleet (i.e. non-global) .yml file.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* GitOps files now support omitting top-level configuration keys instead
of requiring them to be explicitly set to empty values.
* org_settings is now required when team name is not specified.
* **Tests**
* Added integration tests validating behavior when omitting top-level
keys in global and team-level GitOps configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40108
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed double encoding of secret environment variables when configured
through GitOps, ensuring secrets are stored with proper escaping.
* **Tests**
* Added test coverage for configuration profile escaping to verify
proper handling of secret variables and API keys during GitOps
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38432
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40540
go-kit/log is no longer a direct dependency; moved kitlog adapter
required for some 3rd party libraries into its own package
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Present in previous PR
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Modernized logging across the codebase: switched from legacy logging
wrappers to Go's standard slog, updated adapters, tests, tools, and
server components.
* Threaded the new slog logger through test utilities and tooling;
adjusted a small number of logging-related function/constructor
signatures to accept the new logger type (minor compatibility updates).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40540
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Changes present in previous PR
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated internal logging infrastructure to improve consistency and
maintainability across the application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40621
# Details
This PR updates the output from `fleetctl gitops` and `fleetctl apply`
to use the correct terminology:
* "fleet" instead of "team"
* "report" instead of "query" (where appropriate)
* "for unassigned hosts" in place of "No Team" where possible, and "for
fleet Unassigned" otherwise.
All changes other than tests are in `client.go` and are text-only; no
functional code is changed (and no code relies on the strings besides
tests).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
I did a `fleetctl generate-gitops` and a `fleetctl gitops` run and saw
the updated logs:
```
[+] would've applied EULA
[+] would've applied certificate authorities
[+] would've applied fleet config
[+] would've applied MDM profiles
[+] would've applied enroll secrets
[+] would've applied 1 report
[+] would've applied 1 software package for fleet 'Unassigned'
[+] would've applied 0 app store apps for fleet 'Unassigned'
[+] would've applied webhook settings for unassigned hosts
[+] would've applied 1 policy
[!] gitops dry run succeeded
```
but I did not go through and try and replicate every log or error
message. I think the best we can do on this one is eyeball the code
changes for mistakes.
**Related issue:** Resolves#40412
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
- Individually or together setting vpp, software installers, or fleet
maintained apps will send the correct error now when applied to no team
with manual_agent_install enabled.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40054
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Changes included in previous PR
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Consolidated and standardized internal logging infrastructure across
the application by adopting a unified logging package throughout the
codebase, replacing previous external logging dependencies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Steven Palmesano
