<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #40433
# Details
This PR updates `fleetctl gitops` and `fleetctl generate_gitops` to use
`unassigned.yml` in place of `no-team.yml`. The two files are utilized
identically, except that `unassigned.yml` expects the `name:` to be
`Unassigned` rather than `No team`.
Internally, we still map some things to the string "no team" before
sending to the back end so that we don't have to update back-end code
and make more spaghetti to clean up when we 🔪 No Team as a concept in
Fleet 5.
We do pass the filename into the main `DoGitOps` method, but both I and
Claude did our best to determine that it's not used in any way that
would break with this change.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
From test plan:
- [X] With a pre-existing GitOps folder w/ `no-team.yml`, run `fleetctl
gitops -f /path/to/no-team.yml --dry-run
--enable-log-topics=deprecated-field-names` and verify that everything
works as expected and you get the deprecation warning.
- [X] Do the above without `--dry-run` and verify via the UI and/or
`fleetctl generate-gitops` that the Fleet config is as expected.
- [X] Change `no-team.yml` to `unassigned.yml`, try a gitops run with
`fleetctl gitops -f /path/to/unassigned.yml --dry-run
--enable-log-topics=deprecated-field-names` and verify that you get an
error because the `name:` is still `No team`
- [X] Change the `name:` to `Unassigned`, repeat the run above and
verify that the output is the same as with `no-team.yml`, and that no
deprecation warning is listed.
- [X] Do the same as the above without `--dry-run` and verify that the
Fleet config is as expected.
- [X] Run `fleetctl generate-gitops` and verify that `unassigned.yml` is
output rather than `no-team.yml`, and any related files are under the
`lib/unassigned` folder rather than `lib/no-team`, and any paths inside
`unassigned.yml` (e.g. for scripts) are pointed at `lib/unassigned`.
## New Fleet configuration settings
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
@noahtalerman will work on this
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #39344
# Details
This PR builds on the previous PR
(https://github.com/fleetdm/fleet/pull/39847) which added `renameto`
tags to certain API parameters to mark them as deprecated. How this is
used:
### In requests
* When decoding requests, log a warning if a `json` or `query` param is
used that has a `renameto` tag, e.g. if a `team_id` param is sent but
the related struct has `renameto:"fleet_id"` in it.
* If the `renamedto` version (e.g. `fleet_id`) is sent in the request,
rewrite it to the deprecated name so that it can be unmarshalled into
the struct
* If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an
error and quit
* URLs with deprecated terms have new aliases using `WithAltPaths` --
warning on using old URLSs a TODO that will be handled in a subsequent
PR.
### In responses
* Output _both_ the deprecated and new names for fields that have
`renameto` tags, so that we don't break existing workflows expecting the
old keys. Uses a shared `DuplicateJSONKeys` to do the duplication.
* Most API responses are handled in `EncodeCommonResponse`. Exceptions
are activities, failing policy webhooks and the streaming "list hosts"
endpoints which call the function directly.
### In fleetctl
* Similar to requests, log warnings when deprecated keys are used and
rewrite the new keys internally so that they can be unmarshalled.
* For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the
new names
* The set of keys to replace is hardcoded in `fleetctl` rather than
being dynamically generated as it is for API endpoints. Given the
mixture of typed and untyped data and the level of nesting, dynamic map
generation was very fragile and error-prone.
### Performance considerations
* The biggest performance hit is the addition of the JSON key rewriter
to the request pipeline. The rewriter buffers the entire request into
memory before eventually passing it to the decoder than unmarshals the
data into structs. I tried implementing this as a true streaming
rewriter but encountered issues where the request would hang if the
downstream reader (the decoder) encountered any errors. It's possible we
could implement this in a streaming fashion if we replace our [current
request
decoder](https://github.com/fleetdm/fleet/blob/da43bf8371695382c4af0972d5da456c6b94bdaf/server/service/endpoint_utils.go#L108)
with the v2 version, which is a bigger change requiring more thoughtful
discussion in the engineering team. As it stands, memory usage for
requests with deprecated fields will double while the request is being
decoded.
* The "alias rules" used to determine the old and new key names are
cached per struct type and for most endpoints are generated on server
start, so no performance impact is expected.
* Some `fleetctl` commands may have an extra unmarshal/marshal step but
as these are user-initiated and not performed in tight loops, the impact
should be minimal.
### TODO
* Log deprecation warnings when old URLs like "/fleet/teams" are used
* Update API fields that the front-end uses to avoid deprecation
warnings
* Update `fleetctl apply` to accept/return `kind: fleet` rather than
`kind: team`
* Find/update any fleet server config vars with old language
* Update all error messages that use old language
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
* Clicking around the front-end, no broken pages due to request
ingestion errors or bad responses
* Looking in network tab to verify that responses have both the old and
new keys
* Running `fleetctl generate-gitops` and verifying that the output looks
correct and can be ingested by `fleetctl gitops`
* Running `fleetctl get` and `fleetctl apply`
---------
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39265
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35307
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
Fixes#34066. Included a changes file because this catches fields other
than software icon.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
for #32014
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps manual labels can now reference hosts by Fleet host ID in
addition to hostname, hardware serial, or UUID.
- GitOps YAML/JSON accepts integers for host IDs; numeric IDs are
handled seamlessly alongside strings.
- Validation
- Stronger input validation for label hosts: only strings or integers
are allowed.
- Clear error returned for invalid types (e.g., floats) in hosts lists.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #30849.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [n/a] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
For #30095.
#32482 is additional cleanup. Merging this to unblock orchestration
Linux setup experience work. Code has already been reviewed prior to
merging into the feature branch.
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
For #31055.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Also removed the automatic install flag on YAML FMAs as it's
undocumented/unspec'd
Fixes#25636.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#30435.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #28855
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
For #27476
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
# Details
This PR adds a new command `generate-gitops` to the `fleetctl` tool. The
purpose of this command is to output GitOps-ready files that can then be
used with `fleetctl-gitops`.
The general usage of the command is:
```
fleectl generate-gitops --dir /path/to/dir/to/add/files/to
```
By default, the outputted files will not contain sensitive data, but
will instead add comments where the data needs to be replaced by a user.
In cases where sensitive data is redacted, the tool outputs warnings to
the user indicating which keys need to be updated.
The tool uses existing APIs to gather data for use in generating
configuration files. In some cases new API client methods needed to be
added to support the tool:
* ListConfigurationProfiles
* GetProfileContents
* GetScriptContents
* GetSoftwareTitleByID
Additionally, the response for the /api/latest/fleet/software/batch
endpoint was updated slightly to return `HashSHA256` for the software
installers. This allows policies that automatically install software to
refer to that software by hash.
Other options that we may or may not choose to document at this time:
* `--insecure`: outputs sensitive data in plaintext instead of leaving
comments
* `--print`: prints the output to stdout instead of writing files
* `--key`: outputs the value at a keypath to stdout, e.g. `--key
agent_options.config`
* `--team`: only generates config for the specified team name
* `--force`: overwrites files in the given directory (defaults to false,
which errors if the dir is not empty)
# Technical notes
The command is implemented using a `GenerateGitopsCommand` type which
holds some state (like a list of software and scripts encountered) as
well as a Fleet client instance (which may be a mock instance for tests)
and the CLI context (containing things like flags and output writers).
The actual "action" of the CLI command calls the `Run()` method of the
`GenerateGitopsCommand` var, which delegates most of the work to other
methods like `generateOrgSettings()`, `generateControls()`, etc.
Wherever possible, the subroutines use reflection to translate Go struct
fields into JSON property names. This guarantees that the correct keys
are written to config files, and protects against the unlikely event of
keys changing.
When sensitive data is encountered, the subroutines call `AddComment()`
to get a new token to add to the config files. These tokens are replaced
with comments like `# TODO - Add your enrollment secrets here` in the
final output.
# Known issues / TODOs:
* The `macos_setup` configuration is not output by this tool yet. More
planning is required for this. In the meantime, if the tool detects that
`macos_setup` is configured on the server, it outputs a key with an
invalid value and prints a warning to the user that they'll need to
configure it themselves.
* `yara_rules` are not output yet. The tool adds a warning that if you
have Yara rules (which you can only upload via GitOps right now) that
you'll have to migrate them manually. Supporting this will require a new
API that we'll have to discuss the authz for, so punting on it for now.
* Fleet maintained apps are not supported by GitOps yet (coming in
https://github.com/fleetdm/fleet/issues/24469). In the meantime, this
tool will output a `fleet_maintained_apps` key and trigger a warning,
and GitOps will fail if that key is present.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
For #26692.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
Changes file included in FE PR.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
> For #23497
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27267.
Below is what's shown immediately after selecting an EXE:
<img width="1254" alt="image"
src="https://github.com/user-attachments/assets/a28d8565-de88-448a-bdbc-92aefc34ad55"
/>
TODO:
* Tests
* GitOps requirements changes
* Disabling add button/adding errors when required scripts aren't
specified
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: RachelElysia <rachel@fleetdm.com>
For #24473
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
## Details
This PR adds the ability to manage labels via GitOps. Usage is as
follows:
* If a top-level `labels:` key is provided in the global YAML file
provided to GitOps, then any labels in this list will be created (if
using a new name) or updated (if using an existing name).
* If no top-level `labels:` key is provided, no changes will be made to
labels. This allows backwards-compatibility; customers won't blow away
all of their labels if they don't immediately use `labels:` in their
YAML
Additionally, some new validation has been added so that label usage is
checked prior to application. This means that when the gitops command is
run, it will verify that any labels referenced elsewhere in the YAML
(e.g. by software installers or mdm profiles) exist, and will bail with
an error message if they don't.
## Testing
**Test label deletion**
1. Add some labels via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, and verify that it doesn't say it will update or delete
any labels
2. Run `fleetctl gitops` with a default.yml file _without_ `labels:` in
it, and verify that it doesn't modify or remove your labels
4. Run `fleetctl gitops --dry-run` with a default.yml file with
`labels:` in it and nothing underneath, and verify that it says that it
will delete your labels
4. Run `fleetctl gitops` with a default.yml file with `labels:` in it
and nothing underneath, and verify that it removes all your labels
**Test label create/update**
1. Add a label "foo" via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file with two
`labels:` in it, one named "foo" and one named "bar". Verify that the
output says that one label will be created and one will be updated.
2. Run `fleetctl gitops` with a default.yml file with two `labels:` in
it, one named "foo" and one named "bar". Verify that the two labels now
exist in the UI with the configuration you specified.
**Test label usage**
1. Add a label "foo" in the UI.
1. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, where a software installer or mdm profile uses the
"foo" label via `labels_include_any`. Verify that the output doesn't
complain about unknown labels.
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with nothing underneath, and a software installer or mdm
profile uses the "foo" label via `labels_include_any`. Verify that the
output complains about unknown label "foo"
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with a "foo" label defined underneath, and a software
installer or mdm profile uses the "foo" label via `labels_include_any`.
Verify that the output doesn't complain about unknown labels.
For #25648
Fixed issue where `fleetctl gitops` was NOT deleting macOS setup
experience bootstrap package and enrollment profile. GitOps should clear
all settings that are not explicitly set in YAML config files.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25770
We already unmarshal macOS/Windows settings (added by Martin), so we
replace the path with an absolute file path and keep them unmarshalled
so they don't have to be re-unmarshalled later. Note: the custom
UnmarshalJSON method on these structs checks for (and handles) legacy
format (before labels were added).
Also some refactorings:
- extracted `extractControlsForNoTeam`
- reorganized `TestGitOpsBasicGlobalAndNoTeam` with subtests -- I did
not actually change functionality of this test
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #23531.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #24917. Should be worth the extra byte per row for the varchar
field.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For #22244. Previously empty script entries would get parsed and then
cause a panic later in the process.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
For #22787
Also revises the spec check to explain that scripts have to be defined
"controls" when used in policies for the same team, with an explicit
call-out for no-team.yml since this fix doesn't support pulling scripts
from the global file. This is because parsing and script-matching
happens early enough that we can't throw an error in the part of the
code where we bail when controls are defined in both no-team and default
files.
To minimize diff size, we're both "passing-by-ref" and returning the
maps-by-team of scripts and installers, though the former would be
sufficient on its own.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- N/A Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests (sorta)
- [x] Manual QA for all new/changed functionality
#22555
This resolves issues where no-team script paths are resolved relative to
the base rather than the file they're in.
There was a similar issue for software packages, which I also fixed.
This also fixes script-poolicy associations when scripts and policy
files automating those scripts need different relative paths.
Test file moves ensure that these fixes are tested.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#22115, #22116
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
No changes file, as FE changes file covers the entire feature
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
`fleetctl gitops` subtask for #20320
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#21467
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
#20895
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
#20464
Adding gitops support for a top level `software` key to be used to
manage installable software into "no team".
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
---------
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
#19600
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
#17309
I added some missing env var replacement tests for policies, queries,
etc.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
#18467
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
#16989
API draft PR: #17809.
UI draft PR: #18243.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
