**Related issue:** Resolves#39184
show apns expiration banner for the free tier in the UI. Before it was
limited to show only for premium tier.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Nico <32375741+nulmete@users.noreply.github.com>
**Related issue:** Resolves#40855
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40900
# Details
This PR makes the `secrets:` key under the top-level `org_settings` (for
default.yml) or `settings:` (for fleet .yml files) optional. Omitting
the key causes any enroll secrets present on the server to be retained.
There is more to the parent story that will require more design, but I
am getting this one out early because:
1. Our updated it-and-security files will not have `secrets:` and
2. This is not a breaking change, since currently omitting this key
results in a fatal error, _not_ the removal of all secrets (that
requires specifying an empty `secrets:` key)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [x] Using `secrets:` with correct syntax in `defaults.yml` updated
global secrets
- [x] Using `secrets:` with no value in `defaults.yml` removed all
global secrets
- [x] Omitting `secrets:` in `defaults.yml` retained all global secrets
- [x] Using `secrets:` with correct syntax in a fleet .yml file updated
that fleet's secrets
- [x] Using `secrets:` with no value in in a fleet .yml file removed
that fleet's secrets
- [x] Omitting `secrets:` in in a fleet .yml file retained that fleet's
secrets
FYI @allenhouchins: We will want to remove any of the changes made to
the ee folder in this PR before merging since these files were added for
testing
---
**Related issue:** Resolves
https://github.com/fleetdm/fleet/issues/38254
This pull request updates how Fleet-maintained apps handle default
categories, improving compatibility with older Fleet builds that may not
recognize newer categories. It introduces a more flexible approach for
category assignment, ensuring apps can still be added even if some
categories do not exist in the current database.
**Category compatibility improvements:**
* The service now maps only existing categories to IDs when adding
Fleet-maintained apps, allowing apps to be added even if some default
categories (like "Security" or "Utilities") are not present in older
Fleet builds. This prevents errors and improves backward compatibility.
[[1]](diffhunk://#diff-9e807526199f81f987717f1f3c0ec60260510dc79c12cbeb6b20190dcf39caa7L167-R186)
[[2]](diffhunk://#diff-8384a65651f44fd1a1b78da9bad5794155a76b3849d1b846f8cc5e06018c1365R1)
* Introduced a new `GetSoftwareCategoryNameToIDMap` method in the
datastore to return a mapping of known category names to their IDs, only
including those that exist. This is used in both production and test
code.
[[1]](diffhunk://#diff-4fe70646343c5f7a1b61dd6906889113f0e0ae63f684ab74a617acecb39fe6d3R6170-R6197)
[[2]](diffhunk://#diff-2fa819f19cb7709a65ded13aa51ac75cacbeb5764c88e4a64a02f3ab5b8933b2R700-R702)
[[3]](diffhunk://#diff-f4666ccc907f67a670871fd5600a726858ae613f67aeca8f2537b8315f9b1bf0R544-R545)
[[4]](diffhunk://#diff-f4666ccc907f67a670871fd5600a726858ae613f67aeca8f2537b8315f9b1bf0R2526-R2528)
[[5]](diffhunk://#diff-f4666ccc907f67a670871fd5600a726858ae613f67aeca8f2537b8315f9b1bf0R6148-R6154)
**App category updates:**
* Changed the default category for `appcleaner` from "Productivity" to
"Utilities" in both input and output JSON files.
[[1]](diffhunk://#diff-a6ff089976e7e51f336ab7fc67670375498323de9ec3b87d9aa6c6308ab8fb30L7-R7)
[[2]](diffhunk://#diff-a90f40ad8bd457b4977a1d53984999f89e014fa7442851defc559db252bcdf24L13-R13)
* Changed the default category for `nordvpn` from "Productivity" to
"Security" in both input and output JSON files.
[[1]](diffhunk://#diff-f3ea48ab7da0ffc182e6ec342ee426c0280aa105c79282a7284d0fa35e9a1cf5L7-R7)
[[2]](diffhunk://#diff-83f09709778de1c5a20cbfe88125d58c85563bd1afbc76dcf9f4e8731d6884b3L13-R13)
**Test and codebase adjustments:**
* Updated mocks and tests to use the new
`GetSoftwareCategoryNameToIDMap` method instead of the previous category
ID list approach.
[[1]](diffhunk://#diff-d29fdd0261044f92f761a0707ff8417c9478054ca7d9e4ab5b9d04ed95e15573L285-R286)
[[2]](diffhunk://#diff-d29fdd0261044f92f761a0707ff8417c9478054ca7d9e4ab5b9d04ed95e15573L371-R372)
* Removed an unused import (`fmt`) from `maintained_apps.go` for
cleanup.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
## Database migrations
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
## fleetd/orbit/Fleet Desktop
- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
---------
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40108
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed double encoding of secret environment variables when configured
through GitOps, ensuring secrets are stored with proper escaping.
* **Tests**
* Added test coverage for configuration profile escaping to verify
proper handling of secret variables and API keys during GitOps
operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39150
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40302
# Details
This PR adds support for a `paths:` key for scripts declared under
`controls:` in a GitOps fleet file. If supplied, `paths:` must contain a
"glob" expression (as [supported by the doublestar
package](https://github.com/bmatcuk/doublestar?tab=readme-ov-file#patterns)).
The existing `path:` key still works but may not contain glob
expressions. When a `paths:` key is encountered, we expand it and add
all matching valid (as in, `.sh` or `.ps1`) files to the set of script
files to process.
Subsequent PRs will add this functionality to other entities that use
`path:` (such as reports and policies).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
Tried with various combinations of `*` and `**` in gitops runs, and
mixing of `path:` and `paths:`
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40284
Following existing codebase pattern of passing the NOW time to the
method.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
- Via manipulating DB.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced Android certificate renewal system with improved
time-handling capabilities and better testability infrastructure for
certificate template operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38432
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40183 and #40182
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves #
This PR adds the ability to skip any Fleet logic that validates on
whether or not an MDM Declaration is usable with Fleet by setting a
configuration of `FLEET_MDM_SKIP_DECLARATION_VALIDATION` to `True`. We
would like to leverage declarations such as [config
files](https://developer.apple.com/documentation/devicemanagement/servicesconfigurationfiles)
as well as other none configuration declarations that are
[restricted](https://github.com/fleetdm/fleet/blob/main/server/fleet/apple_mdm.go#L736-L738)
like
[Activations](https://developer.apple.com/documentation/devicemanagement/activationsimple),
and any declaration that requires
[AssetData](https://developer.apple.com/documentation/devicemanagement/assetdata).
We understand that these are not usable by most Fleet customers,
especially those that use Fleets cloud hosting option.
Since we run Fleet on our own infrastructure we are able to leverage our
proxy to support all additional data components needed for declarations
above.
## Testing
I've built and run Fleet locally and validated that with the
`FLEET_MDM_SKIP_DECLARATION_VALIDATION` I am able to add any
declarations I please. Without the config I am unable to add the
requested declaration types above.
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added experimental `mdm.allow_all_declarations` configuration option
to permit all Apple MDM declaration types when enabled, overriding
default validation checks.
* **Documentation**
* Updated configuration documentation with details for the new
experimental MDM option, including defaults, environment variables, YAML
format, and safety considerations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35799
Loadtest results for 100K hosts and 300K software titles.
```
=== Performance Test Results: No team_id (all teams) ===
Description Average Worst Results
----------- ------- ----- -------
Page 0, hosts_count DESC 229ms 241ms 20 items
Page 0, hosts_count ASC 203ms 211ms 20 items
Page 1, hosts_count DESC 339ms 423ms 20 items
Page 1000, hosts_count DESC 202ms 219ms 20 items
100 per_page, hosts_count DESC 620ms 708ms 100 items
Default sort (no order params) 229ms 245ms 20 items
Order by name ASC, page 0 4.642s 4.785s 20 items
Order by name ASC, page 1000 6.418s 6.771s 20 items
Vulnerable only 3.431s 3.496s 20 items
Search 'chrome' 9.6s 10.111s 20 items
Known exploit filter 9.792s 10.102s 20 items
Min CVSS score 7.0 12.368s 12.665s 20 items
CVSS range 7.0-9.0 12.221s 12.523s 20 items
Available for install 87ms 93ms NO RESULTS
Self-service only 4.46s 4.757s 20 items
=== Performance Test Results: team_id=0 (no team / unassigned) ===
Description Average Worst Results
----------- ------- ----- -------
Page 0, hosts_count DESC 378ms 404ms 20 items
Page 0, hosts_count ASC 339ms 345ms 20 items
Page 1, hosts_count DESC 478ms 513ms 20 items
Page 1000, hosts_count DESC 398ms 417ms 20 items
100 per_page, hosts_count DESC 864ms 1.025s 100 items
Default sort (no order params) 399ms 411ms 20 items
Order by name ASC, page 0 5.346s 5.41s 20 items
Order by name ASC, page 1000 7.444s 7.615s 20 items
Search 'chrome' 9.051s 9.245s 20 items
Known exploit filter 10.511s 10.884s 20 items
Min CVSS score 7.0 16.589s 16.701s 20 items
CVSS range 7.0-9.0 15.878s 15.999s 20 items
Available for install 1.394s 1.429s 1 items
Self-service only 1.4s 1.456s 1 items
```
Documented in the issue:
The fix includes a small behavior change. The default primary sort of
/software/titles remains host_counts, but the secondary sort is now
software_title_id and not name. This was necessary to optimize the
endpoint. This means that if you have 1 host in your fleet, the software
page will not show the software titles ordered by name anymore. For
large fleets, this does not matter since all titles generally have
different host counts.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Optimized the software titles endpoint for improved query performance
and faster data retrieval.
* Enhanced database efficiency when retrieving software information,
resulting in better overall system responsiveness and reduced query
times.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39247
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
This pull request updates the logic for selecting which user receives
maintenance window calendar events on hosts with multiple users. The
changes clarify and enforce a priority system for choosing the recipient
email, ensuring that IdP-sourced emails are preferred, followed by
Google Chrome profile emails. This affects both user-facing
documentation and backend implementation.
**User-facing behavior and documentation:**
* The end-user documentation now explicitly describes the email
selection priority for calendar event recipients: IdP Username email is
chosen first, then Google Chrome profile email, and if multiple Chrome
emails exist, the first alphabetically is selected.
**Backend logic and data selection:**
* The comment in `calendar_cron.go` is updated to match the new email
selection logic, explaining the prioritization of email sources for
host-user assignment.
* The SQL query in `policies.go` is refactored to implement the new
priority system for selecting user emails per host:
- IdP sources (`mdm_idp_accounts`, `idp`) are considered first,
- then Google Chrome profiles,
- then other sources.
- If multiple emails exist at the same priority, the first
alphabetically is chosen.
---------
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Juan Fernandez <juan-fdz-hawa@users.noreply.github.com>
Co-authored-by: Juan Fernandez <juan@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39673
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
**Related issue:** Resolves#39361
This fixes an issue of the overflow of the resend button off the edge of
the os settting modal table.
We've changed the syling to grow and shrink the error text and column
dynamically so that the table will always be pushed up against the right
edge and the text will grow and shrink as needed so that it wont push
the button any further right
<img width="838" height="436" alt="image"
src="https://github.com/user-attachments/assets/a5acfd44-0d77-4062-92e4-909077827fee"
/>
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
**Related issue:** Resolves#40066
This allows ipados and ios devices to resend their profiles on the host
details and my device pages
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
**Related issue:** Resolves#39324
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
---------
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40685
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40564
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added OpenTelemetry instrumentation to the internal HTTP client to
provide enhanced observability through distributed tracing and metrics
collection for HTTP operations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#37802
Attempt to fix Windows FMAs not showing up as available when the
software titles they match to have upgrade codes. Since we don't surface
upgrade codes in the `fleet_maintained_apps` table and matching exactly
by name could miss some cases, this fix uses `team_titles.name LIKE
CONCAT(LEAST(fma.name, fma.unique_identifier), '%')`. Note the LEAST
there is only for the "Box Drive" app which has a longer name than
unique_identifier, and just compares the strings and not their length.
This isn't optimal for performance or correctness, but it only checks
with titles already available to the team as installers so it shouldn't
be terrible. Until upgrade_code is surfaced in the
`fleet_maintained_apps` table this should be sufficient.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
**Related issue:** Resolves#14401
# Checklist for submitter
this updates the mechanism of storing the auth token for a user that is
used for making requests and validating a user session. We change the
storage from local storage to a cookie. This allow a bit more security
and prepares for a future change where we will allow the browser to
handle setting and passing the auth token in the request.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #40433
# Details
This PR updates `fleetctl gitops` and `fleetctl generate_gitops` to use
`unassigned.yml` in place of `no-team.yml`. The two files are utilized
identically, except that `unassigned.yml` expects the `name:` to be
`Unassigned` rather than `No team`.
Internally, we still map some things to the string "no team" before
sending to the back end so that we don't have to update back-end code
and make more spaghetti to clean up when we 🔪 No Team as a concept in
Fleet 5.
We do pass the filename into the main `DoGitOps` method, but both I and
Claude did our best to determine that it's not used in any way that
would break with this change.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
From test plan:
- [X] With a pre-existing GitOps folder w/ `no-team.yml`, run `fleetctl
gitops -f /path/to/no-team.yml --dry-run
--enable-log-topics=deprecated-field-names` and verify that everything
works as expected and you get the deprecation warning.
- [X] Do the above without `--dry-run` and verify via the UI and/or
`fleetctl generate-gitops` that the Fleet config is as expected.
- [X] Change `no-team.yml` to `unassigned.yml`, try a gitops run with
`fleetctl gitops -f /path/to/unassigned.yml --dry-run
--enable-log-topics=deprecated-field-names` and verify that you get an
error because the `name:` is still `No team`
- [X] Change the `name:` to `Unassigned`, repeat the run above and
verify that the output is the same as with `no-team.yml`, and that no
deprecation warning is listed.
- [X] Do the same as the above without `--dry-run` and verify that the
Fleet config is as expected.
- [X] Run `fleetctl generate-gitops` and verify that `unassigned.yml` is
output rather than `no-team.yml`, and any related files are under the
`lib/unassigned` folder rather than `lib/no-team`, and any paths inside
`unassigned.yml` (e.g. for scripts) are pointed at `lib/unassigned`.
## New Fleet configuration settings
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
@noahtalerman will work on this
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40021
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed false positive detections for Python packages on Ubuntu systems,
addressing misclassification scenarios that were affecting package
detection accuracy on Ubuntu 20.04 and other versions.
* Enhanced handling of duplicate Python package entries to correctly
identify and consolidate multiple occurrences, preventing incorrect
over-reporting of installed packages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [ ] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38536
This PR moves all logic to create new activities to activity bounded
context.
The old service and ActivityModule methods are not facades that route to
the new activity bounded context. The facades will be removed in a
subsequent PR.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added webhook support for activity events with configurable endpoint
and enable/disable settings.
* Enhanced automation-initiated activity creation without requiring a
user context.
* Improved activity service architecture with centralized creation and
management.
* **Improvements**
* Refactored activity creation to use a dedicated service layer for
better separation of concerns.
* Added support for host-specific and automation-originated activities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Extended the statistics payload to include arrays of Fleet-maintained
app slugs in use on macOS and Windows. Updated the datastore to query
and populate these fields, and documented the new fields in the usage
statistics article.
---------
Co-authored-by: Juan Fernandez <juan@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35805
Fixed to make sure software host counts tables never have host counts of
0.
Planning to loadtest this fix along with the follow up fix for
https://github.com/fleetdm/fleet/issues/35799
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed accumulation of dead rows in software host count tracking,
improving data accuracy and system performance.
* Enhanced validation to ensure consistent and reliable software
availability records.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#15744
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
Manually inserted 10k hosts and random emails associated to them.
Response times were always below 200ms locally.
We already have covering indexes so we don't need a migration.
https://github.com/user-attachments/assets/721db4f1-f3c9-4ede-ba62-499ac30c4a02
SQL used:
```sql
-- Seed 10k hosts for performance testing email search.
-- Requires: SET cte_max_recursion_depth = 10000;
--
-- Distribution (by osquery_host_id suffix):
-- 1-2500: Chrome profile email only
-- 2501-5000: IdP email only
-- 5001-7500: Both Chrome profile and IdP emails
-- 7501-10000: No email association
-- Generate 10k hosts using a recursive CTE
INSERT INTO hosts (osquery_host_id, node_key, hostname, uuid, platform, detail_updated_at, label_updated_at, policy_updated_at)
WITH RECURSIVE seq AS (
SELECT 1 AS n
UNION ALL
SELECT n + 1 FROM seq WHERE n < 10000
)
SELECT
CONCAT('perf-osq-', n),
CONCAT('perf-nk-', n),
CONCAT('perf-host-', n),
CONCAT('perf-uuid-', n),
'darwin',
NOW(),
'2000-01-01 00:00:00',
'2000-01-01 00:00:00'
FROM seq;
-- Populate display names so that hosts are visible in the UI.
INSERT INTO host_display_names (host_id, display_name)
SELECT id, hostname
FROM hosts
WHERE osquery_host_id LIKE 'perf-osq-%';
-- 1-2500: Chrome profile email only
INSERT INTO host_emails (host_id, email, source)
SELECT id, CONCAT('chrome-', id, '@example.com'), 'google_chrome_profiles'
FROM hosts
WHERE osquery_host_id LIKE 'perf-osq-%'
AND CAST(SUBSTRING(osquery_host_id, 10) AS UNSIGNED) BETWEEN 1 AND 2500;
-- 2501-5000: IdP email only
INSERT INTO host_emails (host_id, email, source)
SELECT id, CONCAT('idp-', id, '@example.com'), 'mdm_idp_accounts'
FROM hosts
WHERE osquery_host_id LIKE 'perf-osq-%'
AND CAST(SUBSTRING(osquery_host_id, 10) AS UNSIGNED) BETWEEN 2501 AND 5000;
-- 5001-7500: Both Chrome profile and IdP emails
INSERT INTO host_emails (host_id, email, source)
SELECT id, CONCAT('chrome-', id, '@example.com'), 'google_chrome_profiles'
FROM hosts
WHERE osquery_host_id LIKE 'perf-osq-%'
AND CAST(SUBSTRING(osquery_host_id, 10) AS UNSIGNED) BETWEEN 5001 AND 7500;
INSERT INTO host_emails (host_id, email, source)
SELECT id, CONCAT('idp-', id, '@example.com'), 'mdm_idp_accounts'
FROM hosts
WHERE osquery_host_id LIKE 'perf-osq-%'
AND CAST(SUBSTRING(osquery_host_id, 10) AS UNSIGNED) BETWEEN 5001 AND 7500;
-- 7501-10000: no emails (nothing to insert)
-- Cleanup:
-- DELETE FROM host_emails WHERE host_id IN (SELECT id FROM hosts WHERE osquery_host_id LIKE 'perf-osq-%');
-- DELETE FROM host_display_names WHERE host_id IN (SELECT id FROM hosts WHERE osquery_host_id LIKE 'perf-osq-%');
-- DELETE FROM hosts WHERE osquery_host_id LIKE 'perf-osq-%';
```
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40054
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated internal logging infrastructure across Mobile Device
Management (MDM) service modules to improve code consistency and
maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#34068 Adds automatic retries (up to 3 attempts) for failed software installs from host details, self-service, and setup experience across all installer types.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40124
# Details
Implements the proposal in
https://docs.google.com/document/d/16qe6oVLKK25nA9GEIPR9Gw_IJ342_wlJRdnWEMmWdas/edit?tab=t.0#heading=h.nlw4agv1xs3g
Allows doing e.g.
```go
logger.WarnContext(logCtx, "The `team_id` param is deprecated, use `fleet_id` instead", "log_topic", "deprecated-field-names")
```
or
```go
if logging.TopicEnabled("deprecated-api-params") {
logging.WithLevel(ctx, slog.LevelWarn)
logging.WithExtras(
ctx,
"deprecated_param",
queryTagValue,
"deprecation_warning",
fmt.Sprintf("'%s' is deprecated, use '%s'", queryTagValue, renameTo),
)
}
```
Topics can be disabled at the app level, and enabled/disabled at the
command-line level.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
No logs have this in prod yet, but I added some manually in a branch and
verified that I could enable/disable them via CLI options and env vars,
including enabling topics that were disabled on the server. Tested for
both server and `fleetctl gitops`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added per-topic logging control to enable or disable logging for
specific topics via configuration and CLI flags.
* Added context-aware logging methods (ErrorContext, WarnContext,
InfoContext, DebugContext) to support contextual logging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
