Commit Graph
3712 Commits
Author SHA1 Message Date
Victor Lyuboslavsky 949a1eeabb Add sso_server_url configuration for dual URL SSO setups (#31497)
This change allows configuring a separate URL for SSO callbacks, which
is useful when organizations have different URLs for admin access vs
agent/API access.

Fixes #31480 the SSO issue where organizations with dual URL setups were
getting 'Destination does not match requested URL' errors after
upgrading to v4.71.0 with the new SAML library.

Video demo: https://www.youtube.com/watch?v=dFzNpUY3XKI

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually

## New Fleet configuration settings

- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
  - Same PR since this is going to be a 4.71.1 patch
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

* **New Features**
* Added support for configuring a dedicated SSO URL, allowing
organizations to restrict SSO authentication to a specific URL.
* The new SSO URL option is available in both the UI and API
configuration settings.

* **Documentation**
* Updated configuration and API documentation to include the new SSO URL
option with usage examples.

* **Bug Fixes**
* Resolved authentication issues for organizations using separate URLs
for admin and agent/API access.

* **Tests**
* Added new unit and integration tests to verify SSO behavior with and
without the dedicated SSO URL.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-01 20:32:15 +02:00
Juan Fernandez 6d45bc8c4f Ability to set TPM PIN protector policy on host. (#31484)
For #31193.

Added a new detail query used for determining whether the user is able to set up a TPM PIN protector, if not able, an MDM command is queued up to apply the proper policy on the host.
2025-08-01 13:32:19 -04:00
Konstantin Sykulev 828504d038 Automatic install policies in ListHostSoftware (#31469)
Fixes #30197
# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-08-01 10:22:14 -05:00
Jordan Montgomery 5abf101d9a #31450 Fix API errors on Wipe/Lock/Turn off MDM on Personal MDM devices (#31483)
Fixes #31450 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-08-01 10:03:37 -04:00
Magnus Jensen c7c87fce4f Enforce FileVault at login when manually enrolled (#31170)
Done by not allowing any deferrals as before one deferral was allowed

fixes: #29250 

_There is no doc change as we nowhere state that we allow one deferral,
let me know if we want to write somewhere that this is now the standard
behaviour._

_I also investigated trying to force it directly when the profile
arrived but without any luck, so still need a logout/login to get
filevault enabled, but it's no longer possible to cancel/defer it._

**Do verify when testing in automatic enrollment that FileVault is still
enforced as I can't test Automatic enrollment yet.**

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] If database migrations are included, checked table schema to
confirm autoupdate **not relevant as it does not update any schema just
modifies existing entries.**
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-08-01 15:15:11 +02:00
Victor Lyuboslavsky 9ef1772771 Fixed issue ingesting certs with long country codes. (#31443)
Fixes #30390 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked table schema to confirm autoupdate

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Expanded support for certificate country codes up to 32 characters,
allowing non-standard country code values.
* **Bug Fixes**
* Improved certificate ingestion to handle and log fields that exceed
maximum allowed lengths instead of causing failures.
* **Tests**
* Added tests to verify handling of long country codes and truncation of
overly long certificate fields.
* **Chores**
* Updated database schema to increase the allowed length for country
code fields in certificates.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-31 23:06:36 +02:00
Magnus Jensen 57566301e1 Wait for expected profiles to be sent before releasing device (#31381)
This PR addresses the concern of potentially being able to release a
device before any profile is sent, and the check thinking there is no
pending. It addresses both the release worker, but also the orbit setup
experience endpoint, even though that is less likely.

_Checked the query against my host on dogfood where it took 0.1 seconds,
with the single host._

fixes: #31143 

_I also ended up putting my main test in a new file
`integration_mdm_release_worker_test.go` and decided not to do fancy
setup, as there is only one test so no recurring things, and based on
our retro talk also moved the setup experience related tests inside of
`integration_mdm_dep_test.go` into their separate file
`integration_mdm_setup_experience_test.go`_

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [ ] QA'd all new/changed functionality manually (No, since this one is
hard to reproduce, but instead wrote an integration test before doing
the change to verify the behaviour.)
2025-07-31 17:50:57 +02:00
Jordan Montgomery 0b7167b37c Recreate nano_view_queue with explicit collation and SQL SECURITY INVOKER (#31388)
Fixes #28573 
Fixes #31141 

I don't know of a good way to add automated tests for this unfortunately
- there is no easy way that I can see in our tests to really reproduce
either of these issues

Manual testing can be done for 31141 by creating a different user,
recreating the nano_view_queue as that user using the old query(i.e.
without SQL SECURITY INVOKER), and then deleting the user, seeing that
the nano_view_queue gets busted(i.e. no queries can be run against it),
then doing this migration or even just running the new query and
verifying that the nano_view_queue can be queried again.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.


## Testing
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
2025-07-30 15:26:19 -04:00
Konstantin Sykulev 68c3ade0b2 Removed fleet secret validation during gitops dry runs (#31402)
Fixes #30853

Install and uninstall scripts that contain fleet secrets do not need to
be validated in the `batchSetSoftwareInstallersEndpoint` during gitops
dry runs. These secrets are already validated on the gitops side.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-07-30 13:12:39 -05:00
Jordan Montgomery f048df9d60 Add MS-MDE2 Request/Enrollment version 7.0 to allowlist to fix Windows enrollments (#31412)
Fixes #31232

The Microsoft documentation does not directly address
RequestVersion/EnrollmentVersion 7.0 in the allowed versions lists but
grepping the MS-MDE2 protocol docs for EnrollmentVersion finds a few
references related to Federated Authentication and this specifically
seems to be a changeset introduced for AzureAD/Intune related features
and does not impact how we do enrollments, as best I can tell.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually
2025-07-30 11:53:27 -04:00
Gabriel Hernandez 774dfb8b80 add service discovery endpoint test (#31373)
added a quick test for the service discovery endpoint
2025-07-30 11:43:03 +01:00
Juan Fernandez eac86a1224 Added new orbit config flag. (#31332)
For #31065 

Added new orbit config flag 'EnableBitLockerPINProtectorConfig' set iff Disk encryption is enforced and the RequireBitLockerPIN server config flag is set.
2025-07-29 19:22:36 -04:00
5b68247c14 Activity feed: Surface team_id and team_name in policy and query activities (#28394)
fix #27689
# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (added in the
comment)
- [x] Manual QA for all new/changed functionality

output ss
![Screenshot 2025-04-20
100928](https://github.com/user-attachments/assets/b62c1370-18e6-4d0f-a8ab-3a3d2d2084c7)
![Screenshot 2025-04-20
101101](https://github.com/user-attachments/assets/4f052242-d4c0-4027-afef-37f8d29991fc)

Noticed that editing a global policy doesn't generate an edited_policy
activity log in my local environment, while it does in others.
Investigated the gap and shared findings in the issue thread for further
clarification(in https://github.com/fleetdm/fleet/issues/27689)

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: Scott Gress <scott@fleetdm.com>
2025-07-29 17:57:21 -05:00
Ian LittmanandJahziel Villasana-Espinoza 89ca35c66b Switch vulns cron false positive clear to clear vulns based on when the vulns run started, rather than based on periodicity (#31364)
Fixes #26404.

This means that for long vulns runs vulns will stick around longer, so
we don't wind up nuking vulns that were added earlier in the run, and in
cases where the vulns run takes less than 2h we'll see vulns clear
cleanly more quickly.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [ ] QA'd all new/changed functionality manually

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2025-07-29 10:14:14 -05:00
Dan Fuhry 60b3b514c2 [fleetctl] api command: support request body, including file uploads (#30806)
Add the capability to build a request body with `fleetctl api`,
including uploading files.

Example command to upload a software package:

```sh
fleetctl api --debug -X POST -F team_id=0 -F 'software=@./server/service/testdata/software-installers/ruby.deb' software/package
```

Unit tests are included for both simple POST requests and file uploads.

Closes #21754.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-29 08:15:23 -05:00
Ian Littman a24500c937 Skip software installers for which we can't, or don't need to, parse package IDs/create uninstall scripts (#31347)
Fixes #30565. Applies to FMA-only extensions (DMG, ZIP), EXEs, and
tarballs. This means that MSI/PKG FMAs will still have package IDs
populated a day after server start if they aren't filled in, on the off
chance that admins use $PACKAGE_ID on uninstall scripts on either of
those, replicating existing behavior.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-07-28 13:58:19 -05:00
Jahziel Villasana-Espinoza f43d0fcea3 comment out test case (#31304)
# Checklist for submitter

Commenting out test so that CI runs don't fail. This will be fixed in
https://github.com/fleetdm/fleet/issues/31303.

If some of the following don't apply, delete the relevant line.

## Testing

- [x] Added/updated automated tests
2025-07-25 19:54:29 -04:00
Tim Lee f4814f6143 Fix NVD feed generation (#31258) 2025-07-25 08:52:00 -06:00
Ian Littman bed1c6a318 Add software sanitation on ingest back, use it to fix DCV Viewer versions (#31251)
We'll want to pull this into a feed so fixes don't take a Fleet release
to propagate, and some fixes currently in the vulns mutations list
should probably move over here (as they're also dealing with non-semver
versions), but that's out of scope for this particular fix.

Fixes #31123.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-25 08:45:39 -05:00
Ian LittmanandKonstantin Sykulev 71d54e1847 Populate version for macOS Chrome FMA on import, use Chrome Enterprise PKG instead of DMG, add tooltip on "latest" version when adding FMA (#30926)
Fixes #27919.

Here's how the `latest` version shows up in the UI:

<img width="513" height="288" alt="image"
src="https://github.com/user-attachments/assets/76842d1c-36f6-400c-8621-8d067ee410c6"
/>

<img width="785" height="318" alt="image"
src="https://github.com/user-attachments/assets/7077644e-7a0e-4fa4-87ce-56f54db41eb2"
/>

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
2025-07-24 16:14:01 -05:00
Sarah Gillespie 0779ee61db Enable BYOD service discovery cron (#31228) 2025-07-24 13:28:36 -05:00
Sarah Gillespie b0f47725fd Add activity details for BYOD MDM enrollments (#31191) 2025-07-24 10:28:50 -05:00
Scott Gress 4fc07c7738 Add config for requiring BitLocker PIN (#31109)
For #31063 

# Details

This PR adds the `RequireBitLockerPIN` config to app-wide and team
configs. This maps to a new `windows_require_bitlocker_pin` JSON field
for gitops and `fleetctl apply`.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
  * Will add changelog when feature is complete
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
      * Will add to docs when feature is complete
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
       * No UI yet
- [X] Manual QA for all new/changed functionality
   * Tested No Team and team config via Postman API calls
   * Tested Gitops for no-team and team YML files using `fleetctl`
   * Tested `fleetctl generate-gitops`
2025-07-23 14:38:49 -05:00
Konstantin SykulevandIan Littman 7a1e469ac0 Microsoft office FMA version from release notes (#30686)
Fixes #30082

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Microsoft Office applications for Mac now display a simplified short
version identifier, improving consistency with inventory systems.
* Added support for version transformation during ingestion of Homebrew
apps using external reference functions.

* **Bug Fixes**
* Enhanced uninstall process for Microsoft Word on Mac to remove a
broader set of user data and configuration files.

* **Tests**
* Added tests to ensure correct extraction of build numbers and short
version formats from Microsoft Office release notes.

* **Documentation**
* Updated changelog to reflect the addition of short version identifiers
for Microsoft Office Fleet maintained apps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-07-23 12:07:18 -05:00
Gabriel Hernandez 4d0518137e Add service discovery API endpoint (#31089)
relates to #31057 

adds an endpoint to expose the fleet handled service discovery endpoint.

> NOTE: test will be done in a follow up PR

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
2025-07-23 12:11:32 +01:00
Jordan Montgomery fea2b40cd8 BMAA: Add personal enrollment type including aggregate counts (#31091)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [ ] Manual QA for all new/changed functionality
2025-07-22 17:24:19 -04:00
Magnus Jensen a06603b816 Fix flaky test async last seen by using channel sync instead of time sleep (#31128)
Attempt at fix: #24652

I used randokiller with up to 100 and 150 retries, where sometimes it
failed after 89 attempts, to verify the flakiness.

I tried a couple of approaches, with one being just upping the
time.sleep to 200ms still without luck.
I then opted for a _sync_ channel that blocks and waits for the timed
flush to execute which allows it to go further and with a timeout that
fails the test if a deadlock is caused.

I looked into [synctest](https://pkg.go.dev/testing/synctest), but that
is still experimental and expected to become non-experimental in 1.26,
so not really usable for now, but sounds like it will help in time/timer
based tests that are a bit flaky.

Successful randokiller run:
https://github.com/fleetdm/fleet/actions/runs/16446175657/job/46478695973
where retries is set to 100 using the channel approach here.

_I do feel it's a bit weird that the last test `cap and timed flush`
does not have the same timer issues, might be due to the setup of only
having one item before each sleep, where if it had multiple it might be
suspect to the same time issues seen in this test `timed flush`_
2025-07-22 18:19:44 +02:00
Magnus Jensen 6147a12ece Fix stale pending remove apple declarations if host was offline for add and remove declaration (#30981)
Fixes: #29824 

This PR fixes a situtation where Apple Declarations could be lingering
around for hosts, if they were offline when the decl. was added and
removed, and no further declaration config is pushed to force a status
update.

It tackles it by deleting the pending and failed installs from the
table, before setting the remaining (verified and verifying) to be
remove operation, as those have hit the host.

I couldn't come up with a way that would auto-fix the hosts we see in
dogfood, as those have the same declaration identifier and token for
both an install row and pending remove row. Those needs to manually be
adjusted and then it should be good.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-22 11:22:04 +02:00
Konstantin Sykulev 8234e46e7b Exclude pending software installs from host inventory request (#30855)
Fixes https://github.com/fleetdm/fleet/issues/30747

- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-07-21 16:25:00 -05:00
Lucas Manuel Rodriguez d256bfdc71 Add arm64 support for fleetd extensions and fixes on test scripts (#31084)
This was required to test https://github.com/fleetdm/fleet/pull/30864 on
Apple Silicon.

I've created https://github.com/fleetdm/fleet/issues/31092 for tracking
purposes.

Fixes:
- Build univeral binary extension on macOS to test on VMs without
Rosetta.
- Add support for linux and Windows arm64. Which is also needed to test
Linux and Windows on UTM on Apple Silicon.
- Add Linux arm64 & Windows arm64 to the test scripts.

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-07-21 15:47:59 -03:00
Jahziel Villasana-Espinoza 20e0790930 change previously installed apps to managed (#31061)
> Closes #31060

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-07-18 16:49:12 -04:00
Sarah Gillespie ee68607401 Implement service discovery for Apple MDM account-driven enrollment (#31027) 2025-07-18 15:45:05 -05:00
Jordan Montgomery 1b83b1d2bc BMAA: SCEP Renewal (#31006)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.


- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
2025-07-18 16:45:00 -04:00
Lucas Manuel Rodriguez 4948325892 fleetd generate TPM key and issue SCEP certificate (#30932)
#30461

This PR contains the changes for the happy path.
On a separate PR we will be adding tests and further fixes for edge
cases.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for using a TPM-backed key and SCEP-issued certificate
to sign HTTP requests, enhancing security through hardware-based key
management.
* Introduced new CLI and environment flags to enable TPM-backed client
certificates for Linux packages and Orbit.
* Added a local HTTPS proxy that automatically signs requests using the
TPM-backed key.

* **Bug Fixes**
* Improved cleanup and restart behavior when authentication fails with a
host identity certificate.

* **Tests**
* Added comprehensive tests for SCEP client functionality and TPM
integration.

* **Chores**
* Updated scripts and documentation to support TPM-backed client
certificate packaging and configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-18 11:31:52 -03:00
Ian Littman b979330680 Turn Acrobat Reader CPE test back on (#31049)
#29571 resolved itself apparently.

- [x] Added/updated automated tests
2025-07-18 09:10:50 -05:00
Victor Lyuboslavsky 1d1c174d8c TPM osquery-perf support (#30994)
Fixes #30475

# Checklist for submitter

- [x] Manual QA for all new/changed functionality



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Introduced support for HTTP message signing in agent-server
communications, enhancing request authentication.
* Added a configurable option to control the probability of agents using
HTTP message signatures via a new command-line flag.
* **Bug Fixes**
* Improved error logging for issues encountered during HTTP signature
key retrieval, providing better visibility into failures.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-18 10:19:05 -03:00
Jahziel Villasana-Espinoza 5d92e2a5d8 add back test (#31032)
> Closes #28490

This has been fixed on the NVD side. No changes needed other than
re-enabling the test.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-18 09:14:22 -04:00
Victor Lyuboslavsky 909a48b0ea Host identity SCEP rate limit. (#31038)
Fixes #30989 

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added rate limiting to SCEP certificate enrollment, returning a "Too
Many Requests" (HTTP 429) response if hosts request certificates too
frequently.

* **Bug Fixes**
* Improved error handling for rate-limited SCEP requests, providing
clear feedback when rate limits are exceeded.

* **Tests**
  * Introduced integration tests to verify SCEP rate limiting behavior.

* **Chores**
* Enhanced internal configuration handling for SCEP certificate
management.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-18 10:04:14 -03:00
Jahziel Villasana-Espinoza dea236bb62 remove 'lock' on verification commands when MDM is turned off (#31023)
> Closes #31020

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-07-17 21:50:18 -04:00
jacobshandlingandJacob Shandling 7ff7d70d09 Allow users of Fleet in Primo mode to access Software automations and Failing policy ticket & webhook automations (#30865)
## For #30749, #31013

This PR implements changes to the UI and back end to accommodate
Software automations and Failing policy Ticket and Webhook automations
when Fleet is in Primo mode. Follow-up to
https://github.com/fleetdm/fleet/pull/30291

### Software automations
- When on the `/software` page and in Primo mode, the UI is, under the
hood, on "No team," though any reference to "team"s is hidden as much as
possible. In "normal" Fleet, Software automations can only be accessed
when on "All teams." This PR implements a special case in Primo mode:
when on No team and Primo mode is enabled, the user can now access the
"Software automations" modal to configure automation settings, which are
global.
- Simplified some conditions
- Moved logic living in the parent Software page that was specific to
the `SoftwareAutomations` modal into the modal for better encapsulation.

### Policy automations
The calendar, software, and scripts failing policy automations are
currently only configurable on a team (including No team) and not for
All teams. Ticket and webhook automations, accessible via the "Other
workflows" modal, by contrast, are only configurable for All teams and
teams other than No team, but not for No team. This PR updates the
Policies page, when in Primo mode (and therefore forced to be on "No
team") to:
- Continue providing "No team" data to the first 3 mentioned policy
automations modals.
- Include an enabled Other workflows option in the automations dropdown
- Update the submission handler of the Other workflows modal to update
the relevant _global_ config values
- The backend is updated to recognize this case (Failing policy webhook
/ ticket destination, policy on No team, in Primo mode) and handle it
using the global config, making the above logic sound

_Product should consider if any of these changes should be implemented
for "normal" Fleet_

### Listing and deleting policies

- Primo mode presents a pseudo-team-less UX. However, it is still
possible for earlier clients to have policies on "All teams." This
implements the ability to both see and delete "teamless" (No team under
the hood) policies and any such inherited global policies

### Other UI considerations
- Remove teams-related functionality in a couple more places - see
#31013

### Demos

- [Deleting policies, including any potentially inherited from All teams
(possible from before Primo
mode)](https://drive.google.com/file/d/1ZI4MNM3bkiOtD5MInAU32htQw8kDEupK/view?usp=drive_link)

- [x] Changes file added for user-visible changes in `changes/
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-07-17 15:53:31 -07:00
Ian Littman ab958704f7 Fix insufficient deduplication on vulnerabilities count query (#31021)
Fixes #27580.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-17 17:40:21 -05:00
Sarah Gillespie ce02856f85 Potential datastore optimizations for concurrent use of list mdm command API to poll results by host identifier (#30804) 2025-07-17 15:25:31 -05:00
Dante Catalfamo 484e4896de Make sure SAML requests always ask for email format NameID (#31008)
#30978
2025-07-17 14:25:33 -04:00
Jahziel Villasana-EspinozaandIan Littman 3324157511 Use upgrade code if available to improve accuracy of auto-install policy (#30977)
> Closes #27447

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-07-17 12:18:06 -04:00
Ian Littman c6ab9939b5 Extract UpgradeCode from MSI custom packages, use for better uninstall script generation (#30969)
Fixes #27758.

<img width="807" height="303" alt="image"
src="https://github.com/user-attachments/assets/58e5b9bc-42d6-4195-868e-bf6206ec9cd5"
/>

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-07-17 10:33:23 -05:00
Juan Fernandez fdfef5adf1 30311: Fix race condition in test (#30903)
For #30311 

Refactored `AddHostsToTeam` so that batch size can be specified via a
parameter and not a global variable.
2025-07-17 10:20:49 -04:00
Lucas Manuel Rodriguez 8f2f805741 Use UUID if osquery identifier not set (#30990)
Change related to https://github.com/fleetdm/fleet/issues/28818.
2025-07-17 16:03:01 +02:00
Victor Lyuboslavsky d70f5fd501 Deleting host deletes its host identity cert. (#30984)
Fixes #30474

# Checklist for submitter
- [x] Added/updated automated tests


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
* Added new test cases to verify that re-enrollment with a deleted
host's certificate is correctly rejected.
* **Bug Fixes**
* Improved host deletion to ensure related certificate records are
properly cleared.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-17 15:40:05 +02:00
Victor Lyuboslavsky f79fed9712 Added auth.require_http_message_signature server option. (#30983)
For #30947 

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for requiring HTTP message signatures for fleetd
requests, configurable via a new setting.
* Enhanced middleware to enforce HTTP message signature requirements
when enabled.

* **Tests**
* Introduced integration tests to verify host identity endpoints enforce
HTTP message signature requirements.
* Updated test utilities and suite setup to support configurable
signature enforcement.

* **Chores**
* Refactored configuration and test server options to support the new
signature enforcement feature.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-17 13:23:19 +02:00
Victor Lyuboslavsky 836cc044d2 Fleet server verifies HTTP signature (#30825)
Fixes #30473 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for TPM-backed host identity certificates enabling
hardware-backed HTTP signature authentication for hosts.
* Introduced HTTP signature verification middleware for API requests,
applied conditionally for premium licenses.
* Hosts presenting identity certificates must authenticate with matching
HTTP message signatures during enrollment and authentication.
* Added SCEP-based certificate issuance for secure host identity
management.
* Updated enrollment endpoints to use standardized request/response
contract types.

* **Bug Fixes**
* Enhanced authentication logic to verify consistency between host
identity certificates and host records, preventing duplicate or
mismatched identities.

* **Chores**
* Updated dependencies and test infrastructure to support HTTP signature
verification and host identity certificate workflows.
* Added comprehensive integration and datastore tests for host identity
certificate issuance, storage, and authentication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-16 20:08:27 +02:00