Commit Graph
577 Commits
Author SHA1 Message Date
Konstantin Sykulev a3ae1a6f91 Support for fleet maintained apps in gitops (#28751)
https://github.com/fleetdm/fleet/issues/24469

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-07 18:16:08 -05:00
Martin Angers 48de857dca BRP: add batch-resend profile to hosts endpoint based on status (#28871) 2025-05-07 16:48:18 -04:00
Martin Angers 672ea70876 IdP vars: add integration test, address last PR comments (#28794) 2025-05-05 11:46:10 -04:00
Jahziel Villasana-EspinozaandIan Littman db5444d6cd software categories: backend (#28479)
> For #28138 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-05-02 11:41:26 -04:00
Martin Angers 95b80482ba IdP vars: store profile-variable associations when new profile is created/updated (#28645) 2025-04-30 16:03:23 -04:00
Jordan Montgomery e514fc4881 Custom SCEP renewal (#28616)
For https://github.com/fleetdm/fleet/issues/27984

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-30 15:31:45 -04:00
Dante Catalfamo 1ab7bdc923 Bulk script execution backend (#28299)
#28158
2025-04-30 12:54:46 -04:00
Ian Littman 49c49c7433 Implement self-service install status endpoints (#28424)
For #28411.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-24 12:20:21 -05:00
Jordan Montgomery 862739292e Renewal of DigiCert certificates on macOS (#28449)
Adds renewal of Digicert certificates:
https://github.com/fleetdm/fleet/issues/26553 . Does not attempt to
renew custom SCEP or NDES. Also we aren't actually calling the DigiCert
renewal endpoint at this time because we don't believe we need to and we
can't necessarily do that as we weren't previously storing the serial
number however this change adds storage of the serial number.


# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
2025-04-24 08:35:15 -04:00
Jahziel Villasana-Espinoza abd01f2428 enable specifying installers by SHA256 in gitops (#28349)
> For #23497

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-18 16:41:41 -04:00
Victor Lyuboslavsky df71307eeb Added scim/details endpoint (#28007)
For #27281

This PR adds `/api/{version}/fleet/scim/details` endpoint, along with
some frontend fixes.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-10 14:08:45 -05:00
Martin Angers 00149cb5a9 Cancel upcoming activities: create past canceled activities (#27956) 2025-04-09 16:08:51 -04:00
Martin Angers 23186fc7a6 Cancel upcoming activities: ignore cancelled activities in queries, remove cancellable (#27810) 2025-04-08 11:23:28 -04:00
Victor Lyuboslavsky 3d0025c570 SCIM + host integration (#27880)
For #27284

This PR:
- Adds SCIM as a fallback for username during macOS end user
authentication during setup experience
- Adds SCIM/endUsers details to host details

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-08 09:35:06 -05:00
Victor Lyuboslavsky 8658608c37 Add SCIM Groups (#27702)
For #27287

This PR adds SCIM Groups to Fleet's SCIM endpoint as a follow on to SCIM
Users. The logic has been manually tested with Okta, and integration
tests will be in the next PR.

# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-04-02 17:10:40 -05:00
Martin Angers 69fcda9686 Cancel upcoming activities: DB schema and backend (#27710) 2025-04-01 14:08:56 -04:00
Victor Lyuboslavsky 2198fd8d65 Add SCIM Users (#27551)
For #27287

Video explaining the PR: https://www.youtube.com/watch?v=ZHgFUAvrPEI

This PR adds SCIM Users support for Okta. The goal is to first add
Users/Groups support so that the remaining backend SCIM work can be done
in parallel.

This PR does not include the following, which will be added in later PRs
- Changes file
- Groups support for Okta
- Full support for Entra ID
- Integration tests

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-01 11:02:24 -05:00
Jahziel Villasana-Espinoza 2802cf57d5 Update FMA refreshing logic to remove apps that were removed upstream (#27594)
> No issue, we noticed this while testing FMA for Windows

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-03-28 14:04:08 -04:00
Martin Angers 78c0d67e78 Bugfix: re-create deleted iOS/iPadOS host entries in Fleet if it checks in again via MDM (#27231) 2025-03-26 09:33:38 -04:00
Scott Gress 7eebc22693 Add author ID to labels (#27055)
For #27035 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

## Details

This PR adds an `author_id` column to the `labels` table, and adds the
associated properties to the `Label` and `LabelSpec` types. When a new
label is created via the UI or API, an author ID is set on the label if
one can be inferred from the context. Otherwise, the author ID is set to
`null`.

## Authz and Automated testing

Additional backend authorization logic is introduced in a follow-on PR,
https://github.com/fleetdm/fleet/pull/27089, because rconciling all of
the test updates between this PR and
https://github.com/fleetdm/fleet/pull/27038 was getting complicated.

## Manual Testing

* Tested in the UI by creating a new label on the Hosts page
* Tested via Gitops by merging this branch with
https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops`
with a global config with `labels:` in it.
2025-03-20 16:05:16 -05:00
Victor Lyuboslavsky 93bdb437ac Resend Windows profiles on change (#27308)
For #25030 

This PR includes the bug fix and tests.

It also includes the `secrets_updated_at` columns needed for story
#27351. These columns are currently unused and always NULL.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-20 14:43:04 -05:00
Victor Lyuboslavsky 3989d55182 GitOps support for DigiCert and Custom SCEP (#27229)
For #26622

This PR includes:
- Making DigiCert client a real service that can be overridden in tests
- GitOps support for DigiCert and Custom SCEP configs

# Checklist for submitter

- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-20 11:36:00 -05:00
Victor Lyuboslavsky 6b7d232522 Additional CA validation (#27169)
For #26623

- Updated `github.com/groob/plist` to `github.com/micromdm/plist` -- it
was renamed
- Added validation that restricts DigiCert Fleet variables to
`com.apple.security.pkcs12` payloads plus additional restrictions
- Added validation that restricts Custom SCEP Fleet variables to
`com.apple.security.scep` payloads plus additional restrictions
- Enabled multiple CAs (Fleet variables) to be present in an Apple MDM
profile. But each CA can only be used once. For example, we can have
DigiCert CA and Custom SCEP CA in one Apple profile.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-19 08:27:55 -05:00
Ian Littman fdff6e16ca Revise FMA list endpoint to match Windows FMA spec (#27180)
For #26652. No changes file as that'll come in another PR. Will stack
additional PRs on top of this one (for ingestion changes etc.) to get
this merged more quickly.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-17 10:09:39 -05:00
Victor Lyuboslavsky 84d35dc0cd Add custom SCEP configs (#27045)
For #26603 

This PR includes:
- Refactoring of NDES/SCEP verify/timeout logic for easier testing (with
dependency injection)
- Custom SCEP configs
- saving/deleting/updating of encrypted custom SCEP challenges
- validation call to custom SCEP server to verify connection
- Custom SCEP activities
- unit and integration tests for all of the above

This PR does not include the following:
- Changes file (in later PR)

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-03-14 12:16:51 -05:00
Victor Lyuboslavsky a86253d2bf Refactoring Android datastore interface (#26982)
For #26219 

Refactoring the interface between Android service and Android datastore
to use 1 common datastore interface: `fleet.AndroidDatastore`

These changes are based on feedback from the recent Backend Sync.

```mermaid
---
title: Partial class diagram
---
classDiagram
    direction LR
    class `android.Service`
    <<interface>> `android.Service`
    class `android/service.Service`
    `android/service.Service` ..|> `android.Service`: implements

    class `fleet.AndroidDatastore`
    <<interface>> `fleet.AndroidDatastore`
    class `fleet.Datastore`
    <<interface>> `fleet.Datastore`
    class `android.Datastore`
    <<interface>> `android.Datastore`
    `android/service.Service` *-- `fleet.AndroidDatastore`: USES (THIS IS THE KEY CHANGE)
    `fleet.Datastore` *-- `fleet.AndroidDatastore`: contains
    `mysql.Datastore` ..|> `fleet.Datastore`: implements
    `fleet.AndroidDatastore` *-- `android.Datastore`: contains
    `mysql.Datastore` *-- `android.Datastore`: contains
    `android/mysql.Datastore` ..|> `android.Datastore`: implements
```
2025-03-13 14:28:52 -05:00
Dante CatalfamoandScott Gress d5e9153cea Add LabelsIncludeAny to queries endpoint (#26823)
#16413 

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
2025-03-11 09:45:01 -04:00
Victor Lyuboslavsky b42dbf2ff7 DigiCert backend (#26914)
For #26609 

This PR includes
- ability to get a DigiCert certificate to a macOS device
- integration test for the above
- some validation

This PR does not include the following. They will be included in
subsequent PRs:
- support for User Principal Name in certificate
- support for $FLEET_VAR_HOST_HARDWARE_SERIAL
- saving certificate expiration date
- not resending DigiCert profile after failure

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-10 13:02:49 -05:00
Ian LittmanandRachael Shaw 014f10fb46 Add experimental software title name update endpoint for titles with a bundle ID (#26938)
For #26933.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Rachael Shaw <r@rachael.wtf>
2025-03-07 11:36:17 -06:00
Victor Lyuboslavsky 9deabe8f70 DigiCert config (encrypt API token) (#26858)
For #26603

This PR includes:

- DB migration for DigiCert
- new datastore methods and tests
- saving/deleting/updating of encrypted DigiCert API tokens
- Integration test for DigiCert configs

This PR does not include the following:

- Making an HTTP call to DigiCert API for validation (in later PR)
- Changes file (in later PR)

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-03-06 12:47:57 -06:00
Martin Angers a4f104fcce Android: compute MDM enrollment stats (#26854) 2025-03-05 15:47:06 -05:00
Martin Angers 662bb2919c Bugfix: Clear the unified queue when turning off and re-enrolling in MDM (#26816) 2025-03-05 08:59:07 -05:00
Victor Lyuboslavsky 8f1626303a Added custom auth for PubSub push endpoint. (#26664)
For #26219 

Need to get in this security change for the RC

- Added auth (token stored in `mdm_config_assets`) for PubSub endpoint
- unknown notification types, including `test` are not authenticated
- Status reports for deleted devices are not authenticated -- no DB
changes are done for these since they only happen when we disable MDM
right now

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
2025-02-27 15:16:32 -06:00
Victor Lyuboslavsky 67b72764c5 Added Android activity and better handling of deleted users. (#26640)
For #26218

- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities

Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-27 14:19:15 -06:00
Martin Angers 0adf67e538 Fix conflicts 2025-02-25 14:39:35 -05:00
Victor Lyuboslavsky bf7917b503 Basic Android enroll functionality (#26386)
For #26219 

Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50

This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device

Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-24 14:31:21 -06:00
Martin Angers 8477856886 FIx conflicts 2025-02-24 14:28:34 -05:00
Sarah Gillespie 7b6e212003 Add datastore methods for host certificates feature (#26416) 2025-02-18 17:49:02 -06:00
Victor Lyuboslavsky fc33df1981 Basic Android MDM on/off functionality (#26309)
For #26218 

Basic Android MDM on/off backend functionality. Manually tested.

The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```

I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512

Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-18 09:43:11 -06:00
Jahziel Villasana-Espinoza 07486aef30 fix: update App Store app versions on an hourly basis (#26326)
> For #24222

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-17 14:56:24 -05:00
Ian Littman 30e5043919 Switch to a simpler, more reliable query for checking if the initial admin user has been added (#26012)
For #26011, follow-up work for #26003.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-12 07:52:31 -06:00
Martin Angers 606df3f349 Upcoming Activities feature branch (#25450) 2025-02-11 14:53:11 -05:00
Jahziel Villasana-Espinoza 5c730d80ce fix: clear auto install policy statuses for vpp app scope changes (#26121)
> For #26086

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-07 09:53:54 -05:00
d38d180357 Add labels and editing for VPP apps (#25979)
For #24609

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
Co-authored-by: Jahziel Villasana-Espinoza <jahzielv@gmail.com>
2025-02-03 11:16:21 -06:00
Scott Gress 1cd37ef966 Update NewLabel method to use more efficient update mechanism (#25777)
For #25555 

# Checklist for submitter

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

This PR updates the `NewLabel` service to use the
`UpdateLabelMembershipByHostIDs` method previously added by
@jacobshandling rather than using `ApplyLabels`. The latter method has
performance issues when adding large numbers of hosts at once to a
manual label (see #25555) because it does an expensive lookup of host
names before transforming those into Fleet host IDs. The new code skips
the middleman and transforms host identifiers directly to Fleet host
IDs, and does so using a batching strategy to ensure the queries don't
get too large.

This PR does update `UpdateLabelMembershipByHostIDs` slightly to return
an updated Label object and host IDs array, as this is the expected
return value for `NewLabel`. I update the method's tests accordingly. I
don't think any new tests for `NewLabel` are needed as it should have
the same functionality and return values.

## Manual Testing

On the main branch, I launched my local MySQL with the thread stack size
set to the minimal allowed, and used the API to try and create a new
label with 5,000 hosts attached, and received a 422 response from the
server. Server logs showed:
```
level=error ts=2025-01-28T15:08:20.465401Z component=http user=scott@fleetdm.com method=POST 
uri=/api/latest/fleet/labels took=16.610292ms err="get hostnames by identifiers: Error 1436 (HY000): Thread stack 
overrun:  111136 bytes used of a 131072 byte stack, and 20000 bytes needed.  Use 'mysqld --thread_stack=#' to specify 
a bigger stack."
```

On this branch, I kept the same MySQL settings and tried my API request
again and it was successful:
<img width="776" alt="image"
src="https://github.com/user-attachments/assets/c4f0f52b-4d09-457b-8096-4dd3a747b1f4"
/>

## QA

The script I used to create a new manual label with lots of hosts is at:
https://gist.github.com/sgress454/84f12064c437da456c456e25c26d9069

To run it, first grab a bearer token from any API request by opening the
network tab, clicking a Fleet API request, and in the headers tab
scrolling down to Authorization:
<img width="892" alt="image"
src="https://github.com/user-attachments/assets/5680f3bf-8db8-469a-9f03-000b86622c04"
/>
(only take the part _after_ "Bearer")

Then download the script from that gist and in its folder run:
```
NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js <the bearer token> "<a label name>"
```
e.g.
```
NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js U3HpbdtadmJXGKYSB0U/PbwfOpHbBt7FpkWmGKKYolOO1moLNZA6XxP+QO5LVukvAotZ7d+JbNUEEhYHZtxoqg== "some test label"
```
This will invoke the API on https://localhost:8080 and try to add 5000
hosts a new label "some test label".

If you need to change the # of hosts or the url of the server, there are
additional arguments:
```
NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js <the bearer token> "<a label name>" <number of hosts> <url>
```
e.g.
```
NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js U3HpbdtadmJXGKYSB0U/PbwfOpHbBt7FpkWmGKKYolOO1moLNZA6XxP+QO5LVukvAotZ7d+JbNUEEhYHZtxoqg== "some test label" 10000 https://foo.bar
```
2025-01-31 09:19:36 -06:00
Dante Catalfamo 3c8033fa8e Edible Scripts Backend (#25739)
#24602
2025-01-30 13:01:51 -05:00
jacobshandlingandJacob Shandling f93b869f26 Update label membership by host IDs directly (#25687)
## For #25261 

<img width="826" alt="Screenshot 2025-01-23 at 11 07 19 AM"
src="https://github.com/user-attachments/assets/3a2f5d75-c0bf-445a-80dc-976914ff434e"
/>

### [Demo
video](https://drive.google.com/file/d/1ZFcrizkZ6zNODnTXjRC1f-Oeght5zOP4/view?usp=sharing)
- [x] Changes file added for user-visible changes in `changes/`, 
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-01-23 12:38:57 -08:00
Victor Lyuboslavsky 62b7412243 Disk encryption keys are now archived when created/updated (#25638)
For #25609 

Manual QA in progress. Putting this "In Review" since it is a P1.

Video explaining the PR: https://youtu.be/bUwIdjBLqiM

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-01-22 14:54:40 -06:00
Dante CatalfamoandLucas Manuel Rodriguez 347c65b5c6 Add options to populate users and labels on list hosts endpoint (#25621)
#22464

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-01-22 11:17:26 -05:00
Victor Lyuboslavsky 086099631e Fix issue where Windows MDM profiles are not sent to offline hosts (#25619)
For #25615 

The actual fix is these two lines, where we only delete the command from
the queue for the specific host we're processing:
```
const dequeueCommandsStmt = `DELETE FROM windows_mdm_command_queue WHERE enrollment_id = ? AND command_uuid IN (?)`
stmt, params, err = sqlx.In(dequeueCommandsStmt, enrolledDevice.ID, matchingUUIDs)
```

Everything else is tests, cleanup, refactoring for readability.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-01-21 09:59:32 -06:00