Commit Graph
1439 Commits
Author SHA1 Message Date
Victor Lyuboslavsky 38669539e4 Removing t.Parallel from migration tests. (#21589)
To fix nightly CI Go test fail.
2024-08-27 09:03:50 -05:00
Victor Lyuboslavsky dc4b45aba5 Deleting installed software now makes it available for install (#21487)
#21427

Also fixed #21488 
- Demo video:
https://www.loom.com/share/0a97afa680c74fa884ac07e87857098a

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-08-26 17:30:56 -05:00
Roberto Dip 4ee3a2e10a add missing comment (#21576) 2024-08-26 15:31:31 -03:00
Roberto Dip 08783b1a12 adjustments to support 60k MDM hosts after load testing (#21247)
for https://github.com/fleetdm/fleet/issues/20007
2024-08-26 15:20:57 -03:00
Victor Lyuboslavsky fdfc12982b Improvements to go tests in CI (#21545)
#21546 
Some improvements to overall go test CI run time.
2024-08-26 08:55:53 -05:00
Jahziel Villasana-Espinoza 7ed1327093 fix: stop panic when doing self install (#21480)
> Related issue: #21475

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-22 09:46:50 -04:00
Martin Angers e972d38bdc Bugfix: take VPP apps into account in "self-service" filter and My Device page (#21473) 2024-08-21 16:40:01 -04:00
Tim Lee 6a3efced81 Bugfix: Software Titles Filters (#21456) 2024-08-21 13:18:01 -06:00
Sarah Gillespie 27d22d29e5 Enforce latest OS when macOS, iOS, and iPadOS hosts automatically enroll (#21291) 2024-08-21 13:21:11 -05:00
Victor Lyuboslavsky 734357af14 Deleting an ABM iOS/iPadOS host keeps it in Fleet. (#21433)
#21073 
- Deleted iOS/iPadOS host will continue to report to Fleet as long as
host is in Apple Business Manager (ABM).
- Refetching an offline iOS/iPadOS host will not add new MDM commands to
the queue if previous refetch has not completed yet.

Video demo:
https://www.loom.com/share/2f7ecb22e1924d4cbbbdd7dd297439ef?sid=dbfe1939-cb46-47ca-a7a8-84965ed68a7e

I considered `nano_command_results` but could not think of an efficient
way to use `nano_command_results` to determine if refetch was already
done. The problem is that we're overloading `command_uuid` to include
the `REFETCH` identifier. So we can't simply add an index on `(uuid,
command_uuid, updated_at)`. We need to do a partial text match on
`command_uuid` which then requires a full scan of the matching
`updated_at` timestamps. It feels like a search like this could blow up
for a large `nano_enrollment_queue` and `nano_command_results` table.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-08-21 08:51:04 -05:00
Tim Lee d0e271f51f Bugfix: add filter to counts (#21411) 2024-08-19 16:55:59 -06:00
Tim Lee 5b9074836e Bugfix: remove filter validation (#21399) 2024-08-19 14:14:01 -06:00
Victor Lyuboslavsky 48ea9984f8 Updating migrations timestamps for ordering to be correct (#21373)
Updating migrations timestamps for ordering to be correct when migrating
from v4.55.1
2024-08-16 17:53:04 +02:00
Victor Lyuboslavsky 4eb72535dc Support for MySQL 8.4.2 (#21364)
#21270

The main change for MySQL 8.4.2 is that foreign key constraints are
stricter:
https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_restrict_fk_on_non_standard_key

Also, most replica-related commands have been renamed.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
2024-08-16 16:32:38 +02:00
Tim Lee c2635fe05b Add Software Vulnerability Filters (#21312) 2024-08-15 12:36:47 -06:00
Dante Catalfamo 9a5e5372eb SSVPP Backend and CLI (#21132)
#19882 and #20532
2024-08-14 10:25:32 -04:00
Victor Lyuboslavsky 985781a04b Added index to query_results DB table to speed up finding last query timestamp for a given query and host. (#21303)
#21296 
Added index to query_results DB table to speed up finding last query
timestamp for a given query and host.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Manual QA for all new/changed functionality
2024-08-14 15:00:27 +02:00
Martin Angers 08763d59f1 Feature branch for the Store Bootstrap Packages in S3 story (#21260) 2024-08-13 08:27:10 -04:00
Victor Lyuboslavsky ca4c2cea72 Update label name in configuration profile (#21246)
#21163 
Fixed bug where configuration profile was still showing the old label
name after the name was updated.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-13 13:57:32 +02:00
Gabriel HernandezandRoberto Dip 6799cdcb6a fix to return software titles url for all teams context (#21222)
relates to #21058

Makes a change to `GET /software/titles/:id` response so that we return
the data needed to display the VPP app icon for the **All Teams**
context.


![image](https://github.com/user-attachments/assets/6cf48c04-5713-4b9e-b310-cee91367c37f)

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-08-12 17:23:44 -03:00
Sharon Katz 8fdfb9a004 Add AI features, maintenance window, and Fleet Desktop to usage statistics Story #19693 (#21131)
Additional statistics as described in
https://github.com/fleetdm/fleet/pull/20091 :

`aiFeaturesDisabled`:
Whether server_settings.ai_features_disabled is set to true in the
config.

`maintenanceWindowsEnabled`:
Whether at least one team has
integrations.google_calendar.enable_calendar_events set to true

`maintenanceWindowsConfigured`:
Maintenance windows are considered "configured" if:
configuration has value set for integrations.google_calendar[0].domain
configuration has value set for
integrations.google_calendar[0].api_key_json

`numHostsFleetDesktopEnabled`:
The number of hosts with Fleet desktop installed.
2024-08-12 15:12:28 -04:00
Roberto Dip a794bd29a5 Fixed a bug when a cached prepared statement gets deleted in the MySQL server (#21219)
for #20781, I found this hard to reproduce, but apparently can happen in
AWS Aurora when the server is upgraded under the hood.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-09 11:59:24 -03:00
Victor Lyuboslavsky b67017398b Added known_vulnerability to vulnerabilities endpoint. (#21136)
#19857 
For `GET /api/v1/fleet/vulnerabilities` endpoint, added
`known_vulnerability` field to the response. This field is present when
query is a valid CVE format and returns no results. It indicates whether
the vulnerability is in Fleet's DB.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-08 21:37:25 +02:00
Dante Catalfamo 90f0414e21 Add self_service column to vpp_apps_teams (#21106)
Migration for #19882

Making the migration its own PR so it doesn't become stale.
2024-08-07 09:51:24 -04:00
Lucas Manuel Rodriguez fb4292ce08 Fix many issues with VPP apps select queries and host counts (#21039)
Fix for #21013 and other fixes found while testing VPP apps with
iOS/iPadOS/macOS devices.

Regarding the change in ListSoftwareTitles with `TeamID=nil`, from Noah:
https://github.com/fleetdm/fleet/issues/19447#issuecomment-2270006803
2024-08-06 18:00:52 -03:00
Benjamin Edwards 60a0609ef7 add index to jobs table (#21090) 2024-08-06 14:49:01 -04:00
Martin Angers e65d6cfa06 Bugfix: don't show as available for install a software with an install request once host is moved/installer is deleted (#21064) 2024-08-06 12:31:01 -04:00
Martin Angers ef07a406cc Bugfix: list host software "Available for install" filter must show installers that have an install request on the host (#21083) 2024-08-06 10:14:01 -04:00
Jahziel Villasana-EspinozaandSarah Gillespie 18977f324b fix: panic during migration (#21031)
> Related issue: #21030

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2024-08-05 09:23:15 -04:00
Jahziel Villasana-Espinoza 5f7a644e3c fix software array migration (#21010)
> Related issue: #20978 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Manual QA for all new/changed functionality
2024-08-02 18:50:03 -04:00
Lucas Manuel Rodriguez ab7df5155d Use docker compose on CI instead of docker-compose (#21017)
After this is merged I'll cherry pick to `minor-fleet-4.55.0`.
2024-08-02 18:12:36 -03:00
Dante Catalfamo 0a15647e10 Host software deleted at remigration (#20996)
# Recreate out of order migration, replace `docker-compose` with `docker compose` in db test runner
2024-08-02 10:47:40 -04:00
Dante Catalfamo bf3d849bd0 Fix Deleted Host Software Installs Details (#20820)
#20271

This fix requires an accompanying front end fix to pass the
`display_name` from the activity feed to the modal when displaying the
install results.
2024-08-01 15:41:37 -04:00
Victor Lyuboslavsky 580a81ded5 Fixed duplicate app install activity on host (#20944)
Unreleased bug for iOS/iPadOS VPP #19447

# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-08-01 20:32:45 +02:00
Roberto Dip 7a080a9b36 use Escrow Buddy to rotate FileVault keys on macOS (#20842)
back-end and agent part of #13157

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-07-31 16:59:30 -03:00
Sarah Gillespie a33bd65272 Revert "Add minimum os version requirements to DEP enrollment flow" (#20878) 2024-07-31 11:34:01 -05:00
Victor Lyuboslavsky 438b0ff4d6 Added available_for_install query parameter to host details (#20876)
Missing part of #19447
Added `available_for_install` query parameter to `GET
/api/v1/fleet/hosta/:id/software`

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-31 16:27:43 +02:00
Tim Lee bda812c329 Hotfix: Pass sql test dump in via stdin (#20871) 2024-07-31 05:09:52 -06:00
Victor Lyuboslavsky 31224ccd34 Added VPP support for iOS/iPadOS (#20837)
#19447 
iOS and iPadOS apps can be installed using Apple's VPP (Volume Purchase
Program)

VPP apps are now using a composite primary key (Adam ID and platform)
because we want to keep iOS/iPadOS/macOS separate. It is possible for
one app to be installable on all Apple platforms.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-07-30 22:43:51 +02:00
Tim Lee 0e9705fdf7 Add No Team to Software Backend (#20822) 2024-07-30 11:19:05 -06:00
Sarah Gillespie 53ba476516 Fix unreleased bug: API error message for duplicate VPP apps (#20808) 2024-07-29 15:46:27 -05:00
Sarah Gillespie 0f98e84bc8 Add minimum os version requirements to DEP enrollment flow (#20722) 2024-07-29 11:02:27 -05:00
Martin Angers cc4c34ce8a Unreleased bug fix: use of db.writer inside a transaction (#20803) 2024-07-29 11:56:01 -04:00
Victor LyuboslavskyandRoberto Dip 671fc628e7 Fetch installed apps from iPhone/iPad devices. (#20733)
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision

Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-28 16:17:27 +02:00
Jahziel Villasana-Espinoza dd3967c4ec feat: enforce FileVault during Setup Assistant (#20759)
> Related issue: #16866

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-26 15:50:31 -04:00
Jahziel Villasana-Espinoza 5b97d01982 chore: merge main 2024-07-24 16:50:43 -04:00
Jahziel Villasana-Espinoza fda37851c1 fix: move last install data into apps and packages (#20664)
> Related issue: #20662

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 13:39:23 -04:00
Lucas Manuel Rodriguez 70d4558448 Backend support for iOS/iPadOS OS updates (#20649)
#20469 and #20471

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-07-24 14:34:23 -03:00
Roberto Dip e4d8fcc3a5 fix: adjust host filters for VPP software (#20663)
this allows to filter hosts with pending, failed and installed VPP apps.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-23 16:56:24 -03:00
Roberto Dip 845bfc7e38 rearrange migrations and generate dump (#20652)
this re-arranges migrations so the VPP schema changes happen after
already merged migrations and commits the result of running `make
dump-test-schema` using mysql 8.

note that there are a bunch of seemingly unrelated changes in the diff,
but that's just the mysql 8 bump, the diff against `main` should be
smaller.
2024-07-23 11:01:23 -03:00