Commit Graph
2158 Commits
Author SHA1 Message Date
Victor Lyuboslavsky 759003e37d Fixing false negative vulnerabilities on macOS Homebrew python packages. (#17709)
#17061

TODO: Need to also merge this fix into patch branch.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-19 14:12:07 -05:00
Lucas Manuel Rodriguez cf64d85deb Add visual studio extensions to software inventory (#17501)
#17003

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-03-14 16:33:12 -03:00
Sarah Gillespie c2a7c670fa Handle null case in datastore method to get host disk encryption status (#17541) 2024-03-14 10:01:20 -05:00
Jahziel Villasana-Espinoza 2b9487197c feat: script content cleanup and column removal (#17374)
> Related issue: #17374

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-13 15:53:08 -04:00
Tim Lee 8be1d4766f MS Edge Translation (#17171) 2024-03-13 12:47:31 -06:00
Martin Angers 0f8192348b Add missing index to MDM Windows enrollments table (#17559) 2024-03-13 11:27:37 -04:00
Martin Angers 2522cc5ef6 Recategorize MDM endpoints to new mdm-less paths (#17372) 2024-03-13 10:27:29 -04:00
Martin Angers c358bde87b osquery-perf: add support for Windows MDM enrollment and session management. (#17522) 2024-03-13 09:29:25 -04:00
Victor Lyuboslavsky ad5c0a90be Team user should not access OS version on another team. (#17347)
#17117 
For `fleet/os_versions` and `/fleet/os_versions/[id]`, team users can no
longer access os versions on hosts from other teams.

### Team admin /os_versions - only returns os versions for the user's
team(s)
GET https://localhost:8080/api/v1/fleet/os_versions

### Team admin /os_versions/:id on 'No Team' - 403
GET https://localhost:8080/api/v1/fleet/os_versions/5

### Global admin /os_versions/:id?team_id does not exist anywhere - 404
GET https://localhost:8080/api/v1/fleet/os_versions/999999?team_id=1

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-13 08:20:00 -05:00
Martin Angers 8d8181eb0d Fix flaky test due to package-level test variable set in a separate test (#17393) 2024-03-13 09:12:19 -04:00
Victor Lyuboslavsky 1052b6b350 Reducing the number of deadlock errors. (#17290)
#16562

Based on local testing, these changes reduce the number of 'Deadlock
found' errors when multiple hosts share the same UUID.

The performance issues with vulnerability processing will be fixed as
part of issue https://github.com/fleetdm/fleet/issues/16858

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-13 06:58:29 -05:00
Lucas Manuel Rodriguez e9a464e0cf Add exponential backoff to orbit enroll retries (#17368)
#16594

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-13 07:57:00 -03:00
Jahziel Villasana-EspinozaandSarah Gillespie 9e147494a3 fix: process Add commands in Windows MDM profiles (#17570)
> Related issue: #17563

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2024-03-12 15:21:17 -04:00
Tim Lee c4750d61f5 bugfix: add counts_updated_at (#17543) 2024-03-11 14:14:29 -06:00
Tim Lee d4ba30cfa1 17257 validate bulk ops request filters (#17514) 2024-03-11 10:02:51 -06:00
Victor Lyuboslavsky b174a476a2 Fixing unreleased spec bug in team host status webhook feature. (#17502)
Fixing unreleased spec bug in team host status webhook feature #17094.
Bug #17498
# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
  - Not needed. Part of new feature.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-08 15:09:33 -06:00
Roberto Dip 525233efea fix unreleased bug with ADE profile assignment (#17509)
for unreleased #17508
2024-03-08 16:45:18 -03:00
Jacob ShandlingandJacob Shandling 2584f6794a Add integration test for change password endpoint (#17319)
## Addresses #16863 

- [x] Added/updated tests

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-03-08 08:28:00 -08:00
Roberto Dip a39932b013 ammendment to properly fix #17489 (#17499)
for #17489, found an edge case during extra testing.
2024-03-08 13:15:04 -03:00
Lucas Manuel Rodriguez 37885d0932 Remove filtering of private IPs in public IP field (#17352)
#11102

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-08 12:57:18 -03:00
Roberto Dip 1be3aa2ffb improve MDM detection with enrollment state checks (#17494)
for #17489
2024-03-08 12:21:07 -03:00
Tim Lee 2e67ef61d4 Bugfix: Revert host filters fix (#17390) (#17464)
This reverts commit 4b2ebdc8dc.
2024-03-07 12:57:27 -07:00
Victor Lyuboslavsky a173be8f52 For fleetctl gitops, when MDM configs are not explicitly defined in gitops yml file, they are now set to default values. (#17223)
For fleetctl gitops, when MDM configs are not explicitly defined in
gitops yml file, they are now set to default values.
#17209

Gitops role can now read org config/settings. This is used to determine
whether license is Premium.
Doc changes for permission access:
https://github.com/fleetdm/fleet/pull/17238

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-07 13:20:14 -06:00
Roberto Dip e8b7ce81dc don't run SCEP test in parallel to avoid race in dependency (#17457)
The https://pkg.go.dev/go.mozilla.org/pkcs7 package uses an unguarded
package-level variable:


https://github.com/mozilla-services/pkcs7/blob/33d05740a3526e382af6395d3513e73d4e66d1cb/ber.go#L19-L21

This was causing this data race during tests:

```
==================
WARNING: DATA RACE
Read at 0x0000009b89b0 by goroutine 22:
  go.mozilla.org/pkcs7.asn1Structured.EncodeTo()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:21 +0x56
  go.mozilla.org/pkcs7.(*asn1Structured).EncodeTo()
      <autogenerated>:1 +0xa5
  go.mozilla.org/pkcs7.ber2der()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:68 +0xdb
  go.mozilla.org/pkcs7.Parse()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/pkcs7.go:160 +0x12f
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep.ParsePKIMessage()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep.go:234 +0x1d2
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.testParsePKIMessage()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:21 +0x56
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest.func1()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:198 +0x7e5
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Previous write at 0x0000009b89b0 by goroutine 23:
  go.mozilla.org/pkcs7.asn1Structured.EncodeTo()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:21 +0x6e
  go.mozilla.org/pkcs7.(*asn1Structured).EncodeTo()
      <autogenerated>:1 +0xa5
  go.mozilla.org/pkcs7.ber2der()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/ber.go:68 +0xdb
  go.mozilla.org/pkcs7.Parse()
      /home/runner/go/pkg/mod/go.mozilla.org/pkcs7@v0.0.0-20210826202110-33d05740a352/pkcs7.go:160 +0x12f
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep.(*PKIMessage).DecryptPKIEnvelope()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep.go:344 +0xde
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest.func1()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:199 +0x7fa
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Goroutine 22 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x845
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:165 +0x3ac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44

Goroutine 23 (running) created at:
  testing.(*T).Run()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x845
  github.com/fleetdm/fleet/v4/server/mdm/scep/scep_test.TestNewCSRRequest()
      /home/runner/work/fleet/fleet/server/mdm/scep/scep/scep_test.go:165 +0x3ac
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1595 +0x261
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.7/x64/src/testing/testing.go:1648 +0x44
==================
    testing.go:1465: race detected during execution of test
=== NAME  TestNewCSRRequest/KeyEncipherment_not_set_with_NOP_certificates_selector
```
2024-03-07 12:01:52 -03:00
Jacob ShandlingandJacob Shandling e957292746 Update reference in mdm integration test (#17452)
Add one missed update

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-03-06 23:01:21 -03:00
Roberto Dip 9c5b59df0d prevent redundant ADE profile assignment (#17427)
For #17291, this prevent re-assigning profiles to ABM hosts that already
have the right one.

This was happening very frequently for hosts that are in the last page
of the `/sync` request, as there's no indication that the cursor was
exhausted and we keept on assigning profiles to those hosts.

This caused profile assignment to eventually fail, presumably due to
rate limiting.
2024-03-06 18:16:10 -03:00
Tim Lee 4b2ebdc8dc 17257 host filters fix (#17390) 2024-03-06 14:15:53 -07:00
Jahziel Villasana-Espinoza babf4e17c8 fix: return bad request error during MDM migration when appropriate (#16551)
> Related issue: https://github.com/fleetdm/confidential/issues/5138

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-06 15:38:44 -05:00
Jahziel Villasana-Espinoza c808bba438 fix: don't show windows hosts when filtering by bootstrap package status (#17386)
> Related issue: #17341

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-06 15:07:48 -05:00
Sarah Gillespie ba211437c6 Update integration tests for run script by name feature (#17381) 2024-03-05 15:12:52 -06:00
Martin Angers 0bb6d45596 Merge branch 'main' into feat-scripts-char-limit 2024-03-05 10:10:04 -05:00
Sarah Gillespie c29f0abf92 Update API and CLI to enable running scripts by name and team id (#17322)
TODO:
- Integration tests

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-03-05 08:53:17 -06:00
Martin Angers 16ba308039 Store script contents for wipe host via script 2024-03-05 08:51:57 -05:00
Roberto Dip 7a7b043014 allow gitops role to read hosts by identifier and profiles (#16688)
for #15337
2024-03-05 10:43:49 -03:00
Martin Angers 88eeb9c1a4 Reorder database migrations 2024-03-05 08:22:11 -05:00
Martin Angers 98bc2dc29a Fix conflicts 2024-03-05 08:11:28 -05:00
Victor Lyuboslavsky 7bbb4a64bb Reduced vulnerability processing time by grouping vulnerability dictionary by vendor. (#17298)
Reduced vulnerability processing time by grouping vulnerability
dictionary by vendor.
#16858 

Notice `cvefeed.(*Cache).Get`
Before:
<img width="1997" alt="image"
src="https://github.com/fleetdm/fleet/assets/2685025/a3cc5571-cee2-40a6-8b2d-cfed1323d96c">
After:

![image](https://github.com/fleetdm/fleet/assets/2685025/89484f87-a4e9-4c5c-943f-138e1c27a2c6)


# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 16:16:32 -06:00
Victor Lyuboslavsky 05ff1ca52a Adding backend support for running live queries with team_id=0 (No team) (#17258)
- API endpoint GET fleet/targets/count can target 'No team' with
team_id=0
- API endpoint POST fleet/queries/run (for async live queries) can
target 'No team' with team_id=0
#16350

API doc changes PR: https://github.com/fleetdm/fleet/pull/17267

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 16:08:10 -06:00
RachelElysia 316321e889 [released bug] Fleet Full stack: Allow teams filter when transferring hosts by filters (#17241) 2024-03-04 15:48:19 -05:00
Roberto Dip e3712fb263 Don't run cleanup tasks or enqueue commands on SCEP renewals (#17303)
for #17232
2024-03-04 16:53:16 -03:00
Martin Angers 0858f5a6f4 Scripts char limit: improve database migration performance and memory usage (#17338) 2024-03-04 13:51:32 -05:00
Victor Lyuboslavsky 592a7450e3 Enabling setting host status webhook at the team level via REST API and fleetctl apply/gitops. (#17186)
Enabling setting host status webhook at the team level via REST API and
fleetctl apply/gitops.
#14916

Example payload:
```json
{
    "data": {
        "days_unseen": 3,
        "host_ids": [
            10724,
            10726,
            10738,
            10739,
            10740,
            10741,
            10742,
            10744,
            10745,
            10746,
            10747,
            10748,
            10749
        ],
        "team_id": 3,
        "total_hosts": 15,
        "unseen_hosts": 13
    },
    "text": "More than 86.67% of your hosts have not checked into Fleet for more than 3 days. You've been sent this message because the Host status webhook is enabled in your Fleet instance."
}
```

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 12:35:27 -06:00
Lucas Manuel Rodriguez 4acb713bf1 osquery-perf changes for Fleet downtime load test (#17310)
Including all the osquery-perf changes needed to perform #16423.
2024-03-04 15:10:10 -03:00
Jahziel Villasana-Espinoza 33a0324ebb feat: adding, reading, and deleting flows for scripts updated to use new table (#17305)
> Related issues: #16842, #16843

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-04 11:00:08 -05:00
Jahziel Villasana-Espinoza f3754e2674 fix: use correct copy in fleetctl error when applying profiles (#17318)
> Related issue: #16335, fixes issue QA found here:
https://github.com/fleetdm/fleet/issues/16335#issuecomment-1971560894

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-01 18:22:30 -05:00
Roberto Dip 730f8850ff Add mdm wipe host feature (#17272)
relates to #9951

This adds the mdm host wipe feature to fleet. This includes:

1. adding wipe functionality in the fleet web UI
2. adding wipe functionality in the fleetctl CLI
3. adding API endpoints to wipe a host
4. Implementing wipe functionality on the fleet server.
2024-03-01 18:02:00 -03:00
Roberto Dip 152edb838b fix flaky MDM test (#17307)
this one seems to be specially flaky and is blocking a couple of PRs
from being merged.
2024-03-01 17:54:42 -03:00
Gabriel Hernandez e495144023 update schema file 2024-03-01 17:38:08 +00:00
Gabriel Hernandez 306133ae3b udpate migration timestamp for add fleet platform to host mdm action migration 2024-03-01 17:34:05 +00:00
Tim Lee 7727978cb4 Skip nonASCII Software Names in Vuln Processing (#17193) 2024-03-01 10:31:46 -07:00