Commit Graph
134 Commits
Author SHA1 Message Date
Ian Littman e96c70e4c0 Pull xz'd goval-dictionary sqlite files to evaluate vulnerabilities on Amazon Linux hosts (#21506)
#20934

This is tied to https://github.com/fleetdm/vulnerabilities/pull/14; for
supported OS versions (currently Amazon Linux 1/2/2022/2023) we'll pull
XZ'd sqlite files from the vulnerabilities repo and query them to
determine what's vulnerable. See the associated issue for how I
self-QA'd this.

This replaced OVAL parsing for Amazon Linux 2, as we were using the
wrong data source there (Amazon has backported a bunch of fixes to their
own-named releases, so any RHEL fixes don't match).

Some checklist items are missing here; getting this set up in draft to
get code feedback now, and I'll push updates with e.g. docs changes, as
well ass an addition to the changes file.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
    - [x] Add tests to oval_platform
    - [x] Add sync_test
    - [x] Add database_test
- [x] Manual QA for all new/changed functionality
- [x] Update vulnerability management docs
2024-08-26 14:07:42 -05:00
Victor Lyuboslavsky fdfc12982b Improvements to go tests in CI (#21545)
#21546 
Some improvements to overall go test CI run time.
2024-08-26 08:55:53 -05:00
Tim Lee e376d3ddbb Python Vulnerability False Positive (#21493) 2024-08-23 11:59:25 -06:00
Lucas Manuel Rodriguez 415cccc121 Add matching rules for Microsoft 365 for July and August 365 (#21410)
#20409

I used `Current Channel`'s build version from
[here](https://learn.microsoft.com/en-us/officeupdates/microsoft365-apps-security-updates)
for `ResolvedInVersion`. Please @mostlikelee let me know if that's a-ok
(mimicked from June's change).

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-08-20 11:35:44 -03:00
Ian Littman 605779eee3 Warn on CVE description language tags from NVD feed that we haven't seen before (#21310)
We may need more effort to alert on this in a place where we can
see/action it, but for that alerting we can just catch warn (or
warn-and-above) logs, as I just ran the artifact build command and
didn't get any warnings.

Confirmed that this works by starting without the "es" case (since I
wasn't sure which language tags other than "en" and "en-US" we were
seeing) and seeing a bunch of CVEs from 2004 with "es" language tags
reported as warnings. Also confirmed (`gzcat
cvefeed/nvdcve-1.1-2024.json.gz | grep -A 5 -B 5 "description_data"`)
that language tags were correctly set in the resulting feed (caught a
bug locally due to Golang having an implied break at the end of each
switch case).

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
2024-08-19 10:42:24 -05:00
Ian Littman 49300bc844 Don't panic on zero-length NVD description_data array fields (#21250)
#21242

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

QA'd manually (see repro scenario in the linked bug). Happy to dig into
adding tests here if it's worth the time to build them now that the
upstream data feed has been patchd.
2024-08-14 10:53:47 -05:00
Victor Lyuboslavsky 488115d366 CPE fixes (#21244)
#20463 and #21173
- During vulnerability scanning, use 'macos' SW target for CPEs of
homebrew packages
- During vulnerability scanning, don't ignore software with non-ASCII en
dash and em dash characters

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-12 16:14:25 +02:00
Victor Lyuboslavsky 1176afbb0b Vuln cron fail fix (#21240)
#21239 

This PR fixes the generated vuln JSON files, which would get the current
customers up and running.

QA Done:

- New vuln JSON files generated at:
https://github.com/getvictor/vulnerabilities/releases
- Ran local server with new vuln JSON files by setting `export
TEST_VULN_GITHUB_OWNER=getvictor`
- Diff of JSON files from https://github.com/fleetdm/vulnerabilities and
https://github.com/getvictor/vulnerabilities

Steps for diff:
```
mkdir new
cd new
gh release download cve-202408111650 -D . -R getvictor/vulnerabilities
gunzip *.gz
cd ../
mkdir old
cd old
gh release download cve-202408111637 -D . -R fleetdm/vulnerabilities
gunzip *.gz
cd ..
diff old new
```

Diff results also remove a few Rejected CVEs from JSON files. The 2024
results also have a few minor diffs that don't seem significant.
2024-08-11 20:57:29 +02:00
Victor Lyuboslavsky 647ffc6bf1 Exclude ios_apps and ipados_apps from normal vuln processing. (#21143)
#21142
2024-08-07 19:46:33 +02:00
Victor Lyuboslavsky e0d36366fe Moving Go integration tests to integration test job (#21126)
#20929
Moving Go integration tests to integration test job, which runs tests
that start with `TestIntegrations`
Test changes only. No product changes
2024-08-07 14:00:25 +02:00
Tim Lee e3c3c870c9 Ignore Rejected CVEs Test (#19974)
#18913

This is a test to ensure Rejected CVEs do not match against software.
The related PR needs to be merged first, otherwise this test will fail:

https://github.com/fleetdm/fleet/pull/19972

- [X] Added/updated tests
2024-08-02 07:33:57 -03:00
Tim Lee 362a0e545e 18913 ignore rejected NVD vulnerabilities (#20193)
#18913 Recreating PR (ref: https://github.com/fleetdm/fleet/pull/19972)
2024-07-24 10:53:33 -06:00
Victor Lyuboslavsky 5cc5e527e5 Fixing Homebrew intellij/pycharm CE false negative. (#20461)
#19853
Fixing Homebrew intellij/pycharm CE false negative.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-15 12:47:10 -05:00
Tim Lee 2ea369c093 Custom Vulnerability Matching (#20118) 2024-07-09 11:50:22 -06:00
Tim Lee 587a42f2a3 Revert "Ignore Rejected CVEs" (#20183) 2024-07-03 05:58:05 -06:00
Tim Lee 47b7794fe8 Ignore Rejected CVEs (#19972) 2024-07-02 18:42:38 -06:00
Tim Lee 4dce1f6623 Windows Vuln False Positives (#20012) 2024-06-26 09:43:57 -06:00
Tim Lee cb5fb65c5e fix app vuln false positives (#19905) 2024-06-21 08:05:18 -06:00
Tim Lee 2d6458b3f8 Custom Ubuntu Kernel Vuln Scanning (#19588) 2024-06-17 15:44:01 -06:00
Tim Lee b137bcbef2 Hotfix - Vuln matching race (#19786) 2024-06-17 11:02:47 -06:00
Roberto Dip 1cc13a09fb 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774)
`go-kit/kit/log` was deprecated and generating warnings

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-17 10:27:31 -03:00
Tim Lee 0516cd61d0 App OS Vulnerability Matching (#19486) 2024-06-13 11:20:28 -04:00
Tim Lee 6d2471d5e5 vscode false positivies (#19508)
#18733 

Added a rule for the affected CVEs and made a change to the `IfIgnore`
func. Manually tested CVE removals with `/tools/nvd/nvdvuln`. Existing
test coverage ensures the rule doesn't cause false negatives on vscode
extensions.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-13 11:03:53 -04:00
Roberto Dip fa3f87a9a5 skip vuln tests until API keys are restored (#19446) 2024-06-03 14:06:41 -03:00
Tim Lee d1db2e3650 Ubuntu Kernel Vulns Part 2: Matching (#19303) 2024-05-29 06:59:12 -06:00
Tim Lee 14fa3e2cbe Ubuntu Kernel Vulns Part 1: Generate OVAL JSON (#19210) 2024-05-28 13:40:32 -06:00
Tim Lee 6f008b40f2 Fix backoff in vuln stream logging (#19148) 2024-05-28 08:13:46 -06:00
Tim Lee 7bba061ad2 Bugfix: Use hyphen delimiter in resolved_in_version (#17683) 2024-05-09 06:49:55 -06:00
Tim Lee 536ccc91c0 18601 add ubuntu sources (#18602) 2024-05-02 13:20:00 -06:00
Tim Lee 648628d3be Update Windows OS Version Reporting (#17682) 2024-05-01 12:02:16 -06:00
Jacob ShandlingandJacob Shandling d7821685a8 Migrate logic from nvdtools into Fleet (#18244)
## Addresses #15999

- [x] Added/updated tests
- [ ] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-04-24 15:25:59 -07:00
Tim Lee 9e1878e01c 18439 Vulncheck data processing bug (#18440) 2024-04-22 09:36:20 -06:00
Victor Lyuboslavsky 72cc82f60d Ignoring Valve Corporation's Steam client's vulnerabilities on Windows and macOS (#18380)
#18126
Ignoring Valve Corporation's Steam client's vulnerabilities on Windows
and macOS
- On Windows and macOS, the true version of the Steam client (like
2021-04-10) cannot be retrieved by standard methods used on other
software. We would need to create custom logic to retrieve the version
of the Steam client.
- Steam client automatically updates itself, so security risk is
somewhat mitigated.

QA note: Items from software_cve table are cleared after 2 hours.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-04-18 10:12:49 -05:00
Tim Lee 8194b6e379 Optimize cve/generate to use last release (#18269) 2024-04-15 13:17:28 -06:00
Tim Lee a47f59ef36 Implement NVD Feed with Vulncheck Data (#18168) 2024-04-15 09:44:44 -06:00
George Karr e73fd55c5b Updating vuln check support (#15174) 2024-04-10 10:04:50 -05:00
Roberto Dip f1b45a3865 friday tidy up party (#18106)
- add missing tests
- remove unused code
- remove unnecessary nested branches
2024-04-08 08:42:42 -03:00
Victor Lyuboslavsky 759003e37d Fixing false negative vulnerabilities on macOS Homebrew python packages. (#17709)
#17061

TODO: Need to also merge this fix into patch branch.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-19 14:12:07 -05:00
Lucas Manuel Rodriguez cf64d85deb Add visual studio extensions to software inventory (#17501)
#17003

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-03-14 16:33:12 -03:00
Tim Lee 8be1d4766f MS Edge Translation (#17171) 2024-03-13 12:47:31 -06:00
Victor Lyuboslavsky 7bbb4a64bb Reduced vulnerability processing time by grouping vulnerability dictionary by vendor. (#17298)
Reduced vulnerability processing time by grouping vulnerability
dictionary by vendor.
#16858 

Notice `cvefeed.(*Cache).Get`
Before:
<img width="1997" alt="image"
src="https://github.com/fleetdm/fleet/assets/2685025/a3cc5571-cee2-40a6-8b2d-cfed1323d96c">
After:

![image](https://github.com/fleetdm/fleet/assets/2685025/89484f87-a4e9-4c5c-943f-138e1c27a2c6)


# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-04 16:16:32 -06:00
Tim Lee 7727978cb4 Skip nonASCII Software Names in Vuln Processing (#17193) 2024-03-01 10:31:46 -07:00
Jacob Shandling 1cf7362968 Merge branch 'main' into 15919-vulnerabilities-page 2024-02-21 10:58:58 -08:00
Victor Lyuboslavsky 35ca4ee32b Add team filter to software detail APIs (#16876)
#16787
2024-02-18 07:14:20 -06:00
119d1df76f add permission check to software titles/versions endpoints (#16561)
relates to #16052

This adds a team permission check the `GET software/titles/:id`
endpoint. If the user should not be able to get the software title if it
is not on a host that is on the same team as the user (e.g. software
title 1 is on host 1, which is on team 1. A user who is only on team 2
should get a 403 response)

The UI is also updated to show the access denied error page when the we
receive a 403 response for the software title

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
2024-02-15 17:22:27 -03:00
Tim Lee 6d1eee6279 Add resolved_in_version for Windows OS Vulns (#16546) 2024-02-07 06:19:59 -07:00
Victor Lyuboslavsky 66ec651ec4 Switching from FixedBuild to FixedBuilds (part 2). (#16465)
Switching from FixedBuild to FixedBuilds (part 2).
#16412
2024-01-30 14:24:41 -06:00
Victor Lyuboslavsky 72705ef93a Switching from FixedBuild to FixedBuilds (part 1). (#16454)
Switching from FixedBuild to FixedBuilds (part 1).
#16412 

Converting msrc files from using FixedBuild to FixedBuilds. In part 2,
FixedBuild will be completely removed.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
2024-01-30 10:43:28 -06:00
Tim Lee ae87519938 16356 Unreleased bugfix for osversions filter (#16391) 2024-01-29 13:04:26 -07:00
Tim Lee 79b5baa297 4345 OS Vulnerabilities Backend (#16303)
#4345 

This backend feature branch includes the following PRs:

macOS Vuln Matching:
#15837 
#15990 
#16077 

Bugs / Issues:
#16004
 #15905 
#16226 

Windows Vuln Matching
#16047 
#16049 
#16085 
#16099 

API:
#16215
2024-01-24 12:18:57 -07:00