Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

improve vuln cpe matching on macos (#6985)

Browse Source 
* add cpe translations
* fix matching on target_sw
This commit is contained in:
Michal Nicpon
2022-09-01 10:02:07 -06:00
committed by GitHub
parent 93e684d524
commit 0709d1bc5c
17 changed files with 830 additions and 194 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Deploying/Configuration.md
+25 -3
View File
@@ -2056,7 +2056,10 @@ How often vulnerabilities are checked. This is also the interval at which the co
##### cpe_database_url
URL to fetch the CPE dictionary database from. Some users want to control where Fleet gets its database from. When Fleet sees this value defined, it downloads the file directly. It expects a file in the same format as can be found in https://github.com/fleetdm/nvd/releases. If this value is not defined, Fleet checks for the latest release in Github and only downloads it if needed.
You can fetch the CPE dictionary database from this URL. Some users want to control where Fleet gets its database.
When Fleet sees this value defined, it downloads the file directly.
It expects a file in the same format that can be found in https://github.com/fleetdm/nvd/releases.
If this value is not defined, Fleet checks for the latest release in Github and only downloads it if needed.
- Default value: `""`
- Environment variable: `FLEET_VULNERABILITIES_CPE_DATABASE_URL`
@@ -2066,16 +2069,35 @@ URL to fetch the CPE dictionary database from. Some users want to control where
cpe_database_url: ""
```
##### cpe_translations_url
You can fetch the CPE translations from this URL.
Translations are used when matching software to CPE entries in the CPE database that would otherwise be missed for various reasons.
When Fleet sees this value defined, it downloads the file directly.
It expects a file in the same format that can be found in https://github.com/fleetdm/nvd/releases.
If this value is not defined, Fleet checks for the latest release in Github and only downloads it if needed.
- Default value: `""`
- Environment variable: `FLEET_VULNERABILITIES_CPE_TRANSLATIONS_URL`
- Config file format:
```
vulnerabilities:
cpe_translations_url: ""
```
##### cve_feed_prefix_url
Similarly to the CPE dictionary, we allow users to define where to get the CVE feeds from. In this case, the url should be a host that serves the files in the path /feeds/json/cve/1.1/. Fleet expects to find there all the JSON Feeds that can be found in https://nvd.nist.gov/vuln/data-feeds. When not defined, Fleet downloads from the nvd.nist.gov host.
Like the CPE dictionary, we allow users to define where to get the CVE feeds.
In this case, the URL should be a host that serves the files in the path /feeds/json/cve/1.1/.
Fleet expects to find all the JSON Feeds that can be found in https://nvd.nist.gov/vuln/data-feeds.
When not defined, Fleet downloads from the nvd.nist.gov host.
- Default value: `""`
- Environment variable: `FLEET_VULNERABILITIES_CVE_FEED_PREFIX_URL`
- Config file format:
```
vulnerabilities:
cve_database_url: ""
cve_feed_prefix_url: ""
```
##### current_instance_checks
docs/Using-Fleet/Vulnerability-Processing.md
+62 -4
View File
@@ -222,8 +222,6 @@ The whole pipeline exists to compensate for these differences, and it can be div
parse --> execute
```
### Ingesting software lists from hosts
The ingestion of software varies per platform. We run a `UNION` of several queries in each:
@@ -244,13 +242,15 @@ As described briefly above, we do this by translating the NVD database of CPEs i
#### How accurate is this translation process?
This is the most error prone part of the process. The CPE can have some vagueness. This means that parts of it can be a `*`, which means when you match that CPE to a CVE it can match any of that part of the CPE.
This is the most error prone part of the process.
The CPE can have some vagueness.
This means that parts of it can be a `*`, which means when you match that CPE to a CVE it can match any of that part of the CPE.
If the CPE is too vague, the extreme case being all parts are `*`, all CVEs will match. You want a very specific CPE, but not too specific that a small error would make it not match a CVE (false negative).
Let's look into some examples of this stage.
#### tmux
##### Example: tmux
tmux is a Unix terminal utility to multiplex ttys. It appears listed like this in macOS:
@@ -280,6 +280,64 @@ Now things start to get slightly more tricky. We have to remove the `.app` suffi
These are two illustrative examples. The reality is that there is no map or list of all the software available and how it's presented in each platform, so the "software to CPE" translation process is going to be evolving constantly.
#### Improving accuracy
In order to improve the accuracy of matching software to CPEs, CPE translations rules are added for known cases where matching fails.
`server/vulnerabilities/cpe_translations.json` contains these rules and is included in the [NVD release](https://github.com/fleetdm/nvd/releases/latest).
##### Example: `ruby@2.7` installed via `homebrew`
The following CPE translation rule is used to reduce false positives when ruby is installed via homebrew.
This is needed because ruby is commonly included in the title in the CPE database.
This rule matches the software name `ruby` matching a regular expression pattern and installed using `homebrew`.
When searching for CPEs, the specifed `product` and `vendor` will be added to the filter critera.
```
[
{
"software": {
"name": ["/^ruby(@.*)?$/"],
"source": ["homebrew_packages"]
},
"translation": {
"product": ["ruby"],
"vendor": ["ruby-lang"]
}
}
]
```
##### CPE Translations (array[CPE Translation Entry])
##### CPE Translation Entry (object)
The CPE translation rule.
| Name | Type | Description |
| --- | --- | --- |
| `software` | array[CPE Translation Software] | The CPE translation software match criteria. |
| `translation` | array[CPE Translation] | The CPE translation. |
##### CPE Translation Software (object)
The CPE translation software match criteria. Used to match software collected from hosts. Fields are are AND'd together. Values inside each field are OR'd together.
| Name | Type | Description |
| --- | --- | --- |
| `name` | array[string] | The software name to match. Enclose within `/` to specify a regular expression pattern. |
| `bundle_identifer` | array[string] | The software bundle identifier (MacOS apps only) to match. Enclose within `/` to specify a regular expression pattern. |
| `source` | array[string] | The software source to match. Enclose within `/` to specify a regular expression pattern. |
##### CPE Translation (object)
The CPE translation. Used to match CPEs in the CPE database. Fields are are AND'd together. Values inside each field are OR'd together.
| Name | Type | Description |
| --- | --- | --- |
| `product` | array[string] | The CPE product. |
| `vendor` | array[string] | The CPE vendor. |
| `target_sw` | array[string] | The CPE target software. |
### Matching a CPE to a CVE
Once we have a good CPE, we can match it against the CVE database. We download the data streams locally and match each CPE to the whole list. The matching is done using the [nvdtools implementation](https://github.com/facebookincubator/nvdtools).