<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #44746 # Details * Adds the ability to filter historical CVE data by software type, EPSS, CVSS, CVE ID (exclude only) and "has known exploit" * Hard-codes the CVSS filter to 9.0+ for now, since that's the only data that's been collected thus far * Un-gates the collection code so that it will collect CVE data for _all_ severities (but still in the restricted set of software) Related PRs [update the front-end](https://github.com/fleetdm/fleet/pull/47674) to allow sending these filters, and [update GitOps](https://github.com/fleetdm/fleet/pull/47634) to allow changing the default filters. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually ### Manual test plan — CVE chart filtering (backend smoke test) #### Setup - Premium dev server running with a few hosts carrying vulnerable software (so `cve_meta` / `software_cve` / `operating_system_vulnerabilities` are populated) - Chart data present — collector ran once, or seeded: `go run ./tools/charts-backfill --dataset cve --use-tracked-cves --days 7` - API token exported and helper set: ```bash BASE=https://localhost:8080/api/v1/fleet/charts peak() { curl -sk -H "Authorization: Bearer $TOKEN" "$BASE/$1" | jq '[.data[].value] | max'; } #### Checks (compare against the no-filter baseline) - [x] Baseline returns data — GET /charts/cve?days=7 returns a data series; .filters is empty/default - [x] Severity force-pinned to critical — cve?days=7 and cve?days=7&severity_min=0&severity_max=10 give identical peaks (no low-severity leak; client severity ignored) - [x] Category narrowing — software_categories=browsers ≤ baseline; software_categories=os,browsers,office,adobe == baseline - [x] OS category includes kernel — software_categories=os returns OS-vuln + Linux-kernel CVE counts - [x] Known-exploit narrowing — known_exploit=true ≤ baseline - [x] EPSS narrowing — epss_min=0.9 ≤ baseline; epss_min=0&epss_max=1 == baseline (EPSS is 0.0–1.0 on the API) - [x] Exclude is subtractive + tolerant — excluding a visible CVE lowers/keeps counts; exclude_cves=CVE-0000-00000 == baseline (no-op) - [x] Filters echo back — filtered requests return applied values under .filters - [x] Uptime untouched — GET /charts/uptime?days=7 returns its normal series - [x] Free-tier safety (optional) — on non-Premium, /charts/cve returns an empty series, no error - [x] > 0 rows from: SELECT COUNT(DISTINCT scd.entity_id) AS below_critical FROM host_scd_data scd JOIN cve_meta cm ON cm.cve = scd.entity_id WHERE scd.dataset='cve' AND cm.cvss_score < 9.0; - (confirms lower-severity CVEs are stored) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Summary of changes * **New Features** * Added advanced CVE chart request filters: software categories, known-exploit flag, EPSS min/max, severity min/max, and excluded CVEs. * Expanded CVE chart coverage to use the full “collectible” CVE set, with filtering applied when serving chart data. * **Tests** * Added coverage for collecting collectible CVEs and resolving chart entities based on filter combinations and exclusions. * **Chores** * Updated CVE chart backfill to use collectible CVE discovery. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
66 lines
2.6 KiB
Go
66 lines
2.6 KiB
Go
package chart
|
|
|
|
import (
|
|
"context"
|
|
"time"
|
|
|
|
"github.com/RoaringBitmap/roaring"
|
|
"github.com/fleetdm/fleet/v4/server/chart/api"
|
|
)
|
|
|
|
// UptimeDataset implements api.Dataset for host uptime tracking.
|
|
type UptimeDataset struct{}
|
|
|
|
func (u *UptimeDataset) Name() string { return "uptime" }
|
|
func (u *UptimeDataset) DefaultResolutionHours() int { return 3 }
|
|
func (u *UptimeDataset) SampleStrategy() api.SampleStrategy { return api.SampleStrategyAccumulate }
|
|
func (u *UptimeDataset) DefaultVisualization() string { return "checkerboard" }
|
|
|
|
func (u *UptimeDataset) Collect(ctx context.Context, store api.DatasetStore, now time.Time, disabledFleetIDs []uint) error {
|
|
hostIDs, err := store.FindOnlineHostIDs(ctx, now, disabledFleetIDs)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if len(hostIDs) == 0 {
|
|
return nil
|
|
}
|
|
bucketStart := now.UTC().Truncate(time.Hour)
|
|
return store.RecordBucketData(ctx, u.Name(), bucketStart, time.Hour, u.SampleStrategy(),
|
|
// The empty string key means "all entities" since uptime isn't tracked per host.
|
|
// The value is a bitmap of host IDs that were active in this bucket.
|
|
map[string]*roaring.Bitmap{"": NewBitmap(hostIDs)})
|
|
}
|
|
|
|
// CVEDataset implements api.Dataset for host CVE tracking.
|
|
type CVEDataset struct{}
|
|
|
|
func (c *CVEDataset) Name() string { return api.MetricCVE }
|
|
func (c *CVEDataset) DefaultResolutionHours() int { return 3 }
|
|
func (c *CVEDataset) SampleStrategy() api.SampleStrategy { return api.SampleStrategySnapshot }
|
|
func (c *CVEDataset) DefaultVisualization() string { return "line" }
|
|
|
|
func (c *CVEDataset) Collect(ctx context.Context, store api.DatasetStore, now time.Time, disabledFleetIDs []uint) error {
|
|
// Collect CVEs at all severities on the curated set of tracked software and
|
|
// OS vulnerabilities. Display-time narrowing (critical-only this round,
|
|
// plus user filters) happens at read time via ResolveCVEChartEntities.
|
|
// TODO: implement bitmap compression so we can track more CVEs.
|
|
tracked, err := store.CollectibleCVEs(ctx)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
hostIDsByCVE, err := store.AffectedHostIDsByCVE(ctx, disabledFleetIDs, tracked)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
bitmaps := make(map[string]*roaring.Bitmap, len(hostIDsByCVE))
|
|
for cve, hostIDs := range hostIDsByCVE {
|
|
bitmaps[cve] = NewBitmap(hostIDs)
|
|
}
|
|
bucketStart := now.UTC().Truncate(time.Hour)
|
|
// Always call RecordBucketData, even when bitmaps is empty: snapshot
|
|
// semantics use an empty input to close any open rows for entities no
|
|
// longer in the tracked set (recordSnapshot's "absent entities" branch).
|
|
return store.RecordBucketData(ctx, c.Name(), bucketStart, time.Hour, c.SampleStrategy(), bitmaps)
|
|
}
|