Resolves#50266.
At production numbers the table looks like this - 20,691 CVEs × 83,000
hosts, ~268M raw (cve, host) rows (software + OS joins combined):
```
┌─────────────────────────┬─────────────────────────┬───────────────────────┐
│ Shape of host IDs │ Old (map[string][]uint) │ New (roaring bitmaps) │
├─────────────────────────┼─────────────────────────┼───────────────────────┤
│ Dense (contiguous runs) │ 2,479 MB │ 4.5 MB │
├─────────────────────────┼─────────────────────────┼───────────────────────┤
│ Sparse (random) │ 2,488 MB │ 282 MB │
└─────────────────────────┴─────────────────────────┴───────────────────────┘
```
A few things worth noting about how these map to your real data:
- The old cost is shape-independent: ~2.5 GB retained just for the
result map (268M rows × 8 bytes plus append slack), and the peak during
collection is higher still because append doubling leaves garbage
behind. That's the number that was blowing up the cron.
- The new sparse figure is an overstated worst case. Your 268M rows
include duplicates — multiple vulnerable software rows per host for the
same CVE (the multi-kernel case) and overlap between the software and OS
joins. The old code retained every raw row; the bitmap dedupes on Add,
so it's bounded by unique pairs, and real fleets with AUTO_INCREMENT
host IDs sit much closer to the dense row than the sparse one.
- The new representation also has a hard ceiling the old one doesn't: a
roaring bitmap over 83k host IDs maxes out around 16 KB per CVE
regardless of contents, so even a pathological dataset caps at ~330 MB
for all 20,691 CVEs — versus the old form growing linearly with join
rows, unbounded.
TL;DR: At scale the change is roughly a 550× reduction in the realistic
(dense) case, and at minimum ~9× in the theoretical worst case.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [X] Alerted the release DRI if additional load testing is needed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Performance**
- Reduced memory usage for CVE chart data collection.
- Improved efficiency when processing large CVE and affected-host
datasets.
- **Bug Fixes**
- Preserved correct CVE filtering, duplicate-host handling,
disabled-fleet exclusions, and empty-result behavior.
- Added coverage for CVEs sourced from both software and
operating-system data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Lucas Manuel Rodriguez
001b57cb9d