Nico
b0dc97006c
**Related issue:** Resolves #42613 Dedupes errors that report HTTP 408 (request timeouts). As of now, I believe this only fires for timeouts on the **/api/v1/osquery/distributed/write** endpoint. This is so that we have a unique error hash with an incrementing count, instead of thousands of entries each with count: 1, which produces a huge JSON payload when passed to https://fleetdm.com/api/v1/webhooks/receive-usage-analytics for processing. Trade-off: - Before: every occurrence got its own Redis entry so thousands of near-identical examples coexisted. - After: they collapse into one entry whose :json value still contains a representative example, but we'd only keep the last IP+Port instead of all of them. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually Build a ~5 MB JSON body in a temporary file: ```bash { printf '{"node_key":"'; head -c 5000000 /dev/zero | tr '\0' 'x'; printf '"}'; } > /tmp/distwrite-body.json ``` Clear out redis: ```bash docker exec fleet-redis-1 redis-cli FLUSHDB ``` Send a dummy request and throttle the upload at 100 KB/s → ~50s to send, read timeout fires at 25s. I sent this 3 times and got the "request body read error" error back after each request. ```bash curl -sk --limit-rate 100K -X POST -H 'Content-Type: application/json' --data-binary @/tmp/distwrite-body.json https://127.0.0.1:8080/api/v1/osquery/distributed/write { "error": "request body read error: i/o timeout", "uuid": "95937f50-1008-4625-9423-bc19c7be6818" } ``` Count the error keys containing "request body read error" as the value. ```bash docker exec fleet-redis-1 sh -c 'for k in $(redis-cli --scan --pattern "error:*:json"); do v=$(redis-cli GET "$k"); echo "$v" | grep -q "request body read error" && echo "$k count=$(redis-cli GET "${k%:json}:count")"; done'\ error:{Cco_JmAdBVVVJI9k0XjNNUCmG0z1IKguMQD4VDaejfc=}:json count=3 ``` Notice the single entry and count=3 (since I ran the dummy request 3 times). Running this on main outputs three entries each with count=1. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Network error deduplication for request-timeout errors now normalizes socket addresses, preventing the usage statistics cron from failing when many similar network errors accumulate. [](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45142) <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 lines
149 B
Plaintext
2 lines
149 B
Plaintext
- Fixed the `usage_statistics` cron failing against fleetdm.com when a large number of near-identical network errors accumulated in the error store.
|