**Related issue:** Resolves#48697
## Summary
The hourly `UpdateQueryAggregatedStats` cron job currently walks **every
query ID** in the `queries` table and runs 5 expensive
percentile-calculation queries per query against
`scheduled_query_stats`, plus 1 INSERT/UPDATE to store results. Most
queries have no execution data at all (they are saved queries,
live-only, or de-scheduled), so this work is pure waste.
This PR changes the cron to only process queries that actually have
execution data, by querying `scheduled_query_stats` directly instead of
the `queries` table. The now-unused `walkIdsInTable` helper function is
also removed.
### How the calculations work
`CalculateAggregatedPerfStatsPercentiles` computes performance
statistics for each query that has been scheduled and executed by hosts.
For each qualifying query ID, it runs these operations against the read
replica:
1. **P50 user_time** -- Calculates the median (50th percentile) of
per-host average user-mode CPU time. The query groups
`scheduled_query_stats` rows by `host_id`, computes `SUM(user_time) /
SUM(executions)` per host, sorts them, then picks the row at position
`FLOOR(total_rows * 0.5) + 1` using a `@rownum` session variable.
2. **P95 user_time** -- Same calculation but picks the 95th percentile
row (`FLOOR(total_rows * 0.95) + 1`).
3. **P50 system_time** -- Same percentile calculation for kernel/system
CPU time.
4. **P95 system_time** -- 95th percentile of system CPU time.
5. **Total executions** -- `SELECT COALESCE(SUM(executions), 0) FROM
scheduled_query_stats WHERE scheduled_query_id = ?`
6. **INSERT/UPDATE** -- Writes the JSON result (`user_time_p50`,
`user_time_p95`, `system_time_p50`, `system_time_p95`,
`total_executions`) into the `aggregated_stats` table via `INSERT ... ON
DUPLICATE KEY UPDATE`.
### What changed
**Before:** `SELECT id FROM queries` -- walks every query (200-400+ in a
typical deployment).
**After:** `SELECT DISTINCT scheduled_query_id FROM
scheduled_query_stats WHERE executions > 0` -- walks only queries that
have actual execution data (typically 10-20).
### Benchmark results (MySQL 8.0, 300 queries seeded, only 15 with
stats)
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Avg time per cron run | 3.36s | 0.28s | **12.2x faster** |
| DB operations per run | 1,800 | 90 | **95% fewer** |
| DB operations per day | 43,200 | 2,160 | **41,040 eliminated** |
| `aggregated_stats` rows written | 300 (285 empty) | 15 (all
meaningful) | Less table bloat |
| Correctness | baseline | byte-identical JSON | **Zero regression** |
At 500+ queries the current approach **drops MySQL connections**
(`unexpected EOF` / `invalid connection`) because the cursor is held
open across thousands of heavy serial queries. The optimized version
handles any scale trivially.
### Impact analysis
Verified safe across all consumers: all query endpoints use `LEFT JOIN
aggregated_stats` (NULL-safe for missing rows), the frontend explicitly
handles null stats as "Undetermined", live query stats
(`service_campaigns.go`) call `CalculateAggregatedPerfStatsPercentiles`
directly and are unaffected, and query deletion already cleans up both
`scheduled_query_stats` and `aggregated_stats` rows.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Confirmed that the fix is not expected to adversely impact load
test results
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
