Closes#45627
Part 2 of #45553 -- see there for the full behavioral contract and
Oracle.
## Changes
- Integrated the shared `orbit/pkg/backoff` package (shipped in #45624)
into orbit's `ExecuteConfigReceivers` loop
- On error (5xx, network failure): polling interval doubles each failure
(30s, 60s, 120s, ...) capped at 5 minutes
- On success: resets immediately to normal 30s polling
- The inner `retry.Do` in `GetConfig` (transient retry within a single
tick) is unchanged
## Manual testing
### Automated tests
```
go test ./orbit/pkg/backoff/ -race -count=1 # 17 tests, 0 failures
go test ./client/ -count=1 -short # client tests pass
```
### Build verification
```
go build ./orbit/cmd/orbit/ # compiles clean
go build ./orbit/cmd/desktop/ # compiles clean
```
### Dev environment testing
Built orbit from this branch and swapped it into a local dev setup
(`/opt/orbit/bin/orbit/macos/stable/orbit`). Server-side logs confirmed
that after the restart with the new binary, `/api/fleet/orbit/config`
requests stopped arriving at the fixed 30s cadence (old behavior),
consistent with backoff engaging on error responses. The `device_token`
endpoint (not covered by this PR) continued at its normal interval,
confirming the backoff is scoped to the config polling loop only.
Full end-to-end verification of the log messages (`backing off`,
`next_retry`, `exiting backoff`) should be done by QA with `sudo tail -f
/var/log/orbit/orbit.stderr.log`.
### QA manual test plan (cc @xpkoala)
**Setup:** Local Fleet server + orbit built from this branch (see build
steps above). Orbit logs are at `/var/log/orbit/orbit.stderr.log`
(requires `sudo`).
**Test 1 -- Backoff on server failure:**
1. Start Fleet server, verify orbit connects (config requests every ~30s
in server log)
2. Stop the Fleet server (`kill` the process or `docker stop` the
container)
3. Watch orbit logs: `sudo tail -f /var/log/orbit/orbit.stderr.log`
4. **Expected:** Log lines with `"running config receivers, backing
off"` and `next_retry` values increasing: ~60s, ~120s, ~240s, then
capping at ~5m (values include up to 10% random jitter)
**Test 2 -- Recovery resets to normal:**
1. While orbit is in backoff (from Test 1), restart the Fleet server
2. Wait for the next backoff tick to fire
3. **Expected:** Log line `"config receivers succeeded, exiting
backoff"` with `backoff_duration` showing how long the backoff lasted,
then polling resumes at normal 30s
**Test 3 -- Normal operation unchanged:**
1. With both server and orbit running healthy, watch orbit logs for ~2
minutes
2. **Expected:** No backoff-related log lines. Config polling stays at
30s intervals.
---
# Checklist for submitter
- [x] Changes file added for user-visible changes in `orbit/changes/`.
- [x] Input data is properly validated, no SQL changes, no JS changes.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops (backoff caps at 5 min).
- [x] Added/updated automated tests (existing backoff package tests
cover the mechanism).
- [ ] QA'd all new/changed functionality manually.
## fleetd/orbit/Fleet Desktop
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes (backoff is
platform-agnostic).
- [ ] Verified that fleetd runs on macOS, Linux and Windows.
- [ ] Verified auto-update works from the released version of component
to the new version.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Improvements**
* Config polling now implements exponential backoff on network/server
failures, gradually increasing retry intervals up to a 5-minute maximum
instead of fixed intervals.
* After a successful config poll, the retry schedule automatically
resets back to the normal update interval.
* **Tests**
* Added unit tests to verify backoff increases after repeated failures
and resets promptly after recovery.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sharon Katz
7a419d6c3a