<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#46656
`server/mdm/apple/vpp.do` retried transient Apple errors by **calling
itself recursively**, with the rate-limit branch nesting `retry.Do`
inside `retry.Do`.
This change replaces the recursion with a single retry loop (respecting
the prior 1 initial attempt + 3 retries), closes each response before
retrying, honors Apple's `Retry-After` capped at 30s so that a
multi-minute value can't block a synchronous request, and threads
`context` through the VPP calls so the backoff is cancellable. The retry
timings are otherwise unchanged from before.
Following @sgress454 suggestion, I considered routing this through the
shared `retry.Do` helper (a single attempt wrapped in `retry.Do` + an
error filter) but figured out that:
- retry.Do` owns its own wait schedule and its error filter returns an
outcome enum rather than a duration, so it can't honor Apple's
per-response `Retry-After` value.
- also, I'd have to change the `retry` package to receive an extra `ctx`
param so that the backoff is context-aware (which IMHO is more blast
radius than this incident fix should carry).
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
**What was verified.** The new automated test cannot run against `main`
(the fix changes the VPP function signatures and adds the retry knobs),
so to confirm the actual failure mode I checked out `main` and ran a
small repro that drives the VPP client against an Apple endpoint that
always returns the rate-limit error. On `main`, the call **never
returns** — `do()` recurses without bound — and the repro times out:
```
--- FAIL: TestReproUnboundedRecursionOnMain (10.00s)
zz_repro_main_test.go:30: AssociateAssets did NOT return within 10s — unbounded retry recursion in do() on main
FAIL
FAIL github.com/fleetdm/fleet/v4/server/mdm/apple/vpp 10.642s
```
On this branch the same scenario returns a bounded error promptly. That
behavior is covered by the new `TestDoRetryIsBoundedAndNonRecursive`
(bounded rate-limit retries, `Retry-After` honored-but-capped, and
context cancellation), and the full `server/mdm/apple/vpp` package
passes.
**I did not perform an end-to-end QA against a live Apple endpoint**.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed a server out-of-memory crash that occurred when Apple VPP API
repeatedly returned transient errors during VPP operations, including
app installs, user registration, and license seat releases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves # Unreleased bugfix in
https://github.com/fleetdm/fleet/issues/31138
We are setting the email on users Fleet creates via the API. We decided
to remove the existing logic we were using to try and link VPP Users
back to Fleet users if they get removed from the DB but by setting the
email we can follow up(later) with a tool that can query the Apple APIs
and list all users by their emails and we can insert them into the VPP
users table
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* VPP app installation failures now report immediately without automatic
retry or recovery attempts
* Improved error transparency for Apple app provisioning failures
* **Refactor**
* Simplified VPP user management and error handling logic
* Removed redundant user lookup and retry mechanisms from app
distribution workflows
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/46382?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- **Adding vpp users table**
- **Adding ManagedAppleID to HostMDM tables to pull when installing vpp
apps to a BYOD user enrolled device**
- **Adding user create vpp apis**
- **Updating install application to support user enrolled devices**
- **Handling already installed gracefully**
- **Adding provision user logic**
- **Adding logic to associate assets on install**
- **Add license managemnt logic**
- **Adding self service ui**
- **Adding setup experience support**
- **Adding setup experience flow**
- **Fix issue with fleet's docker image in k8s environments (#44373)**
- **Changing how the appleid is captured to use idp mdm users**
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Account-based User Enrollment (iOS/iPadOS) supports VPP and in‑house
.ipa installs, including self‑service installs.
* Enrollment now applies the selected Setup experience automatically for
user‑enrolled hosts.
* Installs and license associations are scoped to account user
associations (user‑scoped installs).
* **Bug Fixes**
* Improved error messaging for license/association failures (including
per‑user device cap cases).
* Mobile UI: self‑service now shows user‑enrolled iOS/iPadOS apps.
* “Already installed” install results are treated as acknowledged
successes.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/fleetdm/fleet/pull/45202)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
Resolves#40593
This PR attempts to fix this error:
```
{"component":"http","err":"associating asset with adamID <adamId> to host <hostId>: making request to Apple VPP endpoint: making request to Apple VPP endpoint: Post \"https://vpp.itunes.apple.com/mdm/v2/assets/associate\": http: ContentLength=111 with Body length 0","host_id":<hostId>,"ip_addr":"<ip_addr>","level":"error","method":"POST","took":"20.748056032s","ts":"2026-02-25T09:53:32.10267006Z","uri":"/api/latest/fleet/device/<deviceId>/software/install/<id>","x_for_ip_addr":"<ip_addr>"}
```
Per my troubleshooting: `client.Do(req)` consumes the request body. When
retrying, the same `req` is reused but its body is not there -- so, the
retry sends `ContentLength=108` with an empty body, producing the `Body
length 0` error.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Ran the test I added without the code fix, and was able to see the exact
same error
<img width="1188" height="567" alt="Screenshot 2026-02-25 at 3 26 12 PM"
src="https://github.com/user-attachments/assets/d7bdfee7-de33-43d0-92c6-e77fa46329d6"
/>
After:
<img width="852" height="140" alt="Screenshot 2026-02-25 at 3 26 55 PM"
src="https://github.com/user-attachments/assets/e7ec3ea5-2b29-463a-9038-e5530d654a4d"
/>
Resolves#38484. This includes a CI job change to make sure we don't
introduce any more env vars that don't get proxied (and thus turned off
outside `--dev`).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
Manual QA touched hot paths, but did _not_ manually test every
FLEET_DEV_* environment variable change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Centralized dev-mode environment management for consistent FLEET_DEV_*
handling and test-friendly overrides.
* Dev-mode allows targeted overrides for certain dev-only configuration
when running with --dev.
* **Chores**
* Migrated environment access to the centralized dev-mode helper across
the codebase.
* Added CI checks to enforce proper usage of FLEET_DEV_* variables.
* **Documentation**
* Added guidance on dev-mode environment variable rules and overrides.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com>
> Related issue: #19868
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
reading the [docs][1] I realized we're missing some recommendations for
error management.
the docs also note that certain operations like assignments happen
asynchronously and you must subscribe to events to get those errors.
this part wasn't estimated nor considered.
[1]:
https://developer.apple.com/documentation/devicemanagement/app_and_book_management/handling_error_responses
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
> Related issue: #19867
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
Nico
