Adam BaaliandClaude
6d708ef67e
## Summary Migrates CIS benchmark files from `ee/cis/` to `docs/solutions/cis/` in Fleet GitOps-compatible format, with configuration profiles, remediation scripts, and policy queries for macOS 13/14/15, Windows 10/11, and Windows 11 Intune. **The original AI-generated content (commits 1–4) contained critical errors that would cause silent deployment failures.** Commits 5–10 are a systematic review that validates every CSP path, Apple payload key, and script against official vendor documentation — fixing 30+ issues. --- ## What was wrong and why ### 1. Windows XML profiles used wrong SyncML format (would be rejected by Fleet) The AI generated full SyncML protocol envelopes (`<SyncML><SyncBody><CmdID>…<Final/>`), but Fleet expects flat `<Replace><Item>…</Item></Replace>` blocks — the same format used in [`docs/solutions/windows/configuration-profiles/`](https://github.com/fleetdm/fleet/tree/main/docs/solutions/windows/configuration-profiles). Every production Fleet profile (e.g., [`enforce device password and lock requirements – [Bundle].xml`](https://github.com/fleetdm/fleet/blob/main/docs/solutions/windows/configuration-profiles/enforce%20device%20password%20and%20lock%20requirements%20%E2%80%93%20%5BBundle%5D.xml)) uses this flat format. This is the same class of error [reported by @AdamBaali in #42748](https://github.com/fleetdm/fleet/issues/42748#issuecomment-4223794562) when testing the `numa` AI-generated security baseline. **Fix:** Rewrote all 13 XML files to match Fleet's expected format. ### 2. Six Audit CSP names were fabricated (`LogonLogoff_` prefix doesn't exist) The AI used `Audit/LogonLogoff_AuditLogon` etc., but the [Microsoft Audit Policy CSP documentation](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-audit) specifies the prefix `AccountLogonLogoff_` — e.g., `Audit/AccountLogonLogoff_AuditLogon`. Using the wrong prefix returns HTTP 500 from the MDM server. **Fix:** Corrected all 6 across 3 platform variants (18 replacements). ### 3. Three Apple Intelligence profiles used fabricated payload keys The AI invented keys that don't exist in [Apple's device-management reference](https://github.com/apple/device-management/blob/release/mdm/profiles/com.apple.applicationaccess.yaml): | Profile | AI-generated (fabricated) | Correct per Apple reference | |---------|--------------------------|----------------------------| | Extensions | `allowIntelligenceExtensions` | `allowExternalIntelligenceIntegrations` | | Mail | `com.apple.mail` / `allowMailIntelligence` | `com.apple.applicationaccess` / `allowMailSmartReplies` + `allowMailSummary` | | Notes | `com.apple.mobilenotes` / `allowNotesIntelligence` | `com.apple.applicationaccess` / `allowNotesTranscription` + `allowNotesTranscriptionSummary` | These profiles would deploy silently but enforce nothing — the keys would be ignored by macOS. The corresponding policy SQL queries also referenced the fabricated keys, so compliance checks would never pass even with the profile applied. **Fix:** Corrected keys, domains, and the corresponding policy SQL queries so compliance checks match what the profiles actually enforce. ### 4. Firewall CSP used wrong URI prefix AI used `./Device/Vendor/MSFT/Firewall/MdmStore/…` but [Microsoft's Firewall CSP documentation](https://learn.microsoft.com/en-us/windows/client-management/mdm/firewall-csp) and Fleet's own production profiles use `./Vendor/MSFT/Firewall/MdmStore/…`. **Fix:** Updated all firewall URIs across 3 platform variants. ### 5. Wrong SIDs in user rights deny policies CIS 2.2.16 (Deny access from network) and 2.2.20 (Deny RDP logon) require denying both **Guests** (`S-1-5-32-546`) and **Local account** (`S-1-5-113`). The AI only included Guests. This is the same error [identified by @AdamBaali in #42748](https://github.com/fleetdm/fleet/issues/42748#issuecomment-4223794562) where the `numa` baseline used `S-1-2-0` (wrong SID entirely). **Fix:** Added `S-1-5-113` to both XML profiles and PS1 scripts. ### 6. macOS scripts had runtime-breaking bugs Scripts copied from `ee/cis/` originals contained issues that would cause failures when deployed via Fleet: | Issue | Scripts affected | Impact | |-------|-----------------|--------| | Hardcoded `<username>` placeholder | CIS_6.1.1, CIS_6.3.6, CIS_2.11.1, CIS_2.6.2 | Shell error — `<username>` parsed as redirect | | Missing `chmod 0440` on sudoers drop-ins | CIS_5.4, CIS_5.5 | macOS sudo silently ignores files without 0440 perms | | `sudo IFS=$'\n'` doesn't set IFS in current shell | CIS_5.1.5, CIS_5.1.6, CIS_5.1.7 | Word splitting breaks on paths with spaces | | Unsafe temp files (`./tmp.txt` in CWD) | CIS_3.2, CIS_3.3, CIS_3.4 | Race condition, fails in read-only CWD | | Missing shebang | CIS_2.6.1.2 | May use wrong shell interpreter | | `not_always_working` scripts with `<password>` | CIS_2.10.1, CIS_2.10.2 | Could never work in automated deployment | **Fix:** Rewrote scripts with dynamic user enumeration, `mktemp`, proper IFS, `visudo` validation. Removed `not_always_working` scripts. ### 7. Policy YAML formatting bugs and spelling - **Computer Browser policy** (win-10, win-11): `resolution: |` was on the same line as description text, causing YAML to swallow the entire resolution into the description — no remediation steps shown. - **Set Time and Date policy** (macOS 13/14/15): Empty `description:` parsed as `null`. - 14 spelling errors across macOS and Windows YAML (`existance`, `Extention` ×8, `recomendation`, `bellow`, `enableds`, `addess` ×2, etc.) - Filename inconsistencies: `on-device-dictiation-enabled` (typo), `2.8.1.disable` (dots vs dashes) --- ## Verification methodology Every CSP OMA-URI and Apple payload key was validated against official vendor documentation: | Category | Count | Source | Result | |----------|-------|--------|--------| | [Audit CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-audit) | 27 | Microsoft docs | 6 corrected | | [UserRights CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-userrights) | 27 | Microsoft docs | All verified correct | | [LocalPoliciesSecurityOptions CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-localpoliciessecurityoptions) | 27 | Microsoft docs | 1 format corrected | | [Firewall CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/firewall-csp) | 14 | Microsoft docs | All correct (after path fix) | | [Defender CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-defender) | 9 | Microsoft docs | All verified correct | | [Apple applicationaccess](https://github.com/apple/device-management/blob/release/mdm/profiles/com.apple.applicationaccess.yaml) | 13 profiles | Apple device-management repo | 3 corrected | | Fleet GitOps YAML | 1,857 policies | [Fleet GitOps docs](https://fleetdm.com/docs/configuration/yaml-files) | 2 format bugs fixed | | Fleet XML format | 13 files | [`docs/solutions/windows/`](https://github.com/fleetdm/fleet/tree/main/docs/solutions/windows/configuration-profiles) | Reformatted to match | | Fleet blocked CSPs | — | `server/fleet/windows_mdm.go` | No conflicts | | [#43598](https://github.com/fleetdm/fleet/issues/43598) glob bug | 305 files | Filename scan | No `*`, `?`, `[`, `{` in any filename | ## Commits 1. `89d9418` — Copy CIS benchmarks to docs/solutions/cis/ in GitOps-compatible format 2. `bbabb13` — Restructure with policies/, configuration-profiles/, scripts/ subdirs 3. `00004f4` — Promote test artifacts, add Contents to READMEs 4. `786591f` — Add configuration profiles and scripts to all CIS benchmark platforms 5. `0a47a53` — Fix macOS scripts (username placeholders, sudoers perms, IFS, temp files, shebang, dictation typo, remove broken scripts) 6. `4efbd40` — Reformat all 13 Windows XML profiles to Fleet's flat Replace/Item format, fix Firewall CSP paths, fix SIDs 7. `f0abb66` — Fix 6 Audit CSP names: `LogonLogoff_` → `AccountLogonLogoff_` 8. `bfb0f95` — Fix `SmartCardRemovalBehavior` format type (`int` → `chr`) 9. `4cbb33e` — Fix 3 Apple Intelligence profiles (fabricated keys → real Apple keys), fix YAML formatting bugs 10. `d3a0031` — Fix spelling (14 corrections), naming conventions, policy-profile key alignment ## Current state | Platform | Policies | Config Profiles | Scripts | |----------|----------|-----------------|---------| | macOS 13 | 1 YAML (111 policies) | 46 .mobileconfig | 43 .sh | | macOS 14 | 1 YAML (111 policies) | 47 .mobileconfig | 43 .sh | | macOS 15 | 1 YAML (113 policies) | 43 .mobileconfig | 44 .sh | | Windows 10 | 1 YAML (510 policies) | 4 SyncML XML | 5 .ps1 | | Windows 11 | 1 YAML (555 policies) | 4 SyncML XML | 5 .ps1 | | Windows 11 Intune | 3 YAML (457 policies) | 5 SyncML XML | 2 .ps1 | The source files in `ee/cis/` are intentionally left untouched. https://claude.ai/code/session_01DUqJK6iJ8MWMdz2d25ZTNW --------- Co-authored-by: Claude <noreply@anthropic.com>
Fleet documentation
Welcome to the documentation for Fleet, the lightweight management platform for laptops and servers.
You can also read the Fleet docs over at https://fleetdm.com/docs.
Using Fleet
Resources for using the Fleet UI, fleetctl CLI, and Fleet REST API.
Deploying
Resources for installing Fleet's infrastructure dependencies, configuring Fleet, deploying osquery to hosts, and viewing example deployment scenarios.
Contributing
If you're interested in interacting with the Fleet source code, you'll find information on modifying and building the code here.
If you have any questions, please don't hesitate to File a GitHub issue or join us on Slack. You can find us in the #fleet channel.