**Related issue:** Resolves#25574
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
---
## Summary
- Adds a new `splunk` log plugin that sends osquery logs directly to
Splunk's HTTP Event Collector (HEC) endpoint
- Eliminates the need for middleware like AWS Firehose when using Splunk
as a log destination
- Follows the same pattern as existing log destinations (Firehose, Kafka
REST, NATS, etc.)
- Includes `insecure_skip_verify` option for environments with
self-signed TLS certs
## UI changes
Follows the same pattern as the NATS log destination PR (#36527) --
adding "Splunk" to the display name, tooltip, and TypeScript type union.
No new components, pages, or styles.
### Manage automations modal -- "Log destination: Splunk"
<img width="822" height="527" alt="image"
src="https://github.com/user-attachments/assets/2533207f-fa95-4364-8ee0-3c39cd3e8e4d"
/>
### Query details page -- "Log destination: Splunk"
<img width="1905" height="662" alt="image"
src="https://github.com/user-attachments/assets/069a5005-f95c-4562-a819-fd8bdcc349f7"
/>
### Tooltip on hover
<img width="639" height="348" alt="image"
src="https://github.com/user-attachments/assets/809a47a6-b82a-4f45-b731-77b2d2c87947"
/>
### Edit query form -- "sent to your log destination: Splunk"
<img width="451" height="814" alt="image"
src="https://github.com/user-attachments/assets/b78b9a57-1f0c-4413-8b7c-654de1fd40a2"
/>
### Save new query modal -- "sent to your log destination: Splunk"
<img width="536" height="698" alt="image"
src="https://github.com/user-attachments/assets/d0a0ab01-66fe-4d63-9190-9c5e840e456d"
/>
---
### How it works
The Splunk writer (`server/logging/splunk.go`) implements the
`fleet.JSONLogger` interface. On startup it performs a health check
against the HEC `/services/collector/health` endpoint. On each `Write()`
call, it wraps each log entry in Splunk's HEC event format (adding
`time`, `index`, `source`, `sourcetype`), batches them up to 1 MB, and
POSTs to `/services/collector/event` with the `Authorization: Splunk
<token>` header. If a batch exceeds 1 MB it flushes and starts a new
one. Events over 1 MB are dropped with a log warning. Transient errors
(HTTP 503) are retried with exponential backoff (up to 8 retries).
### Configuration
```yaml
osquery:
status_log_plugin: splunk
result_log_plugin: splunk
splunk:
url: https://splunk.example.com:8088
token: <HEC token>
index: main
source: fleet
source_type: fleet:json
insecure_skip_verify: false # set true for self-signed certs
```
Or via environment variables:
```
FLEET_OSQUERY_STATUS_LOG_PLUGIN=splunk
FLEET_OSQUERY_RESULT_LOG_PLUGIN=splunk
FLEET_SPLUNK_URL=https://splunk.example.com:8088
FLEET_SPLUNK_TOKEN=<HEC token>
FLEET_SPLUNK_INDEX=main
FLEET_SPLUNK_SOURCE=fleet
FLEET_SPLUNK_SOURCE_TYPE=fleet:json
```
### Files changed
- `server/logging/splunk.go` -- Splunk HEC log writer with batching,
retry, and health check
- `server/logging/splunk_test.go` -- 9 unit tests
- `server/logging/splunk_integration_test.go` -- 3 integration tests
against real Splunk (gated by env var)
- `server/logging/logging.go` -- Added `SplunkConfig` and `case
"splunk"` to factory
- `server/config/config.go` -- Added `SplunkConfig` struct and config
flags
- `cmd/fleet/logging.go` -- Wired Splunk config into logging builder
- `server/fleet/app.go` -- Added `SplunkConfig` type for API responses
(excludes token)
- `server/service/service_appconfig.go` -- Added `case "splunk"` to
logging plugin validation
- `frontend/interfaces/config.ts` -- Added `"splunk"` to LogDestination
type
-
`frontend/components/LogDestinationIndicator/LogDestinationIndicator.tsx`
-- Added Splunk display name and tooltip
- `docs/Configuration/fleet-server-configuration.md` -- Splunk config
documentation
- `docs/Get started/FAQ.md` -- Updated plugin list
- `articles/log-destinations.md` -- Updated Splunk section with native
HEC docs
- `changes/25574-splunk-log-destination` -- Change file
## Test plan
### Unit tests (9 tests)
- [x] `TestSplunkWrite` -- sends 3 events, verifies HEC format, auth
header, index/source/sourcetype
- [x] `TestSplunkWriteEmpty` -- empty logs don't trigger HTTP request
- [x] `TestSplunkServerError` -- HEC 403 propagates as error
- [x] `TestSplunkHealthCheckFailure` -- constructor fails on bad health
- [x] `TestSplunkRecordTooBig` -- oversized events (>1MB) are dropped,
normal events still sent
- [x] `TestSplunkSplitBatchBySize` -- logs exceeding 1MB batch limit are
split into multiple requests
- [x] `TestSplunkRetryOnServiceUnavailable` -- 503 retried with backoff,
succeeds on 3rd attempt
- [x] `TestSplunkRetryExhausted` -- after 9 attempts (1 + 8 retries)
returns error
- [x] `TestSplunkMissingConfig` -- empty URL/token returns descriptive
error
### Integration tests (3 tests, gated by `SPLUNK_INTEGRATION_TEST=1`)
- [x] `TestSplunkIntegration` -- 3 events sent via writer, queried back
from Splunk REST API
- [x] `TestSplunkIntegrationBatch` -- 100 events in one Write(), all
confirmed indexed
- [x] `TestSplunkIntegrationBadToken` -- bad token Write() returns 403
### End-to-end test (macOS ARM64, real osquery agent)
1. Started Splunk Enterprise, MySQL, Redis via Docker
2. Built Fleet server from this branch with
`--osquery_status_log_plugin=splunk`
3. Set up Fleet, enrolled a real osquery 5.23.0 agent on this MacBook
4. **83 real osquery status log events indexed in Splunk** with correct
source/sourcetype/index
5. Each event contained full osquery data (`hostIdentifier`,
`host_uuid`, `calendarTime`, `severity`, `message`, `decorations`)
### Splunk showing real osquery events from Fleet
<img width="1910" height="861" alt="image"
src="https://github.com/user-attachments/assets/192490bf-d594-4424-a3e3-a18306892873"
/>
Generated with [Claude Code](https://claude.ai/code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added native Splunk HEC logging destination for status, result, and
audit logs.
* Updated the log destination UI to display **Splunk** with a dedicated
tooltip.
* Added Splunk HEC configuration (URL/token/index/source/source type)
including TLS verification control.
* **Bug Fixes**
* Improved log delivery with batching, retries for temporary HTTP
failures, and safeguards for oversized events.
* **Tests**
* Added unit tests and optional integration tests covering routing,
batching, retries, and error scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
- Adds a new FAQ entry to `docs/Get started/FAQ.md` explaining that EDR
products (e.g., SentinelOne, CrowdStrike) may occasionally flag the
fleetd agent (orbit) after updates
- Describes the osquery v5.23.0 change that performs temporary keychain
file copies to prevent corruption when querying the `certificates`
table, which can trigger EDR heuristic alerts
- Notes that Fleet is working with EDR vendors to resolve false-positive
classifications and advises customers can safely allowlist the orbit
binary
---
Built for [Mike
McNeil](https://fleetdm.slack.com/archives/C062D0THVV1/p1778015225672909?thread_ts=1778005844.853449&cid=C062D0THVV1)
by [Kilo for Slack](https://kilo.ai/slack)
---------
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Added FAQ section addressing EDR flagging of Fleet, including reasons,
safety, and allowlisting options.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves #
Updating supported version of Windows based on active Windows support:
https://endoflife.date/windows
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** #42144
We're keeping the tested version at 6 for now until we have a CI matrix
to test multiple versions. We run both 6.x and 7.x in production and if
we shipped 7+ code we'd break things.
This PR also fixes a spot I missed when mentioning MySQL version
compatibility.
Fixes#40975.
8.0.32 (was running in Aurora managed cloud at the time) -> 8.0.39 (what
we're running now) 8.0.36 -> 8.0.44 (latest 8.0.x version supported by
Aurora; holding off on 8.0.45 until Aurora supports it) 8.4.7 -> 8.4.8
9.5.0 -> 9.6.0
Also bumped the supported Aurora version from 3.07.0 to 3.08.2 to match
what we're running in managed cloud right now
Fleet might work on older patch versions but we'll no longer dev/test on
them. MySQL 9.x not testing previous minor versions matches with our
previous approach for that version.
Since these are all patch/minor bumps (and the overnight build cases are
patch bumps/are covered by AWS envs) automated testing should be
sufficient here.
**Related issue:** Resolves
[34890](https://github.com/fleetdm/fleet/issues/34890)
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
## New Fleet configuration settings
Looking at other log destinations, I couldn't find anything relevant in
GitOps. Please let me know if I missed something, however.
## fleetd/orbit/Fleet Desktop
I've tested this on both Linux and MacOS.
---------
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Co-authored-by: nulmete <nicoulmete1@gmail.com>
Redis 5 has been EOL for a few years, and didn't get updates for the
latest high-severity CVEs. We're already using 6 in most places
(fleetctl preview, recommended reference architectures, managed cloud
environments) so it's safe to set 6 as the new minimum.
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
- @noahtalerman: During #g-software design review on 2025-08-15, we
learned that Fleet builds and tests against Amazon Linux
---------
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Fixes#31693
Manually forced a run for MySQL 8.4.6 to validate.
# Checklist for submitter
- Changes not needed since this is not a product change.
## Testing
- [x] Added/updated automated tests
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [ ] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
The entry outlining Fleet's commitment to open source stewardship was
accidentally removed from the FAQ. Restoring it to its rightful place.
# Checklist for submitter
Docs only change.
- 1000 => 300
- "cost-efficient" might make it seem like it's cost prohibitive for
everyone (but the real problem is just that we can't quite yet afford to
run thousands of small instances of Fleet for folks with 10-100 hosts,
because it means increasing infrastructure engineering headcount)
