23 Commits
Author SHA1 Message Date
Sharon KatzandClaude Opus 4.6 b36be84e85 Add native Splunk HEC log destination (#48455)
**Related issue:** Resolves #25574

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

---

## Summary

- Adds a new `splunk` log plugin that sends osquery logs directly to
Splunk's HTTP Event Collector (HEC) endpoint
- Eliminates the need for middleware like AWS Firehose when using Splunk
as a log destination
- Follows the same pattern as existing log destinations (Firehose, Kafka
REST, NATS, etc.)
- Includes `insecure_skip_verify` option for environments with
self-signed TLS certs

## UI changes

Follows the same pattern as the NATS log destination PR (#36527) --
adding "Splunk" to the display name, tooltip, and TypeScript type union.
No new components, pages, or styles.

### Manage automations modal -- "Log destination: Splunk"
<img width="822" height="527" alt="image"
src="https://github.com/user-attachments/assets/2533207f-fa95-4364-8ee0-3c39cd3e8e4d"
/>


### Query details page -- "Log destination: Splunk"
<img width="1905" height="662" alt="image"
src="https://github.com/user-attachments/assets/069a5005-f95c-4562-a819-fd8bdcc349f7"
/>



### Tooltip on hover
<img width="639" height="348" alt="image"
src="https://github.com/user-attachments/assets/809a47a6-b82a-4f45-b731-77b2d2c87947"
/>



### Edit query form -- "sent to your log destination: Splunk"
<img width="451" height="814" alt="image"
src="https://github.com/user-attachments/assets/b78b9a57-1f0c-4413-8b7c-654de1fd40a2"
/>



### Save new query modal -- "sent to your log destination: Splunk"
<img width="536" height="698" alt="image"
src="https://github.com/user-attachments/assets/d0a0ab01-66fe-4d63-9190-9c5e840e456d"
/>

---

### How it works

The Splunk writer (`server/logging/splunk.go`) implements the
`fleet.JSONLogger` interface. On startup it performs a health check
against the HEC `/services/collector/health` endpoint. On each `Write()`
call, it wraps each log entry in Splunk's HEC event format (adding
`time`, `index`, `source`, `sourcetype`), batches them up to 1 MB, and
POSTs to `/services/collector/event` with the `Authorization: Splunk
<token>` header. If a batch exceeds 1 MB it flushes and starts a new
one. Events over 1 MB are dropped with a log warning. Transient errors
(HTTP 503) are retried with exponential backoff (up to 8 retries).

### Configuration

```yaml
osquery:
  status_log_plugin: splunk
  result_log_plugin: splunk

splunk:
  url: https://splunk.example.com:8088
  token: <HEC token>
  index: main
  source: fleet
  source_type: fleet:json
  insecure_skip_verify: false  # set true for self-signed certs
```

Or via environment variables:
```
FLEET_OSQUERY_STATUS_LOG_PLUGIN=splunk
FLEET_OSQUERY_RESULT_LOG_PLUGIN=splunk
FLEET_SPLUNK_URL=https://splunk.example.com:8088
FLEET_SPLUNK_TOKEN=<HEC token>
FLEET_SPLUNK_INDEX=main
FLEET_SPLUNK_SOURCE=fleet
FLEET_SPLUNK_SOURCE_TYPE=fleet:json
```

### Files changed
- `server/logging/splunk.go` -- Splunk HEC log writer with batching,
retry, and health check
- `server/logging/splunk_test.go` -- 9 unit tests
- `server/logging/splunk_integration_test.go` -- 3 integration tests
against real Splunk (gated by env var)
- `server/logging/logging.go` -- Added `SplunkConfig` and `case
"splunk"` to factory
- `server/config/config.go` -- Added `SplunkConfig` struct and config
flags
- `cmd/fleet/logging.go` -- Wired Splunk config into logging builder
- `server/fleet/app.go` -- Added `SplunkConfig` type for API responses
(excludes token)
- `server/service/service_appconfig.go` -- Added `case "splunk"` to
logging plugin validation
- `frontend/interfaces/config.ts` -- Added `"splunk"` to LogDestination
type
-
`frontend/components/LogDestinationIndicator/LogDestinationIndicator.tsx`
-- Added Splunk display name and tooltip
- `docs/Configuration/fleet-server-configuration.md` -- Splunk config
documentation
- `docs/Get started/FAQ.md` -- Updated plugin list
- `articles/log-destinations.md` -- Updated Splunk section with native
HEC docs
- `changes/25574-splunk-log-destination` -- Change file

## Test plan

### Unit tests (9 tests)
- [x] `TestSplunkWrite` -- sends 3 events, verifies HEC format, auth
header, index/source/sourcetype
- [x] `TestSplunkWriteEmpty` -- empty logs don't trigger HTTP request
- [x] `TestSplunkServerError` -- HEC 403 propagates as error
- [x] `TestSplunkHealthCheckFailure` -- constructor fails on bad health
- [x] `TestSplunkRecordTooBig` -- oversized events (>1MB) are dropped,
normal events still sent
- [x] `TestSplunkSplitBatchBySize` -- logs exceeding 1MB batch limit are
split into multiple requests
- [x] `TestSplunkRetryOnServiceUnavailable` -- 503 retried with backoff,
succeeds on 3rd attempt
- [x] `TestSplunkRetryExhausted` -- after 9 attempts (1 + 8 retries)
returns error
- [x] `TestSplunkMissingConfig` -- empty URL/token returns descriptive
error

### Integration tests (3 tests, gated by `SPLUNK_INTEGRATION_TEST=1`)
- [x] `TestSplunkIntegration` -- 3 events sent via writer, queried back
from Splunk REST API
- [x] `TestSplunkIntegrationBatch` -- 100 events in one Write(), all
confirmed indexed
- [x] `TestSplunkIntegrationBadToken` -- bad token Write() returns 403

### End-to-end test (macOS ARM64, real osquery agent)

1. Started Splunk Enterprise, MySQL, Redis via Docker
2. Built Fleet server from this branch with
`--osquery_status_log_plugin=splunk`
3. Set up Fleet, enrolled a real osquery 5.23.0 agent on this MacBook
4. **83 real osquery status log events indexed in Splunk** with correct
source/sourcetype/index
5. Each event contained full osquery data (`hostIdentifier`,
`host_uuid`, `calendarTime`, `severity`, `message`, `decorations`)

### Splunk showing real osquery events from Fleet
<img width="1910" height="861" alt="image"
src="https://github.com/user-attachments/assets/192490bf-d594-4424-a3e3-a18306892873"
/>


Generated with [Claude Code](https://claude.ai/code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added native Splunk HEC logging destination for status, result, and
audit logs.
* Updated the log destination UI to display **Splunk** with a dedicated
tooltip.
* Added Splunk HEC configuration (URL/token/index/source/source type)
including TLS verification control.
* **Bug Fixes**
* Improved log delivery with batching, retries for temporary HTTP
failures, and safeguards for oversized events.
* **Tests**
* Added unit tests and optional integration tests covering routing,
batching, retries, and error scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-03 12:14:24 -04:00
Victor Lyuboslavsky 77eb458658 Migrated logging and google calendar files to use slog (#40541)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #40540 

# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
  - Changes present in previous PR

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Switched the application logging to Go's standard slog with
context-aware logging, improving structured logs and observability
across services (status, audit, result, integrations).
* Replaced legacy logging implementations and updated runtime wiring to
propagate contextual loggers for more consistent, searchable log output.

* **Tests**
  * Updated test suites to use the new slog discard/logger setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-26 12:48:54 -06:00
Victor Lyuboslavsky c14bea44de Replaced all kitlog.Logger instances with the intermediate *logging.Logger (#40425)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #40054

# Checklist for submitter

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
  - Changes included in previous PR

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Consolidated and standardized internal logging infrastructure across
the application by adopting a unified logging package throughout the
codebase, replacing previous external logging dependencies.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-24 18:52:45 -06:00
b6d19de0d9 Add support for publishing logs to NATS. (#36527)
**Related issue:** Resolves
[34890](https://github.com/fleetdm/fleet/issues/34890)

# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually

## New Fleet configuration settings
Looking at other log destinations, I couldn't find anything relevant in
GitOps. Please let me know if I missed something, however.

## fleetd/orbit/Fleet Desktop
I've tested this on both Linux and MacOS.

---------

Co-authored-by: Rachael Shaw <r@rachael.wtf>
Co-authored-by: nulmete <nicoulmete1@gmail.com>
2026-01-06 09:10:32 -03:00
Dante Catalfamo 98e92aa9b4 Webhook logging destination (#28692)
#27445
2025-05-01 16:13:04 -04:00
Roberto Dip 1cc13a09fb 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774)
`go-kit/kit/log` was deprecated and generating warnings

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-17 10:27:31 -03:00
Benjamin Edwards 677835e2c3 support external id in STS Assume Role calls (#18901) 2024-05-21 13:19:35 -04:00
Benjamin Edwards f3b0e4da72 add configuration parameters for filesystem logging file rotation (#10048) 2023-02-24 07:44:56 -05:00
Lucas Manuel Rodriguez e1bbcfcfda Generate audit logs for activities (#9001)
* Generate audit logs for activities

* Fix config tests

* Fix TestGetConfig/IncludeServerConfig

* Fix use of AddAttributes in results only

* Stream activities asynchronously

* Fix index and add logging

* Revert change

* Documentation fixes
2022-12-23 19:04:13 -03:00
Martin Angers 421a0ec635 Fix some config mismatches in osquery logging (#5488) 2022-05-10 11:27:32 -04:00
Benjamin Edwards f8cf6ea91c make context type value header configurable (#4441)
* make context type value header configurable
* populate config
2022-03-09 17:22:29 -05:00
Martin Angers 69a4985cac Use new error handling approach in other packages (#2954) 2021-11-22 09:13:26 -05:00
dsbaha 51e35e1ba0 Implementation of a Kafka REST Proxy logging plugin (#2534)
This PR implements the status/result logger functions necessary interface with a Kafka REST Proxy service.  

Specifically, this is compatible with the [Confluent KAFKA Rest Proxy Service ](https://docs.confluent.io/1.0/kafka-rest/docs/intro.html).
2021-10-27 21:51:17 -07:00
Benjamin Edwards c18214be37 add logging settings to config api response (#1467)
- add docker-compose file for locally testing aws dependencies
- update firehose & kinesis configs to optionally supply endpoint url override
- serialize `logging` field in appconfig api response
2021-07-30 11:45:49 -04:00
Zach Wasserman c5280c0517 Add v4 suffix in go.mod (#1224) 2021-06-25 21:46:51 -07:00
Zach Wasserman fb32f0cf40 Remove kolide types and packages from backend (#974)
Generally renamed `kolide` -> `fleet`
2021-06-06 15:07:29 -07:00
Michael Samuel fb45806088 Copy log fields into GCP PubSub attributes (#712)
Add a config setting to allow copying message fields and decorations into Google Pub/Sub attributes, making it possible to use these values for subscription filters.
2021-05-08 12:29:52 -07:00
Zach Wasserman de0b3324b1 Add AWS Lambda as logging plugin (#347)
This plugin invokes the provided function with each log line as the
payload.

Closes #342
2021-02-24 10:02:26 -08:00
Mike Arpaia af96e52a00 Update the Go import paths to new repo name (#27) 2020-11-11 09:59:12 -08:00
James Alseth 3a63dac4a3 Add compression option for filesystem logs when they're rotated (#2292) 2020-09-09 13:33:32 -07:00
billcobblerandBrendan Shaklovitz 20328b0f87 Add stdout and kinesis logger plugins and sts assume role to Firehose (#2282)
Co-authored-by: Brendan Shaklovitz <nyanshak@users.noreply.github.com>
2020-08-19 14:56:44 -07:00
Michael Samuel 969d5f25af Add Google Cloud PubSub logging (#2049)
Adds Google Cloud PubSub logging for status and results.

This also changes the Write interface for logging modules to add a context.Context (only used by pubsub currently).
2019-07-16 15:41:50 -07:00
Zachary Wasserman e59714242e Add Firehose logging capabilities for result and status logs (#2022)
- Refactor configuration for logging to use separate plugins
- Move existing filesystem logging to filesystem plugin
- Create new AWS firehose plugin
- Update documentation around logging
2019-04-08 11:47:15 -07:00