Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

Restructure documentation in preparation for docs living on fleetdm.com (#1143)

Browse Source 
* Remove Orbit top level folder and rename folders to Deploying and Contributing

* Update link in doc README
This commit is contained in:
noahtalerman
2021-06-18 15:58:21 -05:00
committed by GitHub
parent 47b423ee29
commit 17a6c21224
34 changed files with 126 additions and 285 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/1-Using-Fleet/1-Fleet-UI.md
+1 -1
View File
@@ -44,5 +44,5 @@ To add queries to a pack, use the right-hand sidebar. You can take an existing s
![Schedule Query Sidebar](https://raw.githubusercontent.com/fleetdm/fleet/main/docs/images/schedule-query-sidebar.png)
Once you've scheduled queries and curated your packs, you can read our guide to [Working With Osquery Logs](../1-Using-Fleet/4-Osquery-logs.md).
Once you've scheduled queries and curated your packs, you can read our guide to [Working With Osquery Logs](../1-Using-Fleet/5-Osquery-logs.md).
docs/1-Using-Fleet/2-fleetctl-CLI.md
+3 -3
View File
@@ -34,7 +34,7 @@ This guide illustrates:
### Running Fleet
For the sake of this tutorial, I will be using the local development Docker Compose infrastructure to run Fleet locally. This is documented in some detail in the [developer documentation](../4-Contribution/1-Building-Fleet.md#development-infrastructure), but the following are the minimal set of commands that you can run from the root of the repository (assuming that you have a working Go/JavaScript toolchain installed along with Docker Compose):
For the sake of this tutorial, I will be using the local development Docker Compose infrastructure to run Fleet locally. This is documented in some detail in the [developer documentation](../3-Contributing/1-Building-Fleet.md#development-infrastructure), but the following are the minimal set of commands that you can run from the root of the repository (assuming that you have a working Go/JavaScript toolchain installed along with Docker Compose):
```
docker-compose up -d
@@ -80,7 +80,7 @@ It's possible to specify the password via the `--password` flag or the `$PASSWOR
### Connecting a host
For the sake of this tutorial, I'm going to be using Kolide's osquery launcher to start osquery locally and connect it to Fleet. To learn more about connecting osquery to Fleet, see the [Adding Hosts to Fleet](../3-Deployment/3-Adding-hosts.md) documentation.
For the sake of this tutorial, I'm going to be using Kolide's osquery launcher to start osquery locally and connect it to Fleet. To learn more about connecting osquery to Fleet, see the [Adding Hosts to Fleet](./4-Adding-hosts.md) documentation.
To get your osquery enroll secret, run the following:
@@ -556,7 +556,7 @@ spec:
Fleet supports osquery's file carving functionality as of Fleet 3.3.0. This allows the Fleet server to request files (and sets of files) from osquery agents, returning the full contents to Fleet.
File carving data can be either stored in Fleet's database or to an external S3 bucket. For information on how to configure the latter, consult the [configuration docs](https://github.com/fleetdm/fleet/blob/master/docs/3-Deployment/2-Configuration.md#s3-file-carving-backend).
File carving data can be either stored in Fleet's database or to an external S3 bucket. For information on how to configure the latter, consult the [configuration docs](../2-Deploying/2-Configuration.md#s3-file-carving-backend).
### Configuration
docs/1-Using-Fleet/3-REST-API.md
+1 -1
View File
@@ -572,7 +572,7 @@ None.
Returns the information of the specified host.
The endpoint returns the host's installed `software` if the software inventory feature flag is turned on. This feature flag is turned off by default. [Check out the feature flag documentation](../3-Deployment/2-Configuration.md#feature-flags) for instructions on how to turn on the software inventory feature.
The endpoint returns the host's installed `software` if the software inventory feature flag is turned on. This feature flag is turned off by default. [Check out the feature flag documentation](../2-Deploying/2-Configuration.md#feature-flags) for instructions on how to turn on the software inventory feature.
`GET /api/v1/fleet/hosts/{id}`
docs/3-Deployment/3-Adding-hosts.md → docs/1-Using-Fleet/4-Adding-hosts.md
+78 -72
View File
@@ -1,24 +1,93 @@
# Adding hosts
- [Kolide osquery Launcher](#kolide-osquery-launcher)
- [Connecting a single Launcher to Fleet](#connecting-a-single-launcher-to-fleet)
- [Generating packages](#generating-packages)
- [Orbit for osquery](#orbit-for-osquery)
- [Native osquery TLS plugins](#native-osquery-tls-plugins)
- [Set an environment variable with an agent enrollment secret](#set-an-environment-variable-with-an-agent-enrollment-secret)
- [Deploy the TLS certificate that osquery will use to communicate with Fleet](#deploy-the-tls-certificate-that-osquery-will-use-to-communicate-with-fleet)
- [Launching osqueryd](#launching-osqueryd)
- [Using a flag file to manage flags](#using-a-flag-file-to-manage-flags)
- [Kolide osquery Launcher](#kolide-osquery-launcher)
- [Connecting a single Launcher to Fleet](#connecting-a-single-launcher-to-fleet)
- [Generating packages](#generating-packages)
- [Enrolling multiple macOS hosts](#enrolling-multiple-macos-hosts)
- [Multiple enroll secrets](#multiple-enroll-secrets)
Fleet is powered by the open source osquery tool. To connect a host to Fleet, you have two general options. You can install the osquery binaries on your hosts via the packages distributed at https://osquery.io/downloads or you can use the [Kolide Osquery Launcher](https://github.com/kolide/launcher). The Launcher is a light wrapper that aims to make running and deploying osquery easier by adding a few features and minimizing the configuration interface. Some features of The Launcher are:
Fleet is powered by the open source osquery tool. To connect a host to Fleet, you have three general options:
- You can use [Orbit for osquery](https://github.com/fleet/orbit)
- You can install the osquery binaries on your hosts via the packages distributed at https://osquery.io/downloads
- You can use the [Kolide Osquery Launcher](https://github.com/kolide/launcher).
- Secure autoupdates to the latest stable osqueryd
- Remote communication via a strongly-typed, versioned, modern gRPC server API
- a curated `kolide_best_practices` table which includes a curated set of standards for the modern enterprise
## Orbit for osquery
The Launcher also contains robust tooling to help you generate packages for your environment that are designed to work together with Fleet. For specific documentation on using Launcher with Fleet, see the section below called "Kolide Osquery Launcher".
Orbit is an [osquery](https://github.com/osquery/osquery) runtime and autoupdater. With Orbit, it's easy to deploy osquery, manage configurations, and stay up to date. Orbit eases the deployment of osquery connected with a [Fleet server](https://github.com/fleetdm/fleet), and is a (near) drop-in replacement for osquery in a variety of deployment scenarios.
If you'd like to use the native osqueryd binaries to connect to Fleet, this is enabled by using osquery's TLS API plugins that are principally documented on the official osquery wiki: http://osquery.readthedocs.io/en/stable/deployment/remote/. These plugins are very customizable and thus have a large configuration surface. Configuring osqueryd to communicate with Fleet is documented below in the "Native Osquery TLS Plugins" section.
Orbit is the recommended agent for Fleet. But Orbit can be used with or without Fleet, and Fleet can be used with or without Orbit.
Check out the [Orbit Github repository](https://github.com/fleetdm/fleet) for information on using and packaging Orbit for osquery.
## Native osquery TLS plugins
> If you'd like to use the native osqueryd binaries to connect to Fleet, this is enabled by using osquery's TLS API plugins that are principally documented on the official osquery wiki: http://osquery.readthedocs.io/en/stable/deployment/remote/. These plugins are very customizable and thus have a large configuration surface. Configuring osqueryd to communicate with Fleet is documented below in the "Native Osquery TLS Plugins" section.
You can find various ways to install osquery on a variety of platforms at https://osquery.io/downloads. Once you have installed osquery, you need to do two things:
### Set an environment variable with an agent enrollment secret
The enrollment secret is a value that osquery provides to authenticate with Fleet. There are a few ways you can set the enrollment secret on the hosts which you control. You can either set the value as:
- an value of an environment variable (a common name is `OSQUERY_ENROLL_SECRET`)
- the content of a local file (a common path is `/etc/osquery/enrollment_secret`)
The value of the environment variable or content of the file should be a secret shared between the osqueryd client and the Fleet server. This is basically osqueryd's passphrase which it uses to authenticate with Fleet, convincing Fleet that it is actually one of your hosts. The passphrase could be whatever you'd like, but it would be prudent to have the passphrase long, complex, mixed-case, etc. When you launch the Fleet server, you should specify this same value.
If you use an environment variable for this, you can specify it with the `--enroll_secret_env` flag when you launch osqueryd. If you use a local file for this, you can specify it's path with the `--enroll_secret_path` flag.
To retrieve the enroll secret, use the "Add New Host" dialog in the Fleet UI or
`fleetctl get enroll_secret`).
If your organization has a robust internal public key infrastructure (PKI) and you already deploy TLS client certificates to each host to uniquely identify them, then osquery supports an advanced authentication mechanism which takes advantage of this. Fleet can be fronted with a proxy that will perform the TLS client authentication.
### Deploy the TLS certificate that osquery will use to communicate with Fleet
When Fleet uses a self-signed certificate, osquery agents will need a copy of that certificate in order to authenticate the Fleet server. If clients connect directly to the Fleet server, you can download the certificate through the Fleet UI. From the main dashboard (`/hosts/manage`), click "Add New Host" and "Fetch Certificate". If Fleet is running behind a load-balancer that terminates TLS, you will have to talk to your system administrator about where to find this certificate.
It is important that the CN of this certificate matches the hostname or IP that osqueryd clients will use to connect.
Specify the path to this certificate with the `--tls_server_certs` flag when you launch osqueryd.
## Launching osqueryd
Assuming that you are deploying your enrollment secret in the file `/etc/osquery/enroll_secret` and your osquery server certificate is at `/etc/osquery/fleet.crt`, you could copy and paste the following command with the following flags (be sure to replace `fleet.acme.net` with the hostname or IP of your Fleet installation):
```
sudo osqueryd \
--enroll_secret_path=/etc/osquery/enroll_secret \
--tls_server_certs=/etc/osquery/fleet.crt \
--tls_hostname=fleet.acme.net \
--host_identifier=instance \
--enroll_tls_endpoint=/api/v1/osquery/enroll \
--config_plugin=tls \
--config_tls_endpoint=/api/v1/osquery/config \
--config_refresh=10 \
--disable_distributed=false \
--distributed_plugin=tls \
--distributed_interval=10 \
--distributed_tls_max_attempts=3 \
--distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \
--distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \
--logger_plugin=tls \
--logger_tls_endpoint=/api/v1/osquery/log \
--logger_tls_period=10
```
If your osquery server certificate is deployed to a path that is not `/etc/osquery/fleet.crt`, be sure to update the `--tls_server_certs` flag. Similarly, if your enrollment secret is in an environment variable that is not called `OSQUERY_ENROLL_SECRET`, then be sure to update the `--enroll_secret_env` environment variable. If your enroll secret is defined in a local file, specify the file's path with the `--enroll_secret_path` flag instead of using the `--enroll_secret_env` flag.
### Using a flag file to manage flags
For your convenience, osqueryd supports putting all of your flags into a single file. We suggest deploying this file to `/etc/osquery/fleet.flags`. If you've deployed the appropriate osquery flags to that path, you could simply launch osquery via:
```
osqueryd --flagfile=/etc/osquery/fleet.flags
```
## Kolide osquery Launcher
@@ -83,69 +152,6 @@ You can also add the `--mac_package_signing_key` flag to define the name of the
If you want to generate a package for local testing, you can call `package-builder make` with the `--insecure` flag as well and the auto-run command in the resultant packages will include `--insecure` as well.
## Native osquery TLS plugins
You can find various ways to install osquery on a variety of platforms at https://osquery.io/downloads. Once you have installed osquery, you need to do two things:
### Set an environment variable with an agent enrollment secret
The enrollment secret is a value that osquery provides to authenticate with Fleet. There are a few ways you can set the enrollment secret on the hosts which you control. You can either set the value as:
- an value of an environment variable (a common name is `OSQUERY_ENROLL_SECRET`)
- the content of a local file (a common path is `/etc/osquery/enrollment_secret`)
The value of the environment variable or content of the file should be a secret shared between the osqueryd client and the Fleet server. This is basically osqueryd's passphrase which it uses to authenticate with Fleet, convincing Fleet that it is actually one of your hosts. The passphrase could be whatever you'd like, but it would be prudent to have the passphrase long, complex, mixed-case, etc. When you launch the Fleet server, you should specify this same value.
If you use an environment variable for this, you can specify it with the `--enroll_secret_env` flag when you launch osqueryd. If you use a local file for this, you can specify it's path with the `--enroll_secret_path` flag.
To retrieve the enroll secret, use the "Add New Host" dialog in the Fleet UI or
`fleetctl get enroll_secret`).
If your organization has a robust internal public key infrastructure (PKI) and you already deploy TLS client certificates to each host to uniquely identify them, then osquery supports an advanced authentication mechanism which takes advantage of this. Fleet can be fronted with a proxy that will perform the TLS client authentication.
### Deploy the TLS certificate that osquery will use to communicate with Fleet
When Fleet uses a self-signed certificate, osquery agents will need a copy of that certificate in order to authenticate the Fleet server. If clients connect directly to the Fleet server, you can download the certificate through the Fleet UI. From the main dashboard (`/hosts/manage`), click "Add New Host" and "Fetch Certificate". If Fleet is running behind a load-balancer that terminates TLS, you will have to talk to your system administrator about where to find this certificate.
It is important that the CN of this certificate matches the hostname or IP that osqueryd clients will use to connect.
Specify the path to this certificate with the `--tls_server_certs` flag when you launch osqueryd.
## Launching osqueryd
Assuming that you are deploying your enrollment secret in the file `/etc/osquery/enroll_secret` and your osquery server certificate is at `/etc/osquery/fleet.crt`, you could copy and paste the following command with the following flags (be sure to replace `fleet.acme.net` with the hostname or IP of your Fleet installation):
```
sudo osqueryd \
--enroll_secret_path=/etc/osquery/enroll_secret \
--tls_server_certs=/etc/osquery/fleet.crt \
--tls_hostname=fleet.acme.net \
--host_identifier=instance \
--enroll_tls_endpoint=/api/v1/osquery/enroll \
--config_plugin=tls \
--config_tls_endpoint=/api/v1/osquery/config \
--config_refresh=10 \
--disable_distributed=false \
--distributed_plugin=tls \
--distributed_interval=10 \
--distributed_tls_max_attempts=3 \
--distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \
--distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \
--logger_plugin=tls \
--logger_tls_endpoint=/api/v1/osquery/log \
--logger_tls_period=10
```
If your osquery server certificate is deployed to a path that is not `/etc/osquery/fleet.crt`, be sure to update the `--tls_server_certs` flag. Similarly, if your enrollment secret is in an environment variable that is not called `OSQUERY_ENROLL_SECRET`, then be sure to update the `--enroll_secret_env` environment variable. If your enroll secret is defined in a local file, specify the file's path with the `--enroll_secret_path` flag instead of using the `--enroll_secret_env` flag.
### Using a flag file to manage flags
For your convenience, osqueryd supports putting all of your flags into a single file. We suggest deploying this file to `/etc/osquery/fleet.flags`. If you've deployed the appropriate osquery flags to that path, you could simply launch osquery via:
```
osqueryd --flagfile=/etc/osquery/fleet.flags
```
## Enrolling multiple macOS hosts
If you're managing an enterprise environment with multiple Mac devices, you likely have an enterprise deployment tool like [Munki](https://www.munki.org/munki/) or [Jamf Pro](https://www.jamf.com/products/jamf-pro/) to deliver software to your mac fleet. You can deploy osqueryd and enroll all your macs into Fleet using your software management tool of choice.
docs/1-Using-Fleet/4-Osquery-logs.md → docs/1-Using-Fleet/5-Osquery-logs.md
+7 -7
View File
@@ -22,21 +22,21 @@ Fleet supports the following logging plugins for osquery logs:
- [PubSub](#pubsub) - Logs are written to Google Cloud PubSub topics.
- [Stdout](#stdout) - Logs are written to stdout.
To set the osquery logging plugins, use the `--osquery_result_log_plugin` and `--osquery_status_log_plugin` flags (or [equivalents for environment variables or configuration files](../3-Deployment/2-Configuration.md#options)).
To set the osquery logging plugins, use the `--osquery_result_log_plugin` and `--osquery_status_log_plugin` flags (or [equivalents for environment variables or configuration files](../2-Deploying/2-Configuration.md#options)).
### Filesystem
The default logging plugin.
- Plugin name: `filesystem`
- Flag namespace: [filesystem](../3-Deployment/2-Configuration.md#filesystem)
- Flag namespace: [filesystem](../2-Deploying/2-Configuration.md#filesystem)
With the filesystem plugin, osquery result and/or status logs are written to the local filesystem on the Fleet server. This is typically used with a log forwarding agent on the Fleet server that will push the logs into a logging pipeline. Note that if multiple load-balanced Fleet servers are used, the logs will be load-balanced across those servers (not duplicated).
### Firehose
- Plugin name: `firehose`
- Flag namespace: [firehose](../3-Deployment/2-Configuration.md#firehose)
- Flag namespace: [firehose](../2-Deploying/2-Configuration.md#firehose)
With the Firehose plugin, osquery result and/or status logs are written to [AWS Firehose](https://aws.amazon.com/kinesis/data-firehose/) streams. This is a very good method for aggregating osquery logs into AWS S3 storage.
@@ -45,7 +45,7 @@ Note that Firehose logging has limits [discussed in the documentation](https://d
### Kinesis
- Plugin name: `kinesis`
- Flag namespace: [kinesis](../3-Deployment/2-Configuration.md#kinesis)
- Flag namespace: [kinesis](../2-Deploying/2-Configuration.md#kinesis)
With the Kinesis plugin, osquery result and/or status logs are written to
[AWS Kinesis](https://aws.amazon.com/kinesis/data-streams) streams.
@@ -58,7 +58,7 @@ output in the Fleet logs and those logs _will not_ be sent to Kinesis.
### Lambda
- Plugin name: `lambda`
- Flag namespace: [lambda](../3-Deployment/2-Configuration.md#lambda)
- Flag namespace: [lambda](../2-Deploying/2-Configuration.md#lambda)
With the Lambda plugin, osquery result and/or status logs are written to
[AWS Lambda](https://aws.amazon.com/lambda/) functions.
@@ -73,7 +73,7 @@ output in the Fleet logs and those logs _will not_ be sent to Lambda.
### PubSub
- Plugin name: `pubsub`
- Flag namespace: [pubsub](../3-Deployment/2-Configuration.md#pubsub)
- Flag namespace: [pubsub](../2-Deploying/2-Configuration.md#pubsub)
With the PubSub plugin, osquery result and/or status logs are written to [PubSub](https://cloud.google.com/pubsub/) topics.
@@ -82,7 +82,7 @@ Note that messages over 10MB will be dropped, with a notification sent to the fl
### Stdout
- Plugin name: `stdout`
- Flag namespace: [stdout](../3-Deployment/2-Configuration.md#stdout)
- Flag namespace: [stdout](../2-Deploying/2-Configuration.md#stdout)
With the stdout plugin, osquery result and/or status logs are written to stdout
on the Fleet server. This is typically used for debugging or with a log
docs/1-Using-Fleet/5-Monitoring-Fleet.md → docs/1-Using-Fleet/6-Monitoring-Fleet.md
+1 -5
View File
@@ -54,11 +54,7 @@ Scaling Fleet horizontally is as simple as running more Fleet server processes c
The Fleet/osquery system is resilient to loss of availability. Osquery agents will continue executing the existing configuration and buffering result logs during downtime due to lack of network connectivity, server maintenance, or any other reason. Buffering in osquery can be configured with the `--buffered_log_max` flag.
Note that short downtimes are expected during [Fleet server upgrades](./7-Updating-Fleet.md)-fleet.md) that require database migrations.
### Monitoring
More information on monitoring Fleet servers with Prometheus and other tools is available in the [Monitoring Fleet](./5-Monitoring-Fleet.md) documentation.
Note that short downtimes are expected during [Fleet server upgrades](./8-Updating-Fleet.md)-fleet.md) that require database migrations.
### Debugging performance issues
docs/1-Using-Fleet/6-Security-best-practices.md → docs/1-Using-Fleet/7-Security-best-practices.md
+1 -1
View File
@@ -27,7 +27,7 @@ Fleet supports SAML auth which means that it can be configured such that it neve
Passwords are never stored in plaintext in the database. We store a `bcrypt`ed hash of the password along with a randomly generated salt. The `bcrypt` iteration count and salt key size are admin-configurable.
### Authentication tokens
The size and expiration time of session tokens is admin-configurable. See [https://github.com/fleetdm/fleet/blob/master/docs/3-Deployment/2-Configuration.md#session_duration](../3-Deployment/2-Configuration.md#session_duration).
The size and expiration time of session tokens is admin-configurable. See [The documentation on session duration](../2-Deploying/2-Configuration.md#session_duration).
It is possible to revoke all session tokens for a user by forcing a password reset.
docs/1-Using-Fleet/7-Updating-Fleet.md → docs/1-Using-Fleet/8-Updating-Fleet.md
+1 -1
View File
@@ -7,7 +7,7 @@
## Overview
This guide explains how to update and run new versions of Fleet. For initial installation instructions, see [Installing Fleet](../3-Deployment/1-Installation.md).
This guide explains how to update and run new versions of Fleet. For initial installation instructions, see [Installing Fleet](../2-Deploying/1-Installation.md).
There are two steps to perform a typical Fleet update. If any other steps are required, they will be noted in the release notes.
docs/1-Using-Fleet/FAQ.md
+8 -8
View File
@@ -21,7 +21,7 @@ The upgrade from kolide/fleet to fleetdm/fleet works the same as any minor versi
Minor version upgrades in Kolide Fleet often included database migrations and the recommendation to back up the database before migrating. The same goes for FleetDM Fleet versions.
To migrate from Kolide Fleet to FleetDM Fleet, please follow the steps outlined in the [Updating Fleet section](./7-Updating-Fleet.md) of the documentation.
To migrate from Kolide Fleet to FleetDM Fleet, please follow the steps outlined in the [Updating Fleet section](./8-Updating-Fleet.md) of the documentation.
## Has anyone stress tested Fleet? How many clients can the Fleet server handle?
@@ -39,7 +39,7 @@ In the coming months, Fleet will introduce the [Teams feature](https://github.co
## How often do labels refresh? Is the refresh frequency configurable?
The update frequency for labels is configurable with the [—osquery_label_update_interval](https://github.com/fleetdm/fleet/blob/master/docs/3-Deployment/2-Configuration.md#osquery_label_update_interval) flag (default 1 hour).
The update frequency for labels is configurable with the [—osquery_label_update_interval](../2-Deploying/2-Configuration.md#osquery_label_update_interval) flag (default 1 hour).
## How do I revoke the authorization tokens for a user?
@@ -51,7 +51,7 @@ Fleet can live query the `osquery_schedule` table. Performing this live query al
## How do I monitor a Fleet server?
Fleet provides standard interfaces for monitoring and alerting. See the [Monitoring Fleet](./5-Monitoring-Fleet.md) documentation for details.
Fleet provides standard interfaces for monitoring and alerting. See the [Monitoring Fleet](./6-Monitoring-Fleet.md) documentation for details.
## Why is the “Add User” button disabled?
@@ -76,7 +76,7 @@ Live query results (executed in the web UI or `fleetctl query`) are pushed direc
### Scheduled Queries
Scheduled query results (queries that are scheduled to run in Packs) are typically sent to the Fleet server, and will be available on the filesystem of the server at the path configurable by [`--osquery_result_log_file`](../3-Deployment/2-Configuration.md#osquery_result_log_file). This defaults to `/tmp/osquery_result`.
Scheduled query results (queries that are scheduled to run in Packs) are typically sent to the Fleet server, and will be available on the filesystem of the server at the path configurable by [`--osquery_result_log_file`](../2-Deploying/2-Configuration.md#osquery_result_log_file). This defaults to `/tmp/osquery_result`.
It is possible to configure osqueryd to log query results outside of Fleet. For results to go to Fleet, the `--logger_plugin` flag must be set to `tls`.
@@ -84,7 +84,7 @@ It is possible to configure osqueryd to log query results outside of Fleet. For
Folks typically use Fleet to ship logs to data aggregation systems like Splunk, the ELK stack, and Graylog.
The [logger configuration options](https://github.com/fleetdm/fleet/blob/master/docs/3-Deployment/2-Configuration.md#osquery_status_log_plugin) allow you to select the log output plugin. Using the log outputs you can route the logs to your chosen aggregation system.
The [logger configuration options](../2-Deploying/2-Configuration.md#osquery_status_log_plugin) allow you to select the log output plugin. Using the log outputs you can route the logs to your chosen aggregation system.
### Troubleshooting
@@ -94,7 +94,7 @@ Expecting results, but not seeing anything in the logs?
- Check whether the query is scheduled in differential mode. If so, new results will only be logged when the result set changes.
- Ensure that the query is scheduled to run on the intended platforms, and that the tables queried are supported by those platforms.
- Use live query to `SELECT * FROM osquery_schedule` to check whether the query has been scheduled on the host.
- Look at the status logs provided by osquery. In a standard configuration these are available on the filesystem of the Fleet server at the path configurable by [`--filesystem_status_log_file`](../3-Deployment/2-Configuration.md#filesystem_status_log_file). This defaults to `/tmp/osquery_status`. The host will output a status log each time it executes the query.
- Look at the status logs provided by osquery. In a standard configuration these are available on the filesystem of the Fleet server at the path configurable by [`--filesystem_status_log_file`](../2-Deploying/2-Configuration.md#filesystem_status_log_file). This defaults to `/tmp/osquery_status`. The host will output a status log each time it executes the query.
## Why aren’t my live queries being logged?
@@ -110,7 +110,7 @@ As an example, let's say you want to retrieve a host's OS version, installed sof
Each host’s OS version is available using the `api/v1/fleet/hosts` API endpoint. [Check out the API documentation for this endpoint](./3-REST-API.md#list-hosts).
The ability to view each host’s installed software was released behind a feature flag in Fleet 3.11.0 and called Software inventory. [Check out the feature flag documentation for instructions on turning on Software inventory in Fleet](../3-Deployment/2-Configuration.md#feature-flags).
The ability to view each host’s installed software was released behind a feature flag in Fleet 3.11.0 and called Software inventory. [Check out the feature flag documentation for instructions on turning on Software inventory in Fleet](../2-Deploying/2-Configuration.md#feature-flags).
Once the Software inventory feature is turned on, a list of a specific host’s installed software is available using the `api/v1/fleet/hosts/{id}` endpoint. [Check out the documentation for this endpoint](./3-REST-API.md#get-host).
@@ -128,4 +128,4 @@ You can also do this by setting the `targets` field in the [YAML configuration f
The `unknown column` error typically occurs when the database migrations haven't been run during the upgrade process.
Check out the [documentation on running database migrations](https://github.com/fleetdm/fleet/blob/edf1c47298e9355c0a8c29a429db41bcf273cddf/docs/1-Using-Fleet/7-Updating-Fleet.md#running-database-migrations) to resolve this issue.
Check out the [documentation on running database migrations](./8-Updating-Fleet.md#running-database-migrations) to resolve this issue.
docs/1-Using-Fleet/README.md
+7 -4
View File
@@ -9,16 +9,19 @@ Includes resources for setting up and configuring Fleet via the fleetctl CLI
### [REST API](./3-REST-API.md)
Provides resources for working with Fleet's API and includes example code for endpoints
### [Osquery logs](./4-Osquery-logs.md)
### [Adding hosts](./4-Adding-hosts.md)
Provides resources for enrolling your hosts to Fleet
### [Osquery logs](./5-Osquery-logs.md)
Includes documentation on the plugin options for working with osquery logs
### [Monitoring Fleet](./5-Monitoring-Fleet.md)
### [Monitoring Fleet](./6-Monitoring-Fleet.md)
Provides documentation for load balancer health checks and working with Fleet server metrics and performance
### [Security best practices](./6-Security-best-practices.md)
### [Security best practices](./7-Security-best-practices.md)
Includes resources for ways to mitigate against the OWASP top 10 issues
### [Updating Fleet](./7-Updating-Fleet.md)
### [Updating Fleet](./8-Updating-Fleet.md)
Includes a guide for how to update and run new versions of Fleet
### [FAQ](./FAQ.md)
docs/1-Using-Fleet/standard-query-library/README.md
+1 -1
View File
@@ -29,7 +29,7 @@ spec:
```
2. Replace each field and submit a pull request to the fleetdm/fleet GitHub repository.
For instructions on submitting pull requests to Fleet check out [the Committing Changes section](https://github.com/fleetdm/fleet/blob/58445ede82550cb574775a83ae4cf5433f325a7e/docs/4-Contribution/4-Committing-Changes.md#committing-changes) in the Contributors documentation.
For instructions on submitting pull requests to Fleet check out [the Committing Changes section](../../3-Contributing/4-Committing-Changes.md#committing-changes) in the Contributors documentation.
## Additional resources
docs/3-Deployment/1-Installation.md → docs/2-Deploying/1-Installation.md
View File
docs/3-Deployment/2-Configuration.md → docs/2-Deploying/2-Configuration.md
View File
docs/3-Deployment/4-Example-deployment-scenarios.md → docs/2-Deploying/3-Example-deployment-scenarios.md
+2 -2
View File
@@ -186,7 +186,7 @@ See [Running with systemd](./2-Configuration.md#running-with-systemd) for docume
### Installing and running osquery
> Note that this whole process is outlined in more detail in the [Adding Hosts To Fleet](./3-Adding-hosts.md) document. The steps are repeated here for the sake of a continuous tutorial.
> Note that this whole process is outlined in more detail in the [Adding Hosts To Fleet](../1-Using-Fleet/4-Adding-hosts.md) document. The steps are repeated here for the sake of a continuous tutorial.
To install osquery on CentOS, you can run the following:
@@ -355,7 +355,7 @@ See [Running with systemd](./2-Configuration.md#running-with-systemd) for docume
### Installing and running osquery
> Note that this whole process is outlined in more detail in the [Adding Hosts To Fleet](./3-Adding-hosts.md) document. The steps are repeated here for the sake of a continuous tutorial.
> Note that this whole process is outlined in more detail in the [Adding Hosts To Fleet](../1-Using-Fleet/4-Adding-hosts.md) document. The steps are repeated here for the sake of a continuous tutorial.
To install osquery on Ubuntu, you can run the following:
docs/3-Deployment/4-fleetctl-agent-updates.md → docs/2-Deploying/4-fleetctl-agent-updates.md
View File
docs/3-Deployment/FAQ.md → docs/2-Deploying/FAQ.md
+1 -1
View File
@@ -21,7 +21,7 @@ Yes. Fleet scales horizontally out of the box as long as all of the Fleet server
Note that osquery logs will be distributed across the Fleet servers.
Read the [performance documentation](../1-Using-Fleet/5-Monitoring-Fleet.md#fleet-server-performance) for more.
Read the [performance documentation](../1-Using-Fleet/6-Monitoring-Fleet.md#fleet-server-performance) for more.
## Why aren't my osquery agents connecting to Fleet?
docs/3-Deployment/README.md → docs/2-Deploying/README.md
+1 -4
View File
@@ -6,10 +6,7 @@ Provides documentation on installing the Fleet binary and Fleet’s infrastructu
### [Configuration](./2-Configuration.md)
Includes resources for configuring the Fleet binary, managing osquery configurations, and running with systemd
### [Adding hosts](./3-Adding-hosts.md)
Provides resources for enrolling your hosts to Fleet
### [Example deployment scenarios](./4-Example-deployment-scenarios.md)
### [Example deployment scenarios](./3-Example-deployment-scenarios.md)
Includes deployment walkthroughs for Fleet on CentOS, Ubuntu, and Kubernetes.
### [Self-managed agent updates](./4-fleetctl-agent-updates.md)
docs/2-Orbit-osquery/README.md
-158
View File
@@ -1,158 +0,0 @@
# Orbit osquery
Orbit is an [osquery](https://github.com/osquery/osquery) runtime and autoupdater. With Orbit, it's easy to deploy osquery, manage configurations, and stay up to date. Orbit eases the deployment of osquery connected with a [Fleet server](https://github.com/fleetdm/fleet), and is a (near) drop-in replacement for osquery in a variety of deployment scenarios.
Orbit is the recommended agent for Fleet. But Orbit can be used with or without Fleet, and Fleet can be used with or without Orbit.
## Usage
General information and flag documentation can be accessed by running `orbit --help`.
### Permissions
Orbit generally expects root permissions to be able to create and access it's working files.
To get root level permissions:
#### macOS/Linux
Prefix `orbit` commands with `sudo` (`sudo orbit ...`) or run in a root shell.
#### Windows
Run Powershell or cmd.exe with "Run as administrator" and start `orbit` commands from that shell.
### Osquery shell
Run an `osqueryi` shell with `orbit osqueryi` or `orbit shell`.
### Connect to a Fleet server
Use the `--fleet-url` and `--enroll-secret` flags to connect to a Fleet server.
For example:
```sh
orbit --fleet-url=https://localhost:8080 --enroll-secret=the_secret_value
```
Use `--fleet_certificate` to provide a path to a certificate bundle when necessary for osquery to verify the authenticity of the Fleet server (typically when using a Windows client or self-signed certificates):
```sh
orbit --fleet-url=https://localhost:8080 --enroll-secret=the_secret_value --fleet-certificate=cert.pem
```
Add the `--insecure` flag for connections using otherwise invalid certificates:
```sh
orbit --fleet-url=https://localhost:8080 --enroll-secret=the_secret_value --insecure
```
### Osquery flags
Orbit can be used as near drop-in replacement for `osqueryd`, enhancing standard osquery with autoupdate capabilities. Orbit passes through any options after `--` directly to the `osqueryd` instance.
For example, the following would be a typical drop-in usage of Orbit:
```sh
orbit -- --flagfile=flags.txt
```
## Packaging
Orbit, like standalone osquery, is typically deployed via OS-specific packages. Tooling is provided with this repository to generate installation packages.
### Dependencies
Orbit currently supports building packages on macOS and Linux.
Before building packages, clone or download this repository and [install Go](https://golang.org/doc/install).
Building Windows packages requires Docker to be installed.
### Packaging support
- **macOS** - `.pkg` package generation with (optional) [Notarization](https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution) and codesigning - Persistence via `launchd`.
- **Linux** - `.deb` (Debian, Ubuntu, etc.) & `.rpm` (RHEL, CentOS, etc.) package generation - Persistence via `systemd`.
- **Windows** - `.msi` package generation - Persistence via Services.
### Building packages
Use `go run ./cmd/package` from this directory to run the packaging tools.
The only required parameter is `--type`, use one of `deb`, `rpm`, `pkg`, or `msi`.
Configure osquery to connect to a Fleet (or other TLS) server with the `--fleet-url` and `--enroll-secret` flags.
A minimal invocation for communicating with Fleet:
```sh
go run ./cmd/package --type deb --fleet-url=fleet.example.com --enroll-secret=notsosecret
```
This will build a `.deb` package configured to communicate with a Fleet server at `fleet.example.com` using the enroll secret `notsosecret`.
When the Fleet server uses a self-signed (or otherwise invalid) TLS certificate, package with the `--insecure` or `--fleet-certificate` options.
See `go run ./cmd/package` for the full range of packaging options.
#### Update channels
Orbit uses the concept of "update channels" to determine the version of Orbit, osquery, and any extensions (extension support coming soon) to run. This concept is modeled from the common versioning convention for Docker containers.
Configure update channels for Orbit and osqueryd with the `--orbit-channel` and `--osqueryd-channel` flags when packaging.
| Channel | Versions |
| ------- | -------- |
| `4` | 4.x.x |
| `4.6` | 4.6.x |
| `4.6.0` | 4.6.0 |
Additionally `stable` and `edge` are special channel names. `stable` will always return the version Fleet deems to be stable, while `edge` will provide newer releases for beta testing.
#### macOS signing & Notarization
Orbit's packager can automate the codesigning and Notarization steps to allow the resulting package to generate packages that appear "trusted" when install on macOS hosts. Signing & notarization are supported only on macOS hosts.
For signing, a "Developer ID Installer" certificate must be available on the build machine ([generation instructions](https://help.apple.com/xcode/mac/current/#/dev154b28f09)). Use `security find-identity -v` to verify the existence of this certificate and make note of the identifier provided in the left column.
For Notarization, valid App Store Connect credentials must be available on the build machine. Set these in the environment variables `AC_USERNAME` and `AC_PASSWORD`. It is common to configure this via [app-specific passwords](https://support.apple.com/en-ca/HT204397).
Build a signed and notarized macOS package with an invocation like the following:
```sh
AC_USERNAME=zach@example.com AC_PASSWORD=llpk-sije-kjlz-jdzw go run ./cmd/package --type=pkg --fleet-url=fleet.example.com --enroll-secret=63SBzTT+2UyW --sign-identity 3D7260BF99539C6E80A94835A8921A988F4E6498 --notarize
```
This process may take several minutes to complete as the Notarization process completes on Apple's servers.
After successful notarization, the generated "ticket" is automatically stapled to the package.
## FAQs
### How does Orbit compare with Kolide Launcher?
Orbit is inspired by the success of [Kolide Launcher](https://github.com/kolide/launcher), and approaches a similar problem domain with new strategies informed by the challenges encountered in real world deployments. Orbit does not share any code with Launcher.
- Both Orbit and Launcher use [The Update Framework](https://theupdateframework.com/) specification for managing updates. Orbit utilizes the official [go-tuf](https://github.com/theupdateframework/go-tuf) library, while Launcher has it's own implementation of the specification.
- Orbit can be deployed as a (near) drop-in replacement for osquery, supporting full customization of the osquery flags. Launcher heavily manages the osquery flags making deployment outside of Fleet or Kolide's SaaS difficult.
- Orbit prefers the battle-tested plugins of osquery. Orbit uses the built-in logging, configuration, and live query plugins, while Launcher uses custom implementations.
- Orbit prefers the built-in osquery remote APIs. Launcher utilizes a custom gRPC API that has led to issues with character encoding, load balancers/proxies, and request size limits.
- Orbit encourages use of the osquery performance Watchdog, while Launcher disables the Watchdog.
Additionally, Orbit aims to tackle problems out of scope for Launcher:
- Configure updates via release channels, providing more granular control over agent versioning.
- Support for deploying and updating osquery extensions (🔜).
- Manage osquery versions and startup flags from a remote (Fleet) server (🔜).
- Further control of osquery performance via cgroups (🔜).
### Is Orbit Free?
Yes! Orbit is licensed under an MIT license and all uses are encouraged.
### How does orbit update osquery? And how do the stable and edge channels get triggered to update osquery on a self hosted Fleet instance?
Orbit uses a configurable update server. We expect that many folks will just use the update server we manage (similar to what Kolide does with Launcher's update server). We are also offering [tooling for self-managing an update server](https://github.com/fleetdm/fleet/blob/master/docs/3-Deployment/4-fleetctl-agent-updates.md) as part of Fleet Basic (the subscription offering).
docs/4-Contribution/1-Building-Fleet.md → docs/3-Contributing/1-Building-Fleet.md
View File
docs/4-Contribution/2-Testing.md → docs/3-Contributing/2-Testing.md
View File
docs/4-Contribution/3-Migrations.md → docs/3-Contributing/3-Migrations.md
View File
docs/4-Contribution/4-Committing-Changes.md → docs/3-Contributing/4-Committing-Changes.md
View File
docs/4-Contribution/5-Releasing-Fleet.md → docs/3-Contributing/5-Releasing-Fleet.md
+1 -1
View File
@@ -28,7 +28,7 @@ Make note of the SHA256 checksum output at the end of this build command to past
### Upgrading
Please visit our [update guide](https://github.com/fleetdm/fleet/blob/master/docs/1-Using-Fleet/7-Updating-Fleet.md) for upgrade instructions.
Please visit our [update guide](https://github.com/fleetdm/fleet/blob/master/docs/1-Using-Fleet/8-Updating-Fleet.md) for upgrade instructions.
### Documentation
docs/4-Contribution/FAQ.md → docs/3-Contributing/FAQ.md
View File
docs/4-Contribution/README.md → docs/3-Contributing/README.md
View File
docs/README.md
+2 -5
View File
@@ -5,13 +5,10 @@ Welcome to the documentation for the Fleet osquery fleet manager.
### [Using Fleet](./1-Using-Fleet/README.md)
Resources for using the Fleet UI, fleetctl CLI, and Fleet REST API.
### [Orbit osquery](./2-Orbit-osquery/README.md)
Resources for using Orbit for osquery: the recommended agent for Fleet.
### [Deployment](./3-Deployment/README.md)
### [Deploying](./2-Deploying/README.md)
Resources for installing Fleet's infrastructure dependencies, configuring Fleet, deploying osquery to hosts, and viewing example deployment scenarios.
### [Contribution](./4-Contribution/README.md)
### [Contributing](./3-Contributing/README.md)
If you're interested in interacting with the Fleet source code, you'll find information on modifying and building the code here.
---