diff --git a/docs/01-Using-Fleet/04-Adding-hosts.md b/docs/01-Using-Fleet/04-Adding-hosts.md index 2050ad6a29..bbf149e5b5 100644 --- a/docs/01-Using-Fleet/04-Adding-hosts.md +++ b/docs/01-Using-Fleet/04-Adding-hosts.md @@ -1,44 +1,68 @@ # Adding hosts -- [fleetctl packagey](#fleetctl-package) -- [Native osquery TLS plugins](#native-osquery-tls-plugins) - - [Set an environment variable with an agent enrollment secret](#set-an-environment-variable-with-an-agent-enrollment-secret) - - [Deploy the TLS certificate that osquery will use to communicate with Fleet](#deploy-the-tls-certificate-that-osquery-will-use-to-communicate-with-fleet) -- [Launching osqueryd](#launching-osqueryd) - - [Using a flag file to manage flags](#using-a-flag-file-to-manage-flags) -- [Kolide osquery Launcher](#kolide-osquery-launcher) -- [Enrolling multiple macOS hosts](#enrolling-multiple-macos-hosts) -- [Multiple enroll secrets](#multiple-enroll-secrets) +- [Osquery installer](#osquery-installer) +- [Plain osquery](#plain-osquery) -Fleet is powered by the open source osquery tool. To install osquery and connect a host to Fleet, you have three options: -- You can use the `fleetctl package` command to create an osquery installer. -- You can install the osquery binaries on your hosts via the packages distributed at https://osquery.io/downloads. -- You can use the [Kolide Osquery Launcher](https://github.com/kolide/launcher). +Fleet is powered by the open source osquery tool. To add a host to Fleet, you must install osquery on this host. -## fleetctl package +The recommended way to install osquery and add your host to Fleet is with an osquery installer. Fleet provides the tools to generate an osquery installer with the `fleetctl package` command. -You can use the `fleetctl package` command to create a fully bootstraped osquery installer, configured to communicate with your Fleet instance. +To use the `fleetctl package` command, you must first install the `fleetctl` command-line tool. Instructions for installing `fleetctl` can be found on [here fleetdm.com](https://fleetdm.com/get-started) -`fleetctl package` can be used to create an osquery installer for macOS (**.pkg**), Windows (**.msi**), and Linux systems (**.deb** or **.rpm**). +Fleet supports other methods for adding your hosts to Fleet such as the [plain osquery binaries](#plain-osquery) or [Kolide Osquery Launcher](https://github.com/kolide/launcher/blob/master/docs/launcher.md#connecting-to-fleet). -The following command would create an installable `.pkg` file for macOS, located in the folder where the command is run. +## Osquery installer + +To create an osquery installer, you can use the `fleetctl package` command. + +`fleetctl package` can be used to create an osquery installer which adds macOS hosts (**.pkg**), Windows hosts (**.msi**), or Linux hosts (**.deb** or **.rpm**) to Fleet. + +The following command creates an osquery installer, `.pkg` file, which adds macOS hosts to Fleet. This osquery installer is located in the folder where the `fleetctl package` command is run. ```sh fleetctl package --type pkg --fleet-url=[YOUR FLEET URL] --enroll-secret=[YOUR ENROLLMENT SECRET] ``` >**Note:** The only configuration option required to create an installer is `--type`, but to communicate with a Fleet instance you'll need to specify a `--fleet-url` and `--enroll-secret` -When installing osquery using the generated package, host machines will be automatically enrolled in the specified Fleet instance. +When you install the generated osquery installer on a host, this host will be automatically enrolled in the specified Fleet instance. -### Configuration +### Adding multiple hosts -To configure an osquery installer to comminucate with a specific Fleet instance, you can pass in the CLI flags below. +If you're managing an enterprise environment with multiple hosts, you likely have an enterprise deployment tool like [Munki](https://www.munki.org/munki/), [Jamf Pro](https://www.jamf.com/products/jamf-pro/), [Chef](https://www.chef.io/), [Ansible](https://www.ansible.com/), or [Puppet](https://puppet.com/) to deliver software to your hosts. + +You can distribute your osquery installer and add all your hosts to Fleet using your software management tool of choice. + +### Automatically adding hosts to a team + +`Applies only to Fleet Premium` + +``` +ℹ️ In Fleet 4.0, Teams were introduced. +``` + +The teams feature in Fleet allows you to place hosts in exclusive groups. With hosts segmented into teams, you can apply unique queries and give users access to only the hosts in specific teams. + +You can add a host to a team by generating and using a unique osquery installer for a team or by [manually transferring a host to a team in the Fleet UI](../01-Using-Fleet/10-Teams.md#transfer-hosts-to-a-team). + +To generate an osquery installer for a team: + +1. First, create a team in Fleet by selecting "Create team" in **Settings > Teams**. + +2. Then, navigate to **Hosts** and select your team. + +3. Next, select "Generate installer" and copy the `fleetctl package` command for the platform (macOS, Windows, Linux) of the hosts you'd like to add to a team in Fleet. + +4. Run the copied `fleetctl package` command and [distribute your installer](#adding-multiple-hosts) to add your hosts to a team in Fleet. + +### Configuration options + +The following command-line flags allow you to further configure an osquery installer to communicate with a specific Fleet instance. |Flag | Options| |------|--------| | --type | **Required** - Type of package to build.
Options: `pkg`(macOS),`msi`(Windows), `deb`(Debian based Linux), `rpm`(RHEL, CentOS, etc.)| |--enroll-secret | Enroll secret for authenticating to Fleet server | |--fleet-url | URL (`host:port`) of Fleet server | -|--fleet-certificate | Path to server cerificate bundle | +|--fleet-certificate | Path to server certificate bundle | |--identifier | Identifier for package product (default: `com.fleetdm.orbit`) | |--version | Version for package product (default: `0.0.3`) | | --insecure | Disable TLS certificate verification (default: `false`) | @@ -54,20 +78,20 @@ To configure an osquery installer to comminucate with a specific Fleet instance, | --help, -h | show help (default: `false`) | -## Native osquery TLS plugins +## Plain osquery > If you'd like to use the native osqueryd binaries to connect to Fleet, this is enabled by using osquery's TLS API plugins that are principally documented on the official osquery wiki: http://osquery.readthedocs.io/en/stable/deployment/remote/. These plugins are very customizable and thus have a large configuration surface. Configuring osqueryd to communicate with Fleet is documented below in the "Native Osquery TLS Plugins" section. You can find various ways to install osquery on a variety of platforms at https://osquery.io/downloads. Once you have installed osquery, you need to do two things: -### Set an environment variable with an agent enrollment secret +### Set an environment variable with an enroll secret -The enrollment secret is a value that osquery provides to authenticate with Fleet. There are a few ways you can set the enrollment secret on the hosts which you control. You can either set the value as: +The enroll secret is a value that osquery provides to authenticate with Fleet. There are a few ways you can set the enroll secret on the hosts which you control. You can either set the value as: - an value of an environment variable (a common name is `OSQUERY_ENROLL_SECRET`) -- the content of a local file (a common path is `/etc/osquery/enrollment_secret`) +- the content of a local file (a common path is `/etc/osquery/enroll_secret`) -The value of the environment variable or content of the file should be a secret shared between the osqueryd client and the Fleet server. This is basically osqueryd's passphrase which it uses to authenticate with Fleet, convincing Fleet that it is actually one of your hosts. The passphrase could be whatever you'd like, but it would be prudent to have the passphrase long, complex, mixed-case, etc. When you launch the Fleet server, you should specify this same value. +The value of the environment variable or content of the file should be a secret shared between the osqueryd client and the Fleet server. This is osqueryd's passphrase which it uses to authenticate with Fleet, convincing Fleet that it is actually one of your hosts. The passphrase could be whatever you'd like, but it would be prudent to have the passphrase long, complex, mixed-case, etc. When you launch the Fleet server, you should specify this same value. If you use an environment variable for this, you can specify it with the `--enroll_secret_env` flag when you launch osqueryd. If you use a local file for this, you can specify it's path with the `--enroll_secret_path` flag. @@ -84,16 +108,16 @@ It is important that the CN of this certificate matches the hostname or IP that Specify the path to this certificate with the `--tls_server_certs` flag when you launch osqueryd. -## Launching osqueryd +### Launching osqueryd -Assuming that you are deploying your enrollment secret in the file `/etc/osquery/enroll_secret` and your osquery server certificate is at `/etc/osquery/fleet.crt`, you could copy and paste the following command with the following flags (be sure to replace `fleet.acme.net` with the hostname or IP of your Fleet installation): +Assuming that you are deploying your enroll secret in the file `/etc/osquery/enroll_secret` and your osquery server certificate is at `/etc/osquery/fleet.crt`, you could copy and paste the following command with the following flags (be sure to replace `fleet.acme.net` with the hostname or IP of your Fleet installation): ``` sudo osqueryd \ --enroll_secret_path=/etc/osquery/enroll_secret \ --tls_server_certs=/etc/osquery/fleet.crt \ - --tls_hostname=fleet.acme.net \ - --host_identifier=instance \ + --tls_hostname=fleet.example.com \ + --host_identifier=uuid \ --enroll_tls_endpoint=/api/v1/osquery/enroll \ --config_plugin=tls \ --config_tls_endpoint=/api/v1/osquery/config \ @@ -109,7 +133,9 @@ sudo osqueryd \ --logger_tls_period=10 ``` -If your osquery server certificate is deployed to a path that is not `/etc/osquery/fleet.crt`, be sure to update the `--tls_server_certs` flag. Similarly, if your enrollment secret is in an environment variable that is not called `OSQUERY_ENROLL_SECRET`, then be sure to update the `--enroll_secret_env` environment variable. If your enroll secret is defined in a local file, specify the file's path with the `--enroll_secret_path` flag instead of using the `--enroll_secret_env` flag. +If your osquery server certificate is deployed to a path that is not `/etc/osquery/fleet.crt`, be sure to update the `--tls_server_certs` flag. Similarly, if your enroll secret is in an environment variable that is not called `OSQUERY_ENROLL_SECRET`. Then, be sure to update the `--enroll_secret_env` environment variable. + +If your enroll secret is defined in a local file, specify the file's path with the `--enroll_secret_path` flag instead of using the `--enroll_secret_env` flag. ### Using a flag file to manage flags @@ -128,43 +154,36 @@ Ensure that paths to files in the flag file are absolute, and not quoted. For ex --enroll_secret_path=C:\Program Files\osquery\secret.txt ``` -## Kolide osquery Launcher +### Migrating from plain osquery to osquery installer -Instructions on connecting a single Launcher to Fleet can be found [here in the Launcher documentation](https://github.com/kolide/launcher/blob/master/docs/launcher.md#connecting-to-fleet). +The following is a strategy for migrating a plain osquery deployment. Unlike plain osquery, Fleet's +osquery installer supports the automatic updating of osquery on your hosts so that you don't have to +deploy a new package for every new osquery release. -Kolide provides compiled releases of their launcher for all supported platforms. -Those can be found [here](https://github.com/kolide/launcher/releases), or if you’d like to compile from source, the instructions are [here](https://github.com/kolide/launcher/blob/master/docs/launcher.md#building-the-code). +#### Generate installer -## Enrolling multiple macOS hosts +``` +fleetctl package --type [pkg|msi|deb|rpm] --fleet-url [fleet-hostname:port] --enroll-secret [secret] +``` -If you're managing an enterprise environment with multiple Mac devices, you likely have an enterprise deployment tool like [Munki](https://www.munki.org/munki/) or [Jamf Pro](https://www.jamf.com/products/jamf-pro/) to deliver software to your mac fleet. You can deploy osqueryd and enroll all your macs into Fleet using your software management tool of choice. +If you currently ship a certificate (`fleet.pem`), also include this in the generated package with +`--fleet-certificate [/path/to/fleet.pem]`. -First, [download](https://osquery.io/downloads/) and import the osquery package into your software management repository. You can also use the community supported [autopkg recipe](https://github.com/autopkg/keeleysam-recipes/tree/master/osquery) -to keep osqueryd updated. +Fleet automatically manages most of the osquery flags to connect to the Fleet server. There's no +need to set any of the flags mentioned above in [Launching osqueryd](#launching-osqueryd). To +include other osquery flags, provide a flagfile when packaging with `--osquery-flagfile +[/path/to/osquery.flags]`. -Next, you will have to create an enrollment package to get osqueryd running and talking to Fleet. Specifically, you'll have to create a custom package because you have to provide specific information about your Fleet deployment. To make this as easy as possible, we've created a Makefile to help you build a macOS enrollment package. +Test the installers on each platform before initiating the migration. -First, download the Fleet repository from GitHub and navigate to the `tools/mac` directory of the repository. +#### Migrate -Next, you'll have to edit the `config.mk` file. You'll find all of the necessary information by clicking "Add New Host" in your Fleet server. +Using your standard deployment tooling (Chef, Puppet, etc.), install the generated package. At this +time, [uninstall the existing +osquery](https://blog.fleetdm.com/how-to-uninstall-osquery-f01cc49a37b9). - - Set the `KOLIDE_HOSTNAME` variable to the FQDN of your Fleet server. - - Set the `ENROLL_SECRET` variable to the enroll secret you got from Fleet. - - Paste the contents of the Fleet TLS certificate after the following line: - ``` - define KOLIDE_TLS_CERTIFICATE - ``` - -Note that osqueryd requires a full certificate chain, even for certificates which might be trusted by your keychain. The "Fetch Fleet Certificate" button in the Add New Host screen will attempt to fetch the full chain for you. - -Once you've configured the `config.mk` file with the correct variables, you can run `make` in the `tools/mac` directory. Running `make` will create a new `kolide-enroll.pkg` file which you can import into your software repository and deploy to your mac fleet. - -The enrollment package must installed after the osqueryd package, and will install a LaunchDaemon to keep the osqueryd process running. - -## Multiple enroll secrets - -Multiple enroll secrets can be set to allow different groups of hosts to -authenticate with Fleet. When a host enrolls, the corresponding enroll secret is -recorded and can be used to segment hosts. - -To set the enroll secret, use the `fleetctl` tool to apply an [enroll secret spec](../01-Using-Fleet/02-fleetctl-CLI.md#enroll-secrets) +If the existing enrolled hosts use `--host_identifier=uuid` (or the `uuid` setting for Fleet's +[osquery_host_identifier](../02-Deploying/02-Configuration.md#osquery-host-identifier)), the new +installation should appear as the same host in the Fleet UI. If other settings are used, duplicate +entries will appear in the Fleet UI. The older entries can be automatically cleaned up with the host +expiration functionality configured in the application settings (UI or fleetctl). \ No newline at end of file diff --git a/docs/01-Using-Fleet/10-Teams.md b/docs/01-Using-Fleet/10-Teams.md index 4904e900db..0944035fa4 100644 --- a/docs/01-Using-Fleet/10-Teams.md +++ b/docs/01-Using-Fleet/10-Teams.md @@ -8,7 +8,7 @@ - [View teams](#view-teams) - [Create a team](#create-a-teams) -- [Enroll hosts to a team](#enroll-hosts-to-a-team) +- [Automatically adding hosts to a team](#automatically-adding-hosts-to-a-team) - [Transfer hosts to a team](#transfer-hosts-to-a-team) - [Add users to a team](#add-users-to-a-team) - [Remove a member from a team](#remove-a-member-from-a-team) @@ -50,21 +50,13 @@ To create a team: 3. Enter your new team's name and select "Save." -## Enroll hosts to a team +## Automatically adding hosts to a team Hosts can only belong to one team in Fleet. -You can transfer hosts to a new team in Fleet by either enrolling the host with a team's enroll secret or by [transferring the host via the Fleet UI](#transfer-hosts-to-a-team) after the host has been enrolled to Fleet. +You can add hosts to a new team in Fleet by either enrolling the host with a team's enroll secret or by [transferring the host via the Fleet UI](#transfer-hosts-to-a-team) after the host has been enrolled to Fleet. -To enroll hosts to a team: - -1. In the top navigation, select "Hosts" and the on the right side, select "Enroll new host." - -2. In the "Enroll secret" section of the modal, select the team you'd like to transfer your hosts to. - -3. Copy or download the team's enroll secret. Use this enroll secret when installing the osquery agents on your hosts to Fleet. - -Orbit is the recommended agent for Fleet. Check out [the Orbit for osquery documentation](https://github.com/fleetdm/fleet/blob/main/orbit/README.md) for instructions for packaging and deploying Orbit to your hosts. +To automatically add hosts to a team in Fleet check out the ["Adding hosts" documentation](./04-Adding-hosts.md#automatically-adding-hosts-to-a-team). ## Transfer hosts to a team diff --git a/docs/02-Deploying/01-Installation.md b/docs/02-Deploying/01-Installation.md index d59018f353..9fd6bd7e11 100644 --- a/docs/02-Deploying/01-Installation.md +++ b/docs/02-Deploying/01-Installation.md @@ -9,13 +9,13 @@ - [MySQL](#mysql) - [Redis](#redis) -The Fleet application is distributed as a single static binary. This binary serves: +The Fleet application is distributed as a single static binary (or as a Docker container). This binary serves: - The Fleet web interface - The Fleet application API endpoints - The osquery TLS server API endpoints -All of these are served via a built-in HTTP server, so there is no need for complex web server configurations. Once you've installed the `fleet` binary and it's infrastructure dependencies as illustrated below, refer to the [Configuration](./02-Configuration.md) documentation for information on how to use and configure the Fleet application. +All of these are served via a built-in HTTP server, so there is no need for complex web server configurations. Once you've installed the `fleet` binary and its infrastructure dependencies as illustrated below, refer to the [Configuration](./02-Configuration.md) documentation for information on how to use and configure the Fleet application. ## Installing the Fleet binary @@ -69,8 +69,6 @@ Fleet uses MySQL extensively as its main database. Many cloud providers (such as Fleet requires at least MySQL version 5.7. -For host expiry configuration, the [event scheduler](https://dev.mysql.com/doc/refman/5.7/en/events-overview.html) must be enabled. This can be enabled via the command line, configuration file, or a user with the required privileges. - ### Redis Fleet uses Redis to ingest and queue the results of distributed queries, cache data, etc. Many cloud providers (such as [AWS](https://aws.amazon.com/elasticache/) and [GCP](https://console.cloud.google.com/launcher/details/click-to-deploy-images/redis)) host reliable Redis services which you may consider for this purpose. A well supported Redis [Docker container](https://hub.docker.com/_/redis/) also exists if you would rather run Redis in a container. For more information on how to configure the `fleet` binary to use the correct Redis instance, see the [Configuration](./02-Configuration.md) document. diff --git a/docs/02-Deploying/02-Configuration.md b/docs/02-Deploying/02-Configuration.md index 9f3202d368..fafde4029f 100644 --- a/docs/02-Deploying/02-Configuration.md +++ b/docs/02-Deploying/02-Configuration.md @@ -807,7 +807,7 @@ The identifier to use when determining uniqueness of hosts. Options are `provided` (default), `uuid`, `hostname`, or `instance`. -This setting works in combination with the `--host_identifier` flag in osquery. In most deployments, using `instance` will be the best option. The flag defaults to `provided` -- preserving the existing behavior of Fleet's handling of host identifiers -- using the identifier provided by osquery. `instance`, `uuid`, and `hostname` correspond to the same meanings as for osquery's `--host_identifier` flag. +This setting works in combination with the `--host_identifier` flag in osquery. In most deployments, using `uuid` will be the best option. The flag defaults to `provided` -- preserving the existing behavior of Fleet's handling of host identifiers -- using the identifier provided by osquery. `instance`, `uuid`, and `hostname` correspond to the same meanings as for osquery's `--host_identifier` flag. Users that have duplicate UUIDs in their environment can benefit from setting this flag to `instance`. diff --git a/tools/mac/Makefile b/tools/mac/Makefile deleted file mode 100644 index a6a51a60ca..0000000000 --- a/tools/mac/Makefile +++ /dev/null @@ -1,25 +0,0 @@ -PKGNAME=kolide-enroll -PKGVERSION=1.0.0 -PKGID=co.kolide.osquery.enroll - --include config.mk - -export KOLIDE_FLAGS -export KOLIDE_TLS_CERTIFICATE - - -all: clean build -clean: - rm -rf out/ - rm -rf root/etc/osquery - -build: clean - mkdir -p out - mkdir -p root/etc/osquery - echo $(ENROLL_SECRET) > root/etc/osquery/kolide_secret - echo "$$KOLIDE_TLS_CERTIFICATE" > root/etc/osquery/kolide.crt - - # validate the certificate - openssl x509 -in root/etc/osquery/kolide.crt -text > /dev/null - echo "$$KOLIDE_FLAGS" > root/etc/osquery/kolide.flags - pkgbuild --root root --scripts scripts --identifier ${PKGID} --version ${PKGVERSION} out/${PKGNAME}-${PKGVERSION}.pkg diff --git a/tools/mac/config.mk b/tools/mac/config.mk deleted file mode 100644 index 53f7937d89..0000000000 --- a/tools/mac/config.mk +++ /dev/null @@ -1,40 +0,0 @@ -# Kolide hostname. Make sure omit https:// or the path -KOLIDE_HOSTNAME=kolide.acme.co - -# Osquery enroll secret. Replace with the secret set in Kolide. -ENROLL_SECRET=CHANGEME - -# Paste your kolide certificate chain below. -define KOLIDE_TLS_CERTIFICATE -CHANGEME -endef - -# Osquery flag file. No need to modify. -define KOLIDE_FLAGS ---force=true ---host_identifier=instance ---verbose=true ---debug ---tls_dump=true - ---tls_hostname=$(KOLIDE_HOSTNAME) ---tls_server_certs=/etc/osquery/kolide.crt ---enroll_secret_path=/etc/osquery/kolide_secret - ---enroll_tls_endpoint=/api/v1/osquery/enroll - ---config_plugin=tls ---config_tls_endpoint=/api/v1/osquery/config ---config_refresh=10 - ---disable_distributed=false ---distributed_plugin=tls ---distributed_interval=10 ---distributed_tls_max_attempts=3 ---distributed_tls_read_endpoint=/api/v1/osquery/distributed/read ---distributed_tls_write_endpoint=/api/v1/osquery/distributed/write - ---logger_plugin=tls ---logger_tls_endpoint=/api/v1/osquery/log ---logger_tls_period=10 -endef diff --git a/tools/mac/root/Library/LaunchDaemons/co.kolide.osquery.enroll.plist b/tools/mac/root/Library/LaunchDaemons/co.kolide.osquery.enroll.plist deleted file mode 100644 index 0ee18e41ce..0000000000 --- a/tools/mac/root/Library/LaunchDaemons/co.kolide.osquery.enroll.plist +++ /dev/null @@ -1,24 +0,0 @@ - - - - - Label - co.kolide.osquery.enroll - ProgramArguments - - /usr/local/bin/osqueryd - --flagfile=/etc/osquery/kolide.flags - - RunAtLoad - - KeepAlive - - ThrottleInterval - 60 - StandardErrorPath - /var/log/osquery/osquery-error.log - StandardOutPath - /var/log/osquery/osquery-output.log - - - diff --git a/tools/mac/scripts/postinstall b/tools/mac/scripts/postinstall deleted file mode 100755 index 8614d665dd..0000000000 --- a/tools/mac/scripts/postinstall +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash - -[[ $3 != "/" ]] && exit 0 - -/bin/launchctl load /Library/LaunchDaemons/co.kolide.osquery.enroll.plist - -exit 0