diff --git a/.github/workflows/markdown-link-check-config.json b/.github/workflows/markdown-link-check-config.json index 470c4b521d..e7d33f6cbf 100644 --- a/.github/workflows/markdown-link-check-config.json +++ b/.github/workflows/markdown-link-check-config.json @@ -23,6 +23,9 @@ }, { "pattern": "github.com/fleetdm/confidential" + }, + { + "pattern": "osquery.io/schema" } ], "retryOn429": true, diff --git a/docs/1-Using-Fleet/2-fleetctl-CLI.md b/docs/1-Using-Fleet/2-fleetctl-CLI.md index cf85636d6c..2ddba8aeaf 100644 --- a/docs/1-Using-Fleet/2-fleetctl-CLI.md +++ b/docs/1-Using-Fleet/2-fleetctl-CLI.md @@ -211,30 +211,7 @@ Note the token can also be set with `fleetctl config set --token`, but this may ## Using fleetctl for configuration -A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Each message can live in it's own file or multiple in one file, each separated by `---`. Each file/message contains a few required top-level keys: - -- `apiVersion` - the API version of the file/request -- `spec` - the "data" of the request -- `kind ` - the type of file/object (i.e.: pack, query, config) - -The file may optionally also include some `metadata` for more complex data types (i.e.: packs). - -When you reason about how to manage these config files, consider following the [General Config Tips](https://kubernetes.io/docs/concepts/configuration/overview/#general-config-tips) published by the Kubernetes project. Some of the especially relevant tips are included here as well: - -- When defining configurations, specify the latest stable API version. -- Configuration files should be stored in version control before being pushed to the cluster. This allows quick roll-back of a configuration if needed. It also aids with cluster re-creation and restoration if necessary. -- Group related objects into a single file whenever it makes sense. One file is often easier to manage than several. See the [config-single-file.yml](../../examples/config-single-file.yml) file as an example of this syntax. -- Don’t specify default values unnecessarily – simple and minimal configs will reduce errors. - -All of these files can be concatenated together into [one file](../../examples/config-single-file.yml) (separated by `---`), or they can be in [individual files with a directory structure](../../examples/config-many-files) like the following: - -``` -|-- config.yml -|-- labels.yml -|-- packs -| `-- osquery-monitoring.yml -`-- queries.yml -``` +A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Check out the [configuration files](./configuration-files/README.md) section of the documentation for example yaml files. ### Convert osquery JSON diff --git a/docs/1-Using-Fleet/configuration-files/README.md b/docs/1-Using-Fleet/configuration-files/README.md new file mode 100644 index 0000000000..83d8ca7de0 --- /dev/null +++ b/docs/1-Using-Fleet/configuration-files/README.md @@ -0,0 +1,34 @@ +# Configuration files + +Entities in Fleet, such as queries, packs, labels, agent options, and enroll secrets, can be managed with configuration files in yaml syntax. + +This page contains links to examples that can help you understand the configuration options for your Fleet yaml file(s). + +Examples in this directory are presented in two forms: +- [`single-file-configuration.yml`](./single-file-configuration.yml) presents multiple yaml documents in one file. One file is often easier to manage than several. Group related objects into a single file whenever it makes sense. +- The `multi-file-configuration` directory presents multiple yaml documents in separate files. They are in the following structure: + +``` +├─ packs +├ └─ osquery-monitoring.yml +├─ agent-options.yml +├─ enroll-secrets.yml +├─ labels.yml +├─ queries.yml +``` + +## Using yaml files in Fleet + +A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Each message can live in it's own file or multiple in one file, each separated by `---`. Each file/message contains a few required top-level keys: + +- `apiVersion` - the API version of the file/request +- `spec` - the "data" of the request +- `kind ` - the type of file/object (i.e.: pack, query, config) + +The file may optionally also include some `metadata` for more complex data types (i.e.: packs). + +When you reason about how to manage these config files, consider following the [General Config Tips](https://kubernetes.io/docs/concepts/configuration/overview/#general-config-tips) published by the Kubernetes project. Some of the especially relevant tips are included here as well: + +- When defining configurations, specify the latest stable API version. +- Configuration files should be stored in version control before being pushed to the cluster. This allows quick roll-back of a configuration if needed. It also aids with cluster re-creation and restoration if necessary. +- Don’t specify default values unnecessarily – simple and minimal configs will reduce errors. diff --git a/examples/kubernetes/fleet-deployment.yml b/docs/1-Using-Fleet/configuration-files/kubernetes/fleet-deployment.yml similarity index 100% rename from examples/kubernetes/fleet-deployment.yml rename to docs/1-Using-Fleet/configuration-files/kubernetes/fleet-deployment.yml diff --git a/examples/kubernetes/fleet-migrations.yml b/docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml similarity index 100% rename from examples/kubernetes/fleet-migrations.yml rename to docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml diff --git a/examples/kubernetes/fleet-service.yml b/docs/1-Using-Fleet/configuration-files/kubernetes/fleet-service.yml similarity index 100% rename from examples/kubernetes/fleet-service.yml rename to docs/1-Using-Fleet/configuration-files/kubernetes/fleet-service.yml diff --git a/examples/config-many-files/config.yml b/docs/1-Using-Fleet/configuration-files/multi-file-configuration/agent-options.yml similarity index 100% rename from examples/config-many-files/config.yml rename to docs/1-Using-Fleet/configuration-files/multi-file-configuration/agent-options.yml diff --git a/examples/config-many-files/enroll-secret.yml b/docs/1-Using-Fleet/configuration-files/multi-file-configuration/enroll-secrets.yml similarity index 100% rename from examples/config-many-files/enroll-secret.yml rename to docs/1-Using-Fleet/configuration-files/multi-file-configuration/enroll-secrets.yml diff --git a/examples/config-many-files/labels.yml b/docs/1-Using-Fleet/configuration-files/multi-file-configuration/labels.yml similarity index 100% rename from examples/config-many-files/labels.yml rename to docs/1-Using-Fleet/configuration-files/multi-file-configuration/labels.yml diff --git a/examples/config-many-files/packs/osquery-monitoring.yml b/docs/1-Using-Fleet/configuration-files/multi-file-configuration/packs/osquery-monitoring.yml similarity index 100% rename from examples/config-many-files/packs/osquery-monitoring.yml rename to docs/1-Using-Fleet/configuration-files/multi-file-configuration/packs/osquery-monitoring.yml diff --git a/examples/config-many-files/queries.yml b/docs/1-Using-Fleet/configuration-files/multi-file-configuration/queries.yml similarity index 100% rename from examples/config-many-files/queries.yml rename to docs/1-Using-Fleet/configuration-files/multi-file-configuration/queries.yml diff --git a/examples/config-single-file.yml b/docs/1-Using-Fleet/configuration-files/single-file-configuration.yml similarity index 100% rename from examples/config-single-file.yml rename to docs/1-Using-Fleet/configuration-files/single-file-configuration.yml diff --git a/docs/1-Using-Fleet/standard-query-library/README.md b/docs/1-Using-Fleet/standard-query-library/README.md new file mode 100644 index 0000000000..800454449e --- /dev/null +++ b/docs/1-Using-Fleet/standard-query-library/README.md @@ -0,0 +1,38 @@ +# Standard query library + +Fleet's standard query library includes a growing collection of useful queries for organizations deploying Fleet and osquery. + +## Importing the queries in Fleet + +#### After cloning the fleetdm/fleet repo, import the queries using fleetctl: +``` +fleetctl apply -f fleet/docs/1-Using-Fleet/standard-query-library/standard-query-library.yml +``` + +## Contributors + +Want to add your own query? + +1. Please copy the following yaml section and paste it at the bottom of the [`standard-query-library.yml`](./standard-query-library.yml) file. +```yaml +--- +apiVersion: v1 +kind: query +spec: + name: What is your query called? Please use a human readable query name. + platforms: What operating systems support your query? This can usually be determined by the osquery tables included in your query. Heading to the https://osquery.io/schema webpage to see which operating systems are supported by the tables you include. + description: Describe your query. What does information does your query reveal? + query: Insert query here + purpose: What is the goal of running your query? Ex. Detection + remediation: Are there any remediation steps to resolve the detection triggered by your query? If not, insert "N/A." +``` +2. Replace each field and submit a pull request to the fleetdm/fleet GitHub repository. + +For instructions on submitting pull requests to Fleet check out [the Committing Changes section](https://github.com/fleetdm/fleet/blob/58445ede82550cb574775a83ae4cf5433f325a7e/docs/4-Contribution/4-Committing-Changes.md#committing-changes) in the Contributors documentation. + +## Additional resources + +Listed below are great resources that contain additional queries. + +- Osquery (https://github.com/osquery/osquery/tree/master/packs) +- Palantir osquery configuration (https://github.com/palantir/osquery-configuration/tree/master/Fleet) \ No newline at end of file diff --git a/docs/1-Using-Fleet/standard-query-library/standard-query-library.yml b/docs/1-Using-Fleet/standard-query-library/standard-query-library.yml new file mode 100644 index 0000000000..c822af202b --- /dev/null +++ b/docs/1-Using-Fleet/standard-query-library/standard-query-library.yml @@ -0,0 +1,250 @@ +--- +apiVersion: v1 +kind: query +spec: + name: Count Apple applications installed + platforms: macOS + description: Count the number of Apple applications installed on the machine. + query: SELECT COUNT(*) FROM apps WHERE bundle_identifier LIKE 'com.apple.%'; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Detect Linux hosts with high severity vulnerable versions of OpenSSL + platforms: Linux + description: Retrieves the OpenSSL version. + query: SELECT name AS name, version AS version, 'deb_packages' AS source FROM deb_packages WHERE name LIKE 'openssl%' UNION SELECT name AS name, version AS version, 'apt_sources' AS source FROM apt_sources WHERE name LIKE 'openssl%' UNION SELECT name AS name, version AS version, 'rpm_packages' AS source FROM rpm_packages WHERE name LIKE 'openssl%'; + purpose: Detection + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Detect machines with Gatekeeper disabled + platforms: macOS + description: Gatekeeper tries to ensure only trusted software is run on a mac machine. + query: SELECT * FROM gatekeeper WHERE assessments_enabled = 0; + purpose: Detection + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Detect presence of authorized SSH keys + platforms: macOS, Linux + description: Presence of authorized SSH keys may be unusual on laptops. Could be completely normal on servers, but may be worth auditing for unusual keys and/or changes. + query: SELECT username, authorized_keys. * FROM users CROSS JOIN authorized_keys USING (uid); + purpose: Detection + remediation: Check out the linked table (https://github.com/fleetdm/fleet/blob/32b4d53e7f1428ce43b0f9fa52838cbe7b413eed/handbook/queries/detect-hosts-with-high-severity-vulnerable-versions-of-openssl.md#table-of-vulnerable-openssl-versions) to determine if the installed version is a high severity vulnerability and view the corresponding CVE(s) +--- +apiVersion: v1 +kind: query +spec: + name: Get authorized keys + platforms: macOS, Linux + description: List authorized_keys for each user on the system. + query: SELECT * FROM users CROSS JOIN authorized_keys USING (uid); + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get crashes + platforms: macOS + description: Retrieve application, system, and mobile app crash logs. + query: SELECT uid, datetime, responsible, exception_type, identifier, version, crash_path FROM users CROSS JOIN crashes USING (uid); + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed Chrome Extensions + platforms: macOS, Linux, Windows, FreeBSD + description: List installed Chrome Extensions for all users. + query: SELECT uid, datetime, responsible, exception_type, identifier, version, crash_path FROM users CROSS JOIN crashes USING (uid); + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed FreeBSD software + platforms: FreeBSD + description: Get all software installed on a FreeBSD computer, including browser plugins and installed packages. Note, this does not included other running processes in the processes table. + query: SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Package (pkg)' AS type, 'pkg_packages' AS source FROM pkg_packages; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get Homebrew Packages + platforms: macOS + description: Get the installed homebrew package database. + query: SELECT * FROM homebrew_packages; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed Linux software + platforms: Linux + description: Get all software installed on a Linux computer, including browser plugins and installed packages. Note, this does not included other running processes in the processes table. + query: SELECT name AS name, version AS version, 'Package (APT)' AS type, 'apt_sources' AS source FROM apt_sources UNION SELECT name AS name, version AS version, 'Package (deb)' AS type, 'deb_packages' AS source FROM deb_packages UNION SELECT package AS name, version AS version, 'Package (Portage)' AS type, 'portage_packages' AS source FROM portage_packages UNION SELECT name AS name, version AS version, 'Package (RPM)' AS type, 'rpm_packages' AS source FROM rpm_packages UNION SELECT name AS name, '' AS version, 'Package (YUM)' AS type, 'yum_sources' AS source FROM yum_sources UNION SELECT name AS name, version AS version, 'Package (NPM)' AS type, 'npm_packages' AS source FROM npm_packages UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed macOS software + platforms: macOS + description: Get all software installed on a macOS computer, including apps, browser plugins, and installed packages. Note, this does not included other running processes in the processes table. + query: SELECT name AS name, bundle_short_version AS version, 'Application (macOS)' AS type, 'apps' AS source FROM apps UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name As name, version AS version, 'Browser plugin (Safari)' AS type, 'safari_extensions' AS source FROM safari_extensions UNION SELECT name AS name, version AS version, 'Package (Homebrew)' AS type, 'homebrew_packages' AS source FROM homebrew_packages; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed Safari extensions + platforms: macOS + description: Retrieves the list of installed Safari Extensions for all users in the target system. + query: SELECT safari_extensions.* FROM users join safari_extensions USING (uid); + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get installed Windows software + platforms: Windows + description: Get all software installed on a Windows computer, including programs, browser plugins, and installed packages. Note, this does not included other running processes in the processes table. + query: SELECT name AS name, version AS version, 'Program (Windows)' AS type, 'programs' AS source FROM programs UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Browser plugin (IE)' AS type, 'ie_extensions' AS source FROM ie_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name AS name, version AS version, 'Package (Chocolatey)' AS type, 'chocolatey_packages' AS source FROM chocolatey_packages UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get laptops with failing batteries + platforms: macOS + description: + query: SELECT * FROM battery WHERE health != 'Good' AND condition NOT IN ('', 'Normal'); + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get macOS disk free space percentage + platforms: macOS + description: Displays the percentage of free space available on the primary disk partition. + query: SELECT (blocks_available * 100 / blocks) AS pct, * FROM mounts WHERE path = '/'; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get macOS disk free space percentage + platforms: macOS + description: Displays the percentage of free space available on the primary disk partition. + query: SELECT (blocks_available * 100 / blocks) AS pct, * FROM mounts WHERE path = '/'; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get mounts + platforms: macOS, Linux + description: Shows system mounted devices and filesystems (not process specific). + query: SELECT device, device_alias, path, type, blocks_size FROM mounts; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get the version of the resident operating system + platforms: macOS, Linux, Windows, FreeBSD + description: Shows system mounted devices and filesystems (not process specific). + query: SELECT * FROM os_version; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get platform info + platforms: macOS + description: Shows information about the host platform + query: SELECT vendor, version, date, revision from platform_info; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get startup items + platforms: macOS, Linux, Windows, FreeBSD + description: Shows applications and binaries set as user/login startup items. + query: SELECT * FROM startup_items; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get system logins and logouts + platforms: macOS + description: Get a list of system logins and logouts. + query: SELECT * FROM last; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get system uptime + platforms: macOS, Linux, Windows, FreeBSD + description: Shows the system uptime. + query: SELECT * FROM uptime; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get USB devices + platforms: macOS, Linux + description: Shows all USB devices that are actively plugged into the host system. + query: SELECT * FROM usb_devices; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get wifi status + platforms: macOS + description: Shows information about the wifi network that a host is currently connected to. + query: SELECT * FROM wifi_status; + purpose: Informational + remediation: N/A +--- +apiVersion: v1 +kind: query +spec: + name: Get Windows machines with unencrypted hard disks + platforms: Windows + description: + query: SELECT * FROM bitlocker_info WHERE protection_status = 0; + purpose: Informational + remediation: N/A diff --git a/docs/3-Deployment/4-Example-deployment-scenarios.md b/docs/3-Deployment/4-Example-deployment-scenarios.md index f9ba2e7e2a..a99fac5a6d 100644 --- a/docs/3-Deployment/4-Example-deployment-scenarios.md +++ b/docs/3-Deployment/4-Example-deployment-scenarios.md @@ -447,14 +447,14 @@ We will use this address when we configure the Kubernetes deployment and databas The last step is to run the Fleet database migrations on your new MySQL server. To do this, run the following: ``` -kubectl create -f ./examples/kubernetes/fleet-migrations.yml +kubectl create -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml ``` In Kubernetes, you can only run a job once. If you'd like to run it again (i.e.: you'd like to run the migrations again using the same file), you must delete the job before re-creating it. To delete the job and re-run it, you can run the following commands: ``` -kubectl delete -f ./examples/kubernetes/fleet-migrations.yml -kubectl create -f ./examples/kubernetes/fleet-migrations.yml +kubectl delete -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml +kubectl create -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml ``` #### Redis @@ -523,7 +523,7 @@ kubectl create secret generic fleet-server-auth-key --from-file=./build/fleet-se First we must deploy the instances of the Fleet webserver. The Fleet webserver is described using a Kubernetes deployment object. To create this deployment, run the following: ``` -kubectl apply -f ./examples/kubernetes/fleet-deployment.yml +kubectl apply -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-deployment.yml ``` You should be able to get an instance of the webserver running via `kubectl get pods` and you should see the following logs: @@ -539,7 +539,7 @@ ts=2017-11-16T02:48:38.441148166Z transport=https address=0.0.0.0:443 msg=listen Now that the Fleet server is running on our cluster, we have to expose the Fleet webservers to the internet via a load balancer. To create a Kubernetes `Service` of type `LoadBalancer`, run the following: ``` -kubectl apply -f ./examples/kubernetes/fleet-service.yml +kubectl apply -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-service.yml ``` #### Configure DNS diff --git a/handbook/queries/README.md b/handbook/queries/README.md index da038cf0c3..1b56c69ed7 100644 --- a/handbook/queries/README.md +++ b/handbook/queries/README.md @@ -30,13 +30,6 @@ Fleet's standard query library includes a growing collection of useful queries f - [Get system uptime](./get-system-uptime.md) (macOS, Linux, Windows, FreeBSD) - [Get crashes](./get-crashes.md) (macOS) - -### Contributors - -Want to add your own query? - -Please submit a pull request using the [Fleet standard query library template](./standard-query-library-template.md). - ### Additional resources Listed below are great resources that contain additional queries. diff --git a/handbook/queries/get-installed-safari-extensions.md b/handbook/queries/get-installed-safari-extensions.md index d26b050cf4..4e80c0703f 100644 --- a/handbook/queries/get-installed-safari-extensions.md +++ b/handbook/queries/get-installed-safari-extensions.md @@ -1,6 +1,6 @@ # Get installed Safari Extensions -Retreives the list of installed Safari Extensions for all users in the target system. +Retrieves the list of installed Safari Extensions for all users in the target system. ### Support macOS