Convert standard query library to YAML format (#749)
- Create `/configuration-files/` directory inside of `/1-Using-Fleet` directory. This directory contains example Fleet configuration files in yaml format. Replaces the `/examples` directory. - Create `/standard-query-library/` directory inside of `/1-Using-Fleet` directory. This directory contains the new `standard-query-library.yml`. This file will act as the source of community contributions to the standard query library. - Edit references to `/examples` directory
This commit is contained in:
noahtalerman
@@ -23,6 +23,9 @@
|
||||
},
|
||||
{
|
||||
"pattern": "github.com/fleetdm/confidential"
|
||||
},
|
||||
{
|
||||
"pattern": "osquery.io/schema"
|
||||
}
|
||||
],
|
||||
"retryOn429": true,
|
||||
|
||||
@@ -211,30 +211,7 @@ Note the token can also be set with `fleetctl config set --token`, but this may
|
||||
|
||||
## Using fleetctl for configuration
|
||||
|
||||
A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Each message can live in it's own file or multiple in one file, each separated by `---`. Each file/message contains a few required top-level keys:
|
||||
|
||||
- `apiVersion` - the API version of the file/request
|
||||
- `spec` - the "data" of the request
|
||||
- `kind ` - the type of file/object (i.e.: pack, query, config)
|
||||
|
||||
The file may optionally also include some `metadata` for more complex data types (i.e.: packs).
|
||||
|
||||
When you reason about how to manage these config files, consider following the [General Config Tips](https://kubernetes.io/docs/concepts/configuration/overview/#general-config-tips) published by the Kubernetes project. Some of the especially relevant tips are included here as well:
|
||||
|
||||
- When defining configurations, specify the latest stable API version.
|
||||
- Configuration files should be stored in version control before being pushed to the cluster. This allows quick roll-back of a configuration if needed. It also aids with cluster re-creation and restoration if necessary.
|
||||
- Group related objects into a single file whenever it makes sense. One file is often easier to manage than several. See the [config-single-file.yml](../../examples/config-single-file.yml) file as an example of this syntax.
|
||||
- Don’t specify default values unnecessarily – simple and minimal configs will reduce errors.
|
||||
|
||||
All of these files can be concatenated together into [one file](../../examples/config-single-file.yml) (separated by `---`), or they can be in [individual files with a directory structure](../../examples/config-many-files) like the following:
|
||||
|
||||
```
|
||||
|-- config.yml
|
||||
|-- labels.yml
|
||||
|-- packs
|
||||
| `-- osquery-monitoring.yml
|
||||
`-- queries.yml
|
||||
```
|
||||
A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Check out the [configuration files](./configuration-files/README.md) section of the documentation for example yaml files.
|
||||
|
||||
### Convert osquery JSON
|
||||
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Configuration files
|
||||
|
||||
Entities in Fleet, such as queries, packs, labels, agent options, and enroll secrets, can be managed with configuration files in yaml syntax.
|
||||
|
||||
This page contains links to examples that can help you understand the configuration options for your Fleet yaml file(s).
|
||||
|
||||
Examples in this directory are presented in two forms:
|
||||
- [`single-file-configuration.yml`](./single-file-configuration.yml) presents multiple yaml documents in one file. One file is often easier to manage than several. Group related objects into a single file whenever it makes sense.
|
||||
- The `multi-file-configuration` directory presents multiple yaml documents in separate files. They are in the following structure:
|
||||
|
||||
```
|
||||
├─ packs
|
||||
├ └─ osquery-monitoring.yml
|
||||
├─ agent-options.yml
|
||||
├─ enroll-secrets.yml
|
||||
├─ labels.yml
|
||||
├─ queries.yml
|
||||
```
|
||||
|
||||
## Using yaml files in Fleet
|
||||
|
||||
A Fleet configuration is defined using one or more declarative "messages" in yaml syntax. Each message can live in it's own file or multiple in one file, each separated by `---`. Each file/message contains a few required top-level keys:
|
||||
|
||||
- `apiVersion` - the API version of the file/request
|
||||
- `spec` - the "data" of the request
|
||||
- `kind ` - the type of file/object (i.e.: pack, query, config)
|
||||
|
||||
The file may optionally also include some `metadata` for more complex data types (i.e.: packs).
|
||||
|
||||
When you reason about how to manage these config files, consider following the [General Config Tips](https://kubernetes.io/docs/concepts/configuration/overview/#general-config-tips) published by the Kubernetes project. Some of the especially relevant tips are included here as well:
|
||||
|
||||
- When defining configurations, specify the latest stable API version.
|
||||
- Configuration files should be stored in version control before being pushed to the cluster. This allows quick roll-back of a configuration if needed. It also aids with cluster re-creation and restoration if necessary.
|
||||
- Don’t specify default values unnecessarily – simple and minimal configs will reduce errors.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Standard query library
|
||||
|
||||
Fleet's standard query library includes a growing collection of useful queries for organizations deploying Fleet and osquery.
|
||||
|
||||
## Importing the queries in Fleet
|
||||
|
||||
#### After cloning the fleetdm/fleet repo, import the queries using fleetctl:
|
||||
```
|
||||
fleetctl apply -f fleet/docs/1-Using-Fleet/standard-query-library/standard-query-library.yml
|
||||
```
|
||||
|
||||
## Contributors
|
||||
|
||||
Want to add your own query?
|
||||
|
||||
1. Please copy the following yaml section and paste it at the bottom of the [`standard-query-library.yml`](./standard-query-library.yml) file.
|
||||
```yaml
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: What is your query called? Please use a human readable query name.
|
||||
platforms: What operating systems support your query? This can usually be determined by the osquery tables included in your query. Heading to the https://osquery.io/schema webpage to see which operating systems are supported by the tables you include.
|
||||
description: Describe your query. What does information does your query reveal?
|
||||
query: Insert query here
|
||||
purpose: What is the goal of running your query? Ex. Detection
|
||||
remediation: Are there any remediation steps to resolve the detection triggered by your query? If not, insert "N/A."
|
||||
```
|
||||
2. Replace each field and submit a pull request to the fleetdm/fleet GitHub repository.
|
||||
|
||||
For instructions on submitting pull requests to Fleet check out [the Committing Changes section](https://github.com/fleetdm/fleet/blob/58445ede82550cb574775a83ae4cf5433f325a7e/docs/4-Contribution/4-Committing-Changes.md#committing-changes) in the Contributors documentation.
|
||||
|
||||
## Additional resources
|
||||
|
||||
Listed below are great resources that contain additional queries.
|
||||
|
||||
- Osquery (https://github.com/osquery/osquery/tree/master/packs)
|
||||
- Palantir osquery configuration (https://github.com/palantir/osquery-configuration/tree/master/Fleet)
|
||||
@@ -0,0 +1,250 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Count Apple applications installed
|
||||
platforms: macOS
|
||||
description: Count the number of Apple applications installed on the machine.
|
||||
query: SELECT COUNT(*) FROM apps WHERE bundle_identifier LIKE 'com.apple.%';
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Detect Linux hosts with high severity vulnerable versions of OpenSSL
|
||||
platforms: Linux
|
||||
description: Retrieves the OpenSSL version.
|
||||
query: SELECT name AS name, version AS version, 'deb_packages' AS source FROM deb_packages WHERE name LIKE 'openssl%' UNION SELECT name AS name, version AS version, 'apt_sources' AS source FROM apt_sources WHERE name LIKE 'openssl%' UNION SELECT name AS name, version AS version, 'rpm_packages' AS source FROM rpm_packages WHERE name LIKE 'openssl%';
|
||||
purpose: Detection
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Detect machines with Gatekeeper disabled
|
||||
platforms: macOS
|
||||
description: Gatekeeper tries to ensure only trusted software is run on a mac machine.
|
||||
query: SELECT * FROM gatekeeper WHERE assessments_enabled = 0;
|
||||
purpose: Detection
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Detect presence of authorized SSH keys
|
||||
platforms: macOS, Linux
|
||||
description: Presence of authorized SSH keys may be unusual on laptops. Could be completely normal on servers, but may be worth auditing for unusual keys and/or changes.
|
||||
query: SELECT username, authorized_keys. * FROM users CROSS JOIN authorized_keys USING (uid);
|
||||
purpose: Detection
|
||||
remediation: Check out the linked table (https://github.com/fleetdm/fleet/blob/32b4d53e7f1428ce43b0f9fa52838cbe7b413eed/handbook/queries/detect-hosts-with-high-severity-vulnerable-versions-of-openssl.md#table-of-vulnerable-openssl-versions) to determine if the installed version is a high severity vulnerability and view the corresponding CVE(s)
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get authorized keys
|
||||
platforms: macOS, Linux
|
||||
description: List authorized_keys for each user on the system.
|
||||
query: SELECT * FROM users CROSS JOIN authorized_keys USING (uid);
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get crashes
|
||||
platforms: macOS
|
||||
description: Retrieve application, system, and mobile app crash logs.
|
||||
query: SELECT uid, datetime, responsible, exception_type, identifier, version, crash_path FROM users CROSS JOIN crashes USING (uid);
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed Chrome Extensions
|
||||
platforms: macOS, Linux, Windows, FreeBSD
|
||||
description: List installed Chrome Extensions for all users.
|
||||
query: SELECT uid, datetime, responsible, exception_type, identifier, version, crash_path FROM users CROSS JOIN crashes USING (uid);
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed FreeBSD software
|
||||
platforms: FreeBSD
|
||||
description: Get all software installed on a FreeBSD computer, including browser plugins and installed packages. Note, this does not included other running processes in the processes table.
|
||||
query: SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Package (pkg)' AS type, 'pkg_packages' AS source FROM pkg_packages;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get Homebrew Packages
|
||||
platforms: macOS
|
||||
description: Get the installed homebrew package database.
|
||||
query: SELECT * FROM homebrew_packages;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed Linux software
|
||||
platforms: Linux
|
||||
description: Get all software installed on a Linux computer, including browser plugins and installed packages. Note, this does not included other running processes in the processes table.
|
||||
query: SELECT name AS name, version AS version, 'Package (APT)' AS type, 'apt_sources' AS source FROM apt_sources UNION SELECT name AS name, version AS version, 'Package (deb)' AS type, 'deb_packages' AS source FROM deb_packages UNION SELECT package AS name, version AS version, 'Package (Portage)' AS type, 'portage_packages' AS source FROM portage_packages UNION SELECT name AS name, version AS version, 'Package (RPM)' AS type, 'rpm_packages' AS source FROM rpm_packages UNION SELECT name AS name, '' AS version, 'Package (YUM)' AS type, 'yum_sources' AS source FROM yum_sources UNION SELECT name AS name, version AS version, 'Package (NPM)' AS type, 'npm_packages' AS source FROM npm_packages UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed macOS software
|
||||
platforms: macOS
|
||||
description: Get all software installed on a macOS computer, including apps, browser plugins, and installed packages. Note, this does not included other running processes in the processes table.
|
||||
query: SELECT name AS name, bundle_short_version AS version, 'Application (macOS)' AS type, 'apps' AS source FROM apps UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name As name, version AS version, 'Browser plugin (Safari)' AS type, 'safari_extensions' AS source FROM safari_extensions UNION SELECT name AS name, version AS version, 'Package (Homebrew)' AS type, 'homebrew_packages' AS source FROM homebrew_packages;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed Safari extensions
|
||||
platforms: macOS
|
||||
description: Retrieves the list of installed Safari Extensions for all users in the target system.
|
||||
query: SELECT safari_extensions.* FROM users join safari_extensions USING (uid);
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get installed Windows software
|
||||
platforms: Windows
|
||||
description: Get all software installed on a Windows computer, including programs, browser plugins, and installed packages. Note, this does not included other running processes in the processes table.
|
||||
query: SELECT name AS name, version AS version, 'Program (Windows)' AS type, 'programs' AS source FROM programs UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages UNION SELECT name AS name, version AS version, 'Browser plugin (IE)' AS type, 'ie_extensions' AS source FROM ie_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Chrome)' AS type, 'chrome_extensions' AS source FROM chrome_extensions UNION SELECT name AS name, version AS version, 'Browser plugin (Firefox)' AS type, 'firefox_addons' AS source FROM firefox_addons UNION SELECT name AS name, version AS version, 'Package (Chocolatey)' AS type, 'chocolatey_packages' AS source FROM chocolatey_packages UNION SELECT name AS name, version AS version, 'Package (Atom)' AS type, 'atom_packages' AS source FROM atom_packages UNION SELECT name AS name, version AS version, 'Package (Python)' AS type, 'python_packages' AS source FROM python_packages;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get laptops with failing batteries
|
||||
platforms: macOS
|
||||
description:
|
||||
query: SELECT * FROM battery WHERE health != 'Good' AND condition NOT IN ('', 'Normal');
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get macOS disk free space percentage
|
||||
platforms: macOS
|
||||
description: Displays the percentage of free space available on the primary disk partition.
|
||||
query: SELECT (blocks_available * 100 / blocks) AS pct, * FROM mounts WHERE path = '/';
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get macOS disk free space percentage
|
||||
platforms: macOS
|
||||
description: Displays the percentage of free space available on the primary disk partition.
|
||||
query: SELECT (blocks_available * 100 / blocks) AS pct, * FROM mounts WHERE path = '/';
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get mounts
|
||||
platforms: macOS, Linux
|
||||
description: Shows system mounted devices and filesystems (not process specific).
|
||||
query: SELECT device, device_alias, path, type, blocks_size FROM mounts;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get the version of the resident operating system
|
||||
platforms: macOS, Linux, Windows, FreeBSD
|
||||
description: Shows system mounted devices and filesystems (not process specific).
|
||||
query: SELECT * FROM os_version;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get platform info
|
||||
platforms: macOS
|
||||
description: Shows information about the host platform
|
||||
query: SELECT vendor, version, date, revision from platform_info;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get startup items
|
||||
platforms: macOS, Linux, Windows, FreeBSD
|
||||
description: Shows applications and binaries set as user/login startup items.
|
||||
query: SELECT * FROM startup_items;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get system logins and logouts
|
||||
platforms: macOS
|
||||
description: Get a list of system logins and logouts.
|
||||
query: SELECT * FROM last;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get system uptime
|
||||
platforms: macOS, Linux, Windows, FreeBSD
|
||||
description: Shows the system uptime.
|
||||
query: SELECT * FROM uptime;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get USB devices
|
||||
platforms: macOS, Linux
|
||||
description: Shows all USB devices that are actively plugged into the host system.
|
||||
query: SELECT * FROM usb_devices;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get wifi status
|
||||
platforms: macOS
|
||||
description: Shows information about the wifi network that a host is currently connected to.
|
||||
query: SELECT * FROM wifi_status;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: query
|
||||
spec:
|
||||
name: Get Windows machines with unencrypted hard disks
|
||||
platforms: Windows
|
||||
description:
|
||||
query: SELECT * FROM bitlocker_info WHERE protection_status = 0;
|
||||
purpose: Informational
|
||||
remediation: N/A
|
||||
@@ -447,14 +447,14 @@ We will use this address when we configure the Kubernetes deployment and databas
|
||||
The last step is to run the Fleet database migrations on your new MySQL server. To do this, run the following:
|
||||
|
||||
```
|
||||
kubectl create -f ./examples/kubernetes/fleet-migrations.yml
|
||||
kubectl create -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml
|
||||
```
|
||||
|
||||
In Kubernetes, you can only run a job once. If you'd like to run it again (i.e.: you'd like to run the migrations again using the same file), you must delete the job before re-creating it. To delete the job and re-run it, you can run the following commands:
|
||||
|
||||
```
|
||||
kubectl delete -f ./examples/kubernetes/fleet-migrations.yml
|
||||
kubectl create -f ./examples/kubernetes/fleet-migrations.yml
|
||||
kubectl delete -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml
|
||||
kubectl create -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-migrations.yml
|
||||
```
|
||||
|
||||
#### Redis
|
||||
@@ -523,7 +523,7 @@ kubectl create secret generic fleet-server-auth-key --from-file=./build/fleet-se
|
||||
First we must deploy the instances of the Fleet webserver. The Fleet webserver is described using a Kubernetes deployment object. To create this deployment, run the following:
|
||||
|
||||
```
|
||||
kubectl apply -f ./examples/kubernetes/fleet-deployment.yml
|
||||
kubectl apply -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-deployment.yml
|
||||
```
|
||||
|
||||
You should be able to get an instance of the webserver running via `kubectl get pods` and you should see the following logs:
|
||||
@@ -539,7 +539,7 @@ ts=2017-11-16T02:48:38.441148166Z transport=https address=0.0.0.0:443 msg=listen
|
||||
Now that the Fleet server is running on our cluster, we have to expose the Fleet webservers to the internet via a load balancer. To create a Kubernetes `Service` of type `LoadBalancer`, run the following:
|
||||
|
||||
```
|
||||
kubectl apply -f ./examples/kubernetes/fleet-service.yml
|
||||
kubectl apply -f ./docs/1-Using-Fleet/configuration-files/kubernetes/fleet-service.yml
|
||||
```
|
||||
|
||||
#### Configure DNS
|
||||
|
||||
@@ -30,13 +30,6 @@ Fleet's standard query library includes a growing collection of useful queries f
|
||||
- [Get system uptime](./get-system-uptime.md) (macOS, Linux, Windows, FreeBSD)
|
||||
- [Get crashes](./get-crashes.md) (macOS)
|
||||
|
||||
|
||||
### Contributors
|
||||
|
||||
Want to add your own query?
|
||||
|
||||
Please submit a pull request using the [Fleet standard query library template](./standard-query-library-template.md).
|
||||
|
||||
### Additional resources
|
||||
|
||||
Listed below are great resources that contain additional queries.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Get installed Safari Extensions
|
||||
|
||||
Retreives the list of installed Safari Extensions for all users in the target system.
|
||||
Retrieves the list of installed Safari Extensions for all users in the target system.
|
||||
|
||||
### Support
|
||||
macOS
|
||||
|
||||
Reference in New Issue
Block a user