diff --git a/changes/fix-broken-win-10-cis-queries b/changes/fix-broken-win-10-cis-queries new file mode 100644 index 0000000000..7904952c36 --- /dev/null +++ b/changes/fix-broken-win-10-cis-queries @@ -0,0 +1 @@ +* Fix typos in policy queries in the Windows CIS policies yaml (`ee/cis/win-10/cis-policy-queries.yml`). diff --git a/ee/cis/win-10/cis-policy-queries.yml b/ee/cis/win-10/cis-policy-queries.yml index 680ac5b70c..45c9ef15c8 100644 --- a/ee/cis/win-10/cis-policy-queries.yml +++ b/ee/cis/win-10/cis-policy-queries.yml @@ -5589,7 +5589,7 @@ spec: 'Computer Configuration\Policies\Administrative Templates\System\Group Policy\Turn off background refresh of Group Policy' Note: This Group Policy path may not exist by default. It is provided by the Group Policy template GroupPolicy.admx/adml that is included with the Microsoft Windows 8.0 & Server 2012 (non-R2) Administrative Templates (or newer). query: | - # The registry key is not present when policy is disabled, so query below is returning 1 when policy is disabled and registry value does not exist. It also return 1 in case policy is enabled and its registry value is 1 + -- The registry key is not present when policy is disabled, so query below is returning 1 when policy is disabled and registry value does not exist. It also return 1 in case policy is enabled and its registry value is 1 SELECT 1 WHERE ( NOT EXISTS ( SELECT 1 FROM registry WHERE key = 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System' AND name = 'DisableBkGndGroupPolicy' ) ) OR ( @@ -8030,7 +8030,7 @@ spec: 'Computer Configuration\Policies\Administrative Templates\Windows Components\Microsoft Defender Antivirus\MAPS\Join Microsoft MAPS' Note: This Group Policy path may not exist by default. It is provided by the Group Policy template WindowsDefender.admx/adml that is included with the Microsoft Windows 8.1 & Server 2012 R2 Administrative Templates (or newer). query: | - # The registry key is not present when policy is disabled, so query below is returning 1 when policy is disabled and registry value does not exist. It also return 1 in case policy is enabled and its registry value is 1 or 2 + -- The registry key is not present when policy is disabled, so query below is returning 1 when policy is disabled and registry value does not exist. It also return 1 in case policy is enabled and its registry value is 1 or 2 SELECT 1 WHERE ( NOT EXISTS ( SELECT 1 FROM registry WHERE key = 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies\\Microsoft\\Windows Defender\\Spynet' COLLATE NOCASE AND name = 'SpynetReporting' ) ) OR ( @@ -8653,7 +8653,6 @@ spec: To establish the recommended configuration via GP, set the following UI path to 'Disabled': 'Computer Configuration\Policies\Administrative Templates\Windows Components\Search\Allow indexing of encrypted files' query: | - TODO SELECT 1 FROM registry WHERE (path = 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies\\Microsoft\\Windows\\Windows Search\\AllowIndexingEncryptedStoresOrItems' AND data = 0); purpose: Informational tags: compliance, CIS, CIS_Level1, CIS_win10_enterprise_1.12.0, CIS_bullet_18.9.67.5 diff --git a/tools/loadtest/osquery/README.md b/tools/loadtest/osquery/README.md new file mode 100644 index 0000000000..bd6695169e --- /dev/null +++ b/tools/loadtest/osquery/README.md @@ -0,0 +1,150 @@ +# Load test of osquery queries in macOS and Windows + +Following are the steps to load test osquery on macOS and Windows. +The purpose is to know the impact of Fleet provided queries on real devices. + +> The scripts that process osquery logs were written and tested on macOS. + +> At the time of writing, the changes that add watchdog logging needed for this script are +> merged but not released yet (https://github.com/osquery/osquery/pull/8070). +> You will have to download and extract the osqueryd executable from the PR: https://github.com/osquery/osquery/suites/14033523376/artifacts/783724086 + +## Requirements + +- Install gnuplot and ripgrep, e.g. on macOS: +```sh +brew install gnuplot ripgrep +``` + +## Architecture + +We are going to use the [fleetd tables](../../../orbit/cmd/fleetd_tables/README.md) as an extension so that it is also monitored by the watchdog. + +```mermaid +graph LR; + subgraph Device + osquery_watchdog[osqueryd\nwatchdog process]; + osqueryd_worker[osqueryd\nworker process]; + fleetd_tables[fleetd_tables\nextension process]; + + osquery_watchdog -- monitors\nCPU and memory --> osqueryd_worker; + osquery_watchdog -- monitors\nCPU and memory --> fleetd_tables; + end +``` + +## macOS + +### Build fleetd_tables extension + +```sh +make fleetd-tables-darwin-universal +sudo cp fleetd_tables_darwin_universal.ext /usr/local/osquery_extensions/fleetd_tables.ext +echo "/usr/local/osquery_extensions/fleetd_tables.ext" > /tmp/extensions.load +``` + +### Run osquery + +> The following assumes a Fleet server instance running and listening at `localhost:8080`. + +```sh +mkdir -p /Users/luk/osqueryd/osquery_log +``` + +```sh +sudo ENROLL_SECRET=<...> ./osquery/osqueryd \ + --verbose=true \ + --tls_dump=true \ + --pidfile=/Users/luk/osqueryd/osquery.pid \ + --database_path=/Users/luk/osqueryd/osquery.db \ + --logger_path=/Users/luk/osqueryd/osquery_log \ + --host_identifier=instance \ + # /Users/luk/fleetdm/git/fleet is the location of the Fleet mono repository. + --tls_server_certs=/Users/luk/fleetdm/git/fleet/tools/osquery/fleet.crt \ + --enroll_secret_env=ENROLL_SECRET \ + --tls_hostname=localhost:8080 \ + --enroll_tls_endpoint=/api/v1/osquery/enroll \ + --config_plugin=tls \ + --config_tls_endpoint=/api/v1/osquery/config \ + --config_refresh=60 \ + --disable_distributed=false \ + --distributed_plugin=tls \ + --distributed_tls_max_attempts=10 \ + --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \ + --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \ + --logger_plugin=tls,filesystem \ + --logger_tls_endpoint=/api/v1/osquery/log \ + --disable_carver=false \ + --carver_disable_function=false \ + --carver_start_endpoint=/api/v1/osquery/carve/begin \ + --carver_continue_endpoint=/api/v1/osquery/carve/block \ + --carver_block_size=2000000 \ + --extensions_autoload=/tmp/extensions.load \ + --allow_unsafe \ + --enable_watchdog_debug \ + --distributed_denylist_duration 0 \ + --enable_extensions_watchdog 2>&1 | tee /tmp/osqueryd.log +``` + +## Windows + +### Build fleetd_tables extension + +In a macOS device run: +```sh +make fleetd-tables-windows +``` +Choose path to store the extension in the Windows device, on this guide we will use `C:\Program Files\Fleetd\`. + +- Place the generated `fleetd_tables_windows.exe` on the chosen location, on this guide it would be `C:\Program Files\Fleetd\fleetd_tables_windows.exe`. +- Create a text file `C:\Program Files\Fleetd\extensions.load` with the following line in it: `C:\Program Files\Fleetd\fleetd_tables_windows.exe` (the location of the extension). + +### Run osquery + +> The following assumes a Fleet server instance running and listening at `localhost:8080`. + +Create the following directories: +```sh +mkdir C:\Users\Lucas Rodriguez\Downloads\osqueryd\local +mkdir C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\osqueryd_log +``` + +Copy the Fleet test certificate (`./tools/osquery/fleet.crt`) into a known path: +```sh +C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\fleet.crt +``` + +```sh +set ENROLL_SECRET=<...> + +osqueryd.exe --verbose=true --tls_dump=true --pidfile="C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\osquery.pid" --database_path="C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\osquery.db" --logger_path="C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\osqueryd_log" --host_identifier=instance --tls_server_certs="C:\Users\Lucas Rodriguez\Downloads\osqueryd\local\fleet.crt" --enroll_secret_env=ENROLL_SECRET --tls_hostname=host.docker.internal:8080 --enroll_tls_endpoint=/api/v1/osquery/enroll --config_plugin=tls --config_tls_endpoint=/api/v1/osquery/config --config_refresh=60 --disable_distributed=false --distributed_plugin=tls --distributed_tls_max_attempts=10 --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write --logger_plugin=tls --logger_tls_endpoint=/api/v1/osquery/log --disable_carver=false --carver_disable_function=false --carver_start_endpoint=/api/v1/osquery/carve/begin --carver_continue_endpoint=/api/v1/osquery/carve/block --carver_block_size=2000000 --extensions_autoload="C:\Program Files\Fleetd\extensions.load" --allow_unsafe --enable_watchdog_debug --distributed_denylist_duration 0 --enable_extensions_watchdog > osqueryd.log 2>&1 +``` + +## Log analysis + +The following (macOS) commands and scripts can be used to analyze the load in the device (as monitored by the watchdog process). + +### Watchdog process kills + +Run the following commands to check if watchdog trigger a worker kill: +```sh +rg "utilization limit" /tmp/osqueryd.log +rg "Memory limit" /tmp/osqueryd.log +``` +If the above commands return no output then the load on the device was below the limits configured by osquery. + +### Render CPU and memory usage + +The following script renders the CPU and memory utilization throughout the load test: + +On macOS (while osqueryd is running): +```sh +./tools/loadtest/osquery/gnuplot_osqueryd_cpu_memory.sh +``` + +For Windows, first, locate the `osqueryd.log` generated by `osqueryd.exe` and place it in the macOS host in `/tmp/osqueryd.log`. +Then, grab the osquery worker pid and run the following: +```sh +OSQUERYD_PID=7732 ./tools/loadtest/osquery/gnuplot_osqueryd_cpu_memory.sh +``` + +> The horizontal red line is the configured CPU usage limit (hardcoded to `1200ms` in the `gnuplot_osqueryd_cpu_memory.sh`) diff --git a/tools/loadtest/osquery/macos/gnuplot_osqueryd_cpu_memory.sh b/tools/loadtest/osquery/gnuplot_osqueryd_cpu_memory.sh similarity index 90% rename from tools/loadtest/osquery/macos/gnuplot_osqueryd_cpu_memory.sh rename to tools/loadtest/osquery/gnuplot_osqueryd_cpu_memory.sh index d44902771e..2da8c151e0 100755 --- a/tools/loadtest/osquery/macos/gnuplot_osqueryd_cpu_memory.sh +++ b/tools/loadtest/osquery/gnuplot_osqueryd_cpu_memory.sh @@ -7,7 +7,11 @@ set -e # - gnuplot # Get PID of the osquery worker process. -osquery_pid=$(ps aux | grep -E "osqueryd\s*$" | awk {'print $2'}) +if [ -n "$OSQUERYD_PID" ]; then + osquery_pid=$OSQUERYD_PID +else + osquery_pid=$(ps aux | grep -E "osqueryd\s*$" | awk {'print $2'}) +fi # Extract CPU and memory data points from logs. rg " (\d\d:\d\d:\d\d).* pid: $osquery_pid, cpu: (\d+)ms/\d+ms, memory: ([\d.]+)" -or '$1 $2 $3' /tmp/osqueryd.log > /tmp/osqueryd.dat diff --git a/tools/loadtest/osquery/macos/README.md b/tools/loadtest/osquery/macos/README.md deleted file mode 100644 index 8324dcdb40..0000000000 --- a/tools/loadtest/osquery/macos/README.md +++ /dev/null @@ -1,87 +0,0 @@ -# Load test of osquery queries in macOS - -Following are the steps to load test osquery on macOS. -The purpose is to know the impact of Fleet provided queries on real devices. - -> At the time of writing, the changes that add watchdog logging needed for this script are -> merged but not released yet (https://github.com/osquery/osquery/pull/8070). -> You will have to download and extract the osqueryd executable from the PR: https://github.com/osquery/osquery/suites/14033523376/artifacts/783724086 - -## Requirements - -- Install gnuplot and ripgrep: -```sh -brew install gnuplot ripgrep -``` -- Tooling to build osqueryd from source (at the time of writing this is needed), see https://osquery.readthedocs.io/en/stable/development/building/. - -## Build fleetd_tables - -We are going to use the fleetd tables as an extension so that it is also monitored by the watchdog. - -```sh -make fleetd-tables-darwin-universal -sudo cp fleetd_tables_darwin_universal.ext /usr/local/osquery_extensions/fleetd_tables.ext -echo "/usr/local/osquery_extensions/fleetd_tables.ext" > /tmp/extensions.load -``` - -## Run osquery - -> The following assumes a Fleet server instance running and listening at `localhost:8080`. - -```sh -mkdir -p /Users/luk/osqueryd/osquery_log -``` - -```sh -sudo ENROLL_SECRET=<...> ./osquery/osqueryd \ - --verbose=true \ - --tls_dump=true \ - --pidfile=/Users/luk/osqueryd/osquery.pid \ - --database_path=/Users/luk/osqueryd/osquery.db \ - --logger_path=/Users/luk/osqueryd/osquery_log \ - --host_identifier=instance \ - # /Users/luk/fleetdm/git/fleet is the location of the Fleet mono repository. - --tls_server_certs=/Users/luk/fleetdm/git/fleet/tools/osquery/fleet.crt \ - --enroll_secret_env=ENROLL_SECRET \ - --tls_hostname=localhost:8080 \ - --enroll_tls_endpoint=/api/v1/osquery/enroll \ - --config_plugin=tls \ - --config_tls_endpoint=/api/v1/osquery/config \ - --config_refresh=60 \ - --disable_distributed=false \ - --distributed_plugin=tls \ - --distributed_tls_max_attempts=10 \ - --distributed_tls_read_endpoint=/api/v1/osquery/distributed/read \ - --distributed_tls_write_endpoint=/api/v1/osquery/distributed/write \ - --logger_plugin=tls,filesystem \ - --logger_tls_endpoint=/api/v1/osquery/log \ - --disable_carver=false \ - --carver_disable_function=false \ - --carver_start_endpoint=/api/v1/osquery/carve/begin \ - --carver_continue_endpoint=/api/v1/osquery/carve/block \ - --carver_block_size=2000000 \ - --extensions_autoload=/tmp/extensions.load \ - --allow_unsafe \ - --enable_watchdog_debug \ - --distributed_denylist_duration 0 \ - --enable_extensions_watchdog 2>&1 | tee /tmp/osqueryd.log -``` - -## Check that the watchdog didn't trigger a worker kill - -The following commands should return no output: -```sh -rg "utilization limit" /tmp/osqueryd.log -rg "Memory limit" /tmp/osqueryd.log -``` - -## Render CPU and memory usage - -(Nice to have.) - -```sh -./tools/loadtest/osquery/macos/gnuplot_osqueryd_cpu_memory.sh -``` - -> The horizontal red line is the configured CPU usage limit (hardcoded to `1200ms` in the `gnuplot_osqueryd_cpu_memory.sh`)