Commit Graph
200 Commits
Author SHA1 Message Date
Lucas Manuel Rodriguez f4d3159cc9 Fleetctl to package .app bundles for osquery (and changes for orbit to support them) (#4393)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Trigger workflow on PR

* Fixes to ease handling of artifact

* Fix, do not use target name as dir

* Remove workaround
2022-03-15 16:04:12 -03:00
Noah Talerman 937cd7f2eb Update changelog and bump versioning (#4497) 2022-03-07 11:08:51 -06:00
0af5e161d0 Remove numbers from documentation filenames in Fleet repo (#4313)
* Renaming files and a lot of find and replace

* pageRank meta tags, sorting by page rank

* reranking

* removing numbers

* revert changing links that are locked to a commit

* update metatag name, uncomment github contributers

* Update basic-documentation.page.js

* revert link change

* more explicit errors, change pageOrderInSection numbers, updated sort

* Update build-static-content.js

* update comment

* update handbook link

* handbook entry

* update sort

* update changelog doc links to use fleetdm.com

* move standard query library back to old location, update links/references to location

* revert unintentional link changes

* Update handbook/community.md

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>

Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
2022-02-23 12:17:55 -06:00
Lucas Manuel Rodriguez b5be858071 Fix update checks for orbit at startup (#3835)
* Fix update checks for orbit at startup

* Add tests

* Add scripts for testing local TUF server

* Remove -x used for debugging
2022-02-23 14:58:07 -03:00
Lucas Manuel Rodriguez e39ba93a90 Orbit to wait for osqueryd extension manager socket to be ready (#3836)
* Wait for osquery extension manager socket to be ready

* PR review feedback

* Add WIP Github Action to test orbit

* Set fleet address on fleetctl

* Add logging for troubleshooting

* Add prepare db statement

* Use tls-skip-verify on fleetctl

* Move steps around

* Fix addresses

* Fix fleetctl path

* Add certs.pem to orbit root dir

* Add orbit logs

* Increase timeout

* Add proper orbit log paths

* Fix tmp path orbit

* Add get hosts command to troubleshoot

* Fix orbit job termination

* Add comments to workflows

* Wait for server to go down

* Add orbit Windows job

* Use bash on windws-latest

* Fix missing quote

* Run orbit on Ubuntu

* Bump host count

* Increase timeout for extensions and not terminate on err

* Add comment to clarify high timeout value

* Revert change, we do want to exit in case of err
2022-02-22 15:05:32 -03:00
RachelElysia 7cdc7bbec2 E2e Test: Team admin flow (#4301) 2022-02-21 21:09:36 -05:00
Benjamin Edwards 12eac152c4 GCP Terraform (#4303)
* gcp wip
* more edits on services, secrets manager, readme
* updated readme with required variables
2022-02-18 20:01:42 -05:00
Zachary Winnerman 899a643c15 Elastic is now managed through ansible for quicker config changes and more stability (#4243)
* Add elasticsearch being managed through ansible for better managability

* Testing

* testing

* testing

* testing

* testing

* testing

* testing

* fixup

* fixup

* Added docker stuff

* fixup

* fixup

* fixup

* Remove old code

* fixup

* fix health checks

* Add elastic apm configuration

* fixup

* fixup

* fixup

* testing

* restart always

* fixup

* fixup
2022-02-18 11:07:32 -05:00
Zachary Winnerman b72e9bd1c1 Split elasticsearch's security groups (#4231) 2022-02-16 11:11:39 -05:00
Zachary Winnerman 69c0434e88 Add terraform for the loadtesting environment (#4172)
* Add terraform for the loadtesting environment

* Add some checkov stuff and minor updates

* Remove defunct commented stuff

* Add separate cloudwatch namespace for migrations

* Remove defunct code

* checkin
2022-02-15 13:00:24 -05:00
dependabot[bot] 8589cdd553 Bump follow-redirects from 1.14.4 to 1.14.8 in /tools/fleetctl-npm (#4183)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.14.4 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.14.4...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-15 10:08:07 -06:00
Noah TalermanandZach Wasserman 67827474c2 Prepare for Fleet 4.10.0 (#4161)
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-02-13 18:13:06 -08:00
eashaw 9af92b23aa Update links to documentation (#4163)
* update links to documentation

* revert .sailsrc change

* fix broken link
2022-02-10 19:26:18 -06:00
Zach Wasserman 600549d8a4 Prepare for 4.9.1 release (#3996) (#4005)
Merging back to `main` from patch branch.
2022-02-10 10:46:21 -05:00
Luke Heath 5866d68f13 Set max length for name inputs (#4063) 2022-02-08 19:40:38 -06:00
Benjamin Edwards d4312bec59 remove broken terraform references (#3856) 2022-01-24 15:58:36 -05:00
35f806f7d1 Merge Release candidate 4.9.0 into main (#3844)
* Better jitter (#3716) (#3744)

* Better jitter (#3716)

* Better jitter

* Fix lint

* Use milliseconds

* Make duration milliseconds

* Update based on Lucas' suggestion

* Add changes file

* Panic on error

* Fix compilation error

* Use time.after instead of time.tick to not leak

* Include browser extensions in software inventory (#3733) (#3757)

Use appropriate JOINs against users table to include all results.

For #3557

* Build Docker images on push to any branch (#3756) (#3758)

Allows for testing release branches and any others.

* Optimize users detail query (#3754) (#3770)

@Smjert reported instances of Windows Domain Controllers having massive
resource utilization and being killed by the watchdog when running this
query. In his test environment, this new query performs much better.

* Refactor async host processing to avoid redis SCAN keys (for policies) (#3657) (#3755)

* Issue 3707 clean targets on delete (#3739) (#3776)

* wip

* Delete targets when deleting teams, hosts, and labels

* Add changes file

* Fix error message

* Remove unused teamsTable

* Cleanup new pack

* Clean new packs at end of test

* Update operating system policy (#3779)

- Update policy's query to check for operating system versions greater than or equal to

* Improve loading of manage policies page (#3695) (#3781)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Fix logo (#3765)

* fixed duplicate query modal not closing (#3787)

* fixed style for connection line on activity feed (#3789)

* Improve loading state for query platform compatibility (#3752) (#3783)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* Disable pagination for teams page (#3794)

* Shorten "simple" query API period to 25s (#3775) (#3804)

This helps the period stay under the default request timeouts for most
load balancers.

Some default timeouts:
* AWS ALB - 60s
* Nginx - 60s
* GCP LB - 30s

* Clear user checkbox selections after successfully removing selected row from table UI #3798 (#3801)

* Alias fleetctl sandbox to fleetctl preview (#3803) (#3806)

Preparing for some work the product team is planning to rebrand preview
to sandbox.

* Remove flash of spinner to no spinner (#3799)

* Revert "Better jitter (#3716) (#3744)" (#3807)

This reverts commit c857b52d2b373ed9e302d643e6d2212af7353408.

* Fix teams dropdown sizing (#3759) (#3810)

Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>

* fixed link for expiration message (#3788)

* Fixed empty string for compatible platforms (#3812)

* fixed empty string for compatible platforms

* default checked for platforms on policy query

* Block autocomplete for username and password in SMTP settings (#3732) (#3816)

* No hosts for software/policy renders better message (#3701) (#3821)

Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>

* Get failing policies webhook data on page load (#3824)

* Fix reset password flow (#3826)

* Prepare for 4.9.0 (#3820)

- Add CHANGELOG
- Bump versioning
- Remove changes files
- Spelling fix for operating system policy's description

* Remove merge artifact

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: gillespi314 <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Martavis Parker <47053705+martavis@users.noreply.github.com>
2022-01-24 15:32:25 -03:00
Benjamin Edwards d650423be0 Reference Architectures (#3712)
Added reference architectures using https://docs.gitlab.com/ee/administration/reference_architectures/ as inspiration.

- updated terraform based on some feedback of usage
- pinned fleet docker version in terraform as to no get unexpected upgrades when applying
- updated some documentation around apply migration tasks
2022-01-21 16:27:55 -08:00
Tomas Touceda 43b78e6d0d Add redis stress (#3363)
* Add redis stress tool

* Parametrize more things

* Rename folder back

* Log error

* Fix lint
2022-01-20 16:18:17 -03:00
Benjamin Edwards 6b551acf95 add percona terraform (#3468) 2022-01-06 18:07:40 -05:00
Noah Talerman 09654d77ee Prepare for Fleet 4.8.0 (#3536) 2021-12-31 09:58:08 -08:00
Noah Talerman 8957f00d86 Prepare for Fleet 4.7.0 (#3355) 2021-12-14 11:25:13 -08:00
Zach Wasserman a6d60cc193 Configure nofiles ulimit in Terraform ECS task (#3249)
The low default ulimit `nofiles` value (`4096`) in Fargate was observed
to cause errors when running with a large number of hosts and a small
number of servers. Each server should be able to server more than 4096
simultaneous clients.
2021-12-08 15:08:48 -08:00
Tomas Touceda 142006cbdd Issue 2099 enable vulnerability processing new installs (#3104)
* Enable vulnerability processing by default

* Add changes file

* Remove wrong env var

* Fix import

* Fix tests

* Update log line

* Remove unneeded import
2021-12-03 14:44:57 -03:00
Tomas ToucedaandNoah Talerman 052e41851b Prepare for Fleet 4.6.2 (#3146) (#3179)
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2021-12-02 16:56:56 -03:00
Martin Angers c997f853e5 Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Zach Wasserman 2b2e9f9f44 Prepare for 4.6.1 release (#3067) 2021-11-21 21:23:25 -08:00
Noah Talerman cca1e2e043 Update "Adding hosts" documentation to emphasize osquery installers (#2956)
- Emphasize generating an "osquery installer" (also referred as "Orbit") using the `fleetctl package` command
- Add instructions for adding multiple hosts and automatically adding hosts to a team
- Remove instructions for generating an enrollment package with the tooling in `tools/mac/`
- Remove the `tools/mac/` directory from the repository because it is no longer used or referenced in any documentation
- Update "Automatically adding hosts to a team" section of "Teams" documentation to point to "Adding hosts" documentation
- Add instructions for migrating from plain osquery to Fleet's osquery installers
2021-11-19 09:38:01 -05:00
Noah Talerman a6e9d551fd Prepare for Fleet 4.6.0 (#3018) 2021-11-18 17:46:57 -08:00
Zach Wasserman 1bd50d8965 Prepare for 4.5.1 release (#2913) 2021-11-12 17:03:19 -08:00
Zach Wasserman e1f273fdc0 Add multiple users for SSO testing (#2885) 2021-11-10 15:15:41 -08:00
Benjamin Edwards dac043f1f5 refactor terraform to allow bootstrapping (#2662)
* refactor to allow bootstrapping* move monitoring into its own package, update readme
* add variable for license_key
* replication lag alarm less sensitive
2021-11-09 20:14:05 -05:00
Lucas Manuel Rodriguez 660796bf72 Add documentation for running live queries with GET request (#2755) 2021-11-03 14:35:17 -03:00
Noah Talerman 249927f750 Prepare for Fleet 4.5.0 release (#2741) 2021-11-01 11:39:55 -04:00
Zach Wasserman 8dc34be3a3 Updates and fixes for packaging (#2682)
- Fix Windows MSI generation by changing permissions (#2655).
- Refactor temp directory initialization.
- Use root user for Wine in WiX Docker container.
- Support .pkg packaging on Linux without dependencies (besides Docker)
2021-10-27 16:17:41 -07:00
Zach Wasserman 32bee9f975 Update fleetctl NPM dependencies (#2658)
These dependencies had vulnerabilities that caused alerting on `npm
install` and in GitHub.

The vulnerabilities did not effect the use in our context.
2021-10-24 19:24:44 -07:00
Benjamin Edwards 0fb6416d45 Add Cloudwatch monitoring to AWS Terraform configs (#2485)
* add support for minio backend file carving
* add changes file
* rds alarm and sns topic
* added cloudwatch alarm documenation
* Update docs/01-Using-Fleet/06-Monitoring-Fleet.md
* update aws provider version to fix bug in ecs container insights, add more redis alerts
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-22 15:38:00 -04:00
Zach Wasserman 03b81245ee Update files for 4.4.3 release (#2636) 2021-10-22 09:41:50 -07:00
Lucas Manuel Rodriguez 91ae5ba870 By default, force password reset on CreateUser (#2570)
* By default, force password reset on CreateUser

* Use ptr.Bool and add test

* Add changes file

* Simple fix for e2e tests

* Add API changes
2021-10-21 15:00:38 -03:00
Zach Wasserman 42c7933b22 Add WiX Dockerfile and update image name (#2548)
Use a different base image and newer version of Wine to try to mitigate
crashes experienced by users in #2527.
2021-10-20 15:30:46 -07:00
Martin Angers 07cf136d03 Add TLS support to Redis connections (#2568) 2021-10-20 10:09:18 -04:00
Martin Angers eda6d60be8 Add replicas to the redis docker-compose setup. (#2511) 2021-10-18 08:33:26 -04:00
Zach Wasserman a6e8f22d83 Update changelog and configs for patch releases (#2531) 2021-10-14 18:17:47 -07:00
Tomas Touceda d904d501ac Add infra for loadtest (#2218)
* Add infra for loadtest

* Move loadtest stuff to a new file and parametrize fleet min/max capacity

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* Update to be ready for review

* Update link and other variables needed

* Address review comments and update links
2021-10-14 12:04:27 -03:00
Benjamin Edwards 16c5823692 add support for minio backend file carving (#2448)
* add support for minio backend file carving
* add changes file
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-10-12 15:32:06 -04:00
Luke Heath 94a10cb6b9 Update README.md 2021-10-12 10:02:31 -05:00
RachelElysia 981028705e Sentence Case: Throughout UI, tests, validators, docs (#2455) 2021-10-11 10:58:27 -04:00
Zach Wasserman e61dc765fd Prepare for 4.4.0 release (#2423) 2021-10-06 14:14:12 -07:00
Zach Wasserman 980a853e3c Update changelog for 4.3.2 release (#2276) 2021-09-30 15:23:22 -07:00
Benjamin Edwards 1cc68eea3f Feature/infra updates (#2183)
* complete terraform state migration

* split firehose results & status streams

* extract more variables, with sane defaults

* fix fargate configs
2021-09-30 16:22:34 -04:00