Commit Graph
401 Commits
Author SHA1 Message Date
Roberto Dip 3a349bb07b fix data race in TestAutomationsSchedule (#7952) 2022-09-26 17:19:38 -03:00
Martin Angers 1fd6844bb0 Make osquery-perf aware of kubequery so that hosts are not reported as such erroneously (#7949) 2022-09-26 15:39:39 -04:00
gillespi314 639b85e47b Refactor cron worker to schedule package (#7886) 2022-09-23 11:45:48 -05:00
Martin Angers db24cf32fa Add new low_disk_space URL parameter to the GET /hosts endpoint (and GET /hosts/count) (#7853) 2022-09-21 15:16:31 -04:00
gillespi314 34688f531a Refactor webhooks cron to new schedule package (#7840) 2022-09-20 14:26:36 -05:00
Juan Fernandez f78dc8febd Use directories to organize msrc assets (#7851)
Use directories to organize MSRC assets
2022-09-20 14:12:19 -04:00
Martin Angers 478b4d3f69 Validate team and appconfig payloads, with dry-run and force modes (#7731) 2022-09-19 13:53:44 -04:00
gillespi314 6a3d9959fc Refactor vulnerabilities cron to scheduler package (#7650) 2022-09-16 10:08:51 -05:00
Juan Fernandez 765031d962 Bug 6717: Duplicate calls on vulnerability third party integrations. (#7698)
If the same vulnerability was present in more than one software, then duplicated third party integration calls were being issued.
2022-09-13 10:41:52 -04:00
Juan Fernandez b60d535d4a Feature 7084: Add new EE endpoint for Fleet Desktop (#7530)
Added new EE endpoint, that is meant to be used by Fleet Desktop only. The new endpoint will return the number of failed policies.
2022-09-12 15:37:38 -04:00
Juan Fernandez 98e4d6e0bc Feature 7394: Use MSRC parser to generate security bulletin artifacts (#7491)
Generate security artifacts using the MSRC parser.
2022-09-12 15:17:13 -04:00
Juan Fernandez b6d084deec Feature 7559: Include 3rd party severity scores in the vulnerability web-hook payload (#7581)
Premium users should receive vulnerability scores as part of the web-hook payload.
2022-09-06 15:12:17 -04:00
Michal Nicpon 0709d1bc5c improve vuln cpe matching on macos (#6985)
* add cpe translations
* fix matching on target_sw
2022-09-01 10:02:07 -06:00
gillespi314 8157359f84 Ingest Windows host display version for hosts table (#7403) 2022-08-30 10:16:48 -05:00
Roberto Dip eeefe2fab9 enable controlled rollout of features by teams (#7408) 2022-08-30 08:13:09 -03:00
Martin Angers f602ea3446 Provide Munki issues for MacOS hosts (#7280) 2022-08-29 14:40:16 -04:00
Roberto Dip f9bb0b3dc0 adjust osquery_perf to respect config settings (#7418)
users, software_macos and software_linux queries can be optionally disabled in the fleet server via the enable_host_users and enable_software_inventory config settings, however osquery_perf is always running them.

this adjusts the code accordingly to only execute them if the server asks.
2022-08-29 13:34:40 -03:00
Juan Fernandez 0572c621d5 Fix osquery-perf Ubuntu software path issue (#7417)
Fixed issue with software fixture path used in osquery-perf
2022-08-26 15:43:06 -04:00
Juan Fernandez 3048a07fd1 Feature 7076: Ingest installed windows updates (#7138)
* Ingest installed Windows updates and store them in the windows_updates table. 
* Added config option for enabling/disabling Windows update ingestion and Windows OS vuln. detection.
2022-08-26 14:55:03 -04:00
Roberto Dip 9a1d2066bb deprecate host_settings in favor of features (#7358)
Related to #7312, this makes use of the changes introduced in #7353 to rename host_settings to features while keeping backwards compatibility.
2022-08-25 13:41:50 -03:00
Martin Angers 1ed6512c5b Implement retries when building MSI on Apple M1 architecture (#7367) 2022-08-24 14:52:32 -04:00
Martin Angers dfd7d3f244 Add activity types for applied team spec and edited agent options (#7355) 2022-08-24 08:32:45 -04:00
Juan Fernandez 7e1e69c9bc Fixed flaky vulnerability CRON tests (#7311)
* Fixed flaky vuln cron tests

* Fixed typo

* Remove sub-test
2022-08-23 08:22:08 -03:00
gillespi314 7fb109e6b3 Handle errors for duplicate os records (#7294) 2022-08-22 14:34:00 -05:00
2103de275a improve error handling in vulnerabilities cron (#7276)
* improve error handling in vulnerabilities cron

* fix tests

* Use errHandler and go mod tidy

* Add dep that got removed by mod tidy

* add dsl to tools

* Add changes file

Co-authored-by: Michal Nicpon <michal@fleetdm.com>
Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-08-18 16:53:58 -03:00
gillespi314 90f57f4849 Enhance API endpoints with host operating systems info (#7154) 2022-08-12 14:23:25 -05:00
Zach Wasserman acf60189ff Add message for fleetctl when prepare commands are used (#7152) 2022-08-11 12:26:52 -07:00
Martin Angers c8cdddf0ea Update /macadmins endpoints to include MDM name and aggregate count (#7137) 2022-08-10 15:15:01 -04:00
Roberto Dip fc8c15c0d1 add application config setting to enable JIT provisioning (#7140)
As part of #7053, this adds a config setting to enable JIT provisioning.
2022-08-10 15:15:35 -03:00
gillespi314 e2194be61c Add schedule package and refactor cron jobs for cleanups, aggregations, and usage statistics (#6618) 2022-08-10 11:00:56 -05:00
gillespi314 3a88afaf98 Add new database tables and queries for host operating systems (#6920) 2022-08-09 13:34:41 -05:00
Lucas Manuel Rodriguez 6dcff28be0 Move specs parsing functionality to a new pkg/spec package (#7050) 2022-08-05 19:07:32 -03:00
Martin Angers 012869f971 Fleet Premium: de-anonymize usage stats (#7013) 2022-08-03 14:44:34 -04:00
Martin Angers c1d38598e2 Prevent removing team enroll secrets when applying team specs without new secrets (#6890) 2022-08-02 09:51:03 -04:00
Gabriel HernandezandLucas Rodriguez f61a17bd9b add google analytics to sandbox instances (#6941)
* add google analytics to sandbox instances

* Add serverType variable to frontend handler

* update version of html-webpack-plugin

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-08-01 11:27:12 +01:00
Roberto Dip 780db937de pass a pointer to config instead of the whole config to goroutines (#6823) 2022-07-26 12:27:30 -03:00
Roberto Dip d63f56f8c0 add support for notarization in fleetdm/fleetctl images (#6818)
#6674
2022-07-25 20:06:10 -03:00
gillespi314 cc34585a47 Add statistics for number of hosts that have not submitted results for distributed queries (#6495) 2022-07-20 20:54:10 -05:00
Michal Nicpon d4be5ad2a1 add upgrade tests (#6596)
* add upgrade tests

* fix lint issues

go.mod

* remove req.cnf

* revert unrelated changes

* make version configurable in test

* fix golangci-lint ruleguard issue

Related to https://github.com/go-critic/go-critic/issues/1152
Need to have github.com/quasilyte/go-ruleguard/dsl

* fix lint issues

* fix

* clean up docker-compose.yml

* fix http request

* add readme

* fix lint issues

* address feedback

* fix

* add platform

* address feedback

* run go fmt
2022-07-19 15:11:51 -06:00
Eng Zer Jun 1ab171faf3 test: use T.Setenv to set env vars in tests (#6714)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-18 14:22:28 -03:00
Roberto Dip 69f8f2a73b add API endpoints to retrieve pre-built installers (#6672)
Rel: #6365, this adds a new endpoint to check and download pre-built installers.
2022-07-18 13:44:30 -03:00
Roberto Dip 86366b2ed8 add a new S3 datastore to retrieve pre-built packages (#6631)
Related to #6365, this extends the datastore/s3 package to retrieve installers from S3 according to the conventions listed in the parent issue. This also includes:

- A minor refactor to decouple Carves-related functionality from the core S3 functionality
- Set-up to run tests using minio (only enabled via the FILE_STORAGE_TEST env flag)
2022-07-14 14:14:24 -03:00
Roberto Dip 08cd206de7 only send vuln configs to goroutine (#6650)
In https://github.com/fleetdm/fleet/pull/6630 I added a new config for packaging, but when I started the server I got:

```
~/fleet $ ./build/fleet serve --dev --dev_license
level=info ts=2022-07-13T21:36:06.055998Z component=redis mode=standalone
fatal error: newproc: function arguments too large for new goroutine

runtime stack:
runtime.throw({0x103d85259, 0x37})
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/panic.go:1198 +0x54
runtime.newproc1(0x104569a30, 0x14000ffda28, 0x7f8, 0x140000001a0, 0x103bc85a4)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4299 +0x4d0
runtime.newproc.func1()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4255 +0x4c
runtime.systemstack()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:230 +0x6c

goroutine 1 [running]:
runtime.systemstack_switch()
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/asm_arm64.s:187 +0x8 fp=0x14000ffd9c0 sp=0x14000ffd9b0 pc=0x102b60958
runtime.newproc(0x7f8, 0x104569a30)
	/Users/roperzh/.gvm/gos/go1.17/src/runtime/proc.go:4254 +0x54 fp=0x14000ffda10 sp=0x14000ffd9c0 pc=0x102b38034
main.runCrons({0x1045cd500, 0x14000262990}, {0x104611a38, 0x14000262930}, 0x14000298190, {0x1045a0480, 0x140009c9f20}, {{{0x103d1adc4, 0x3}, {0x103d2b188, ...}, ...}, ...}, ...)
	/Users/roperzh/fleet/cmd/fleet/serve.go:694 +0x2c4 fp=0x14000ffe260 sp=0x14000ffda10 pc=0x103bc85a4
main.createServeCmd.func1(0x1400027ca00, {0x140000bcb40, 0x0, 0x2})
```

With my local changes, `serve.go:694` is this line:

https://github.com/fleetdm/fleet/blob/7559988000efc3d7408d9914edd5a3f2f1052138/cmd/fleet/serve.go#L685-L686

After passing only a subset of the config the issue was solved.
2022-07-14 08:04:29 -03:00
Roberto Dip 6faee84f57 allow to configure a default global enroll secret (#6609)
Related to #6365 this adds a new config to set a global enroll token that will be used by the server.
2022-07-12 19:12:10 -03:00
Roberto Dip f7dd8c86cd implement a docker image to package orbit natively in Linux (#6504)
Related to #6364 and #6363, this:

- Adds a new Docker image, `fleetdm/fleetctl` equipped with all necessary dependencies to build Fleet-osquery binaries for all platforms
- Modifies the package generation logic to special case this scenario via an environment variable `FLEETCTL_NATIVE_TOOLING`
- Adds a new GitHub workflow to test this

There are more details in the README, but part of the special-casing logic is in place to output the binaries to a folder named `build` when they are run with `FLEETCTL_NATIVE_TOOLING`, this is so we can persist the binary generated by the docker container via a bind mount:

```bash
docker run -v "$(pwd):/build" fleetdm/fleetctl package --type=msi
```

To test this changeset, I have generated packages for all platforms, both via the new Docker image and via the classic `fleetctl package`.
2022-07-11 09:49:13 -03:00
Lucas Manuel Rodriguez a336ed61e5 Add gotestfmt to improve test output and fix flaky tests (#6528) 2022-07-11 08:12:33 -03:00
Luke Heath 7408dc0aae Update dev license (#6476) 2022-07-01 10:17:19 -07:00
Zach Wasserman db22f68c88 Separate health checks for MySQL and Redis (#6468)
This required a bit of refactoring of some mocking due to how the code
generation does not handle having the same function in different types.
2022-07-01 08:08:03 -03:00
Zach Wasserman 974c8e008d Improve logging for fleetctl with permissions errors (#6460) 2022-06-30 13:24:19 -07:00
Roberto Dip b2d07e56b5 report error analytics (#6341)
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.

For each error stored, this includes:

- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12

This also includes a minor refactor around error types.
2022-06-28 16:31:14 -03:00