Commit Graph
79 Commits
Author SHA1 Message Date
Lucas Manuel Rodriguez a5349672eb Amend fleetctl package to support /var/lib legacy orbit (legacy would mean <= 0.0.11) (#5532)
* Add logs to troubleshoot orbit

* Run journalctl on a different step

* Add legacy orbit support to opt version of fleetctl

* Fix macos logs permission error

* Checkout repository

* Compile fleetctl from branch
2022-05-03 16:46:02 -03:00
gillespi314 1dabf52834 Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Michal NicponandLucas Rodriguez 15c69058bb fix SELinux issue (#5335)
Install orbit to /opt instead of /var/lib. When installing to /var/lib,
the default selinux context of var_lib_t gets applied, which results in
an AVC error when running via systemd.

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2022-05-02 12:18:59 -06:00
Zach Wasserman e5a80fa3f5 Add user agent to Orbit HTTP client (#5429)
Allows identification of which Orbit versions are in use from the update
server.

Refactored the build information into a separate `package build` to
support importing it from multiple places.
2022-05-02 11:03:49 -07:00
Martin Angers d890970844 Implement the Jira API call (#4938) 2022-04-05 12:58:34 -04:00
Lucas Manuel Rodriguez c82c580716 Orbit: Add Fleet Desktop support to Windows (#4873)
* Orbit: Add Fleet Desktop support to Windows

* Rename workflow, fix linux build

* Do not compile systray on linux

* nolint on unused

* Fix lint properly

* nolint both checkers

* Fix monitor logic in desktopRunner

* Fix interrupt and execute order
2022-04-01 17:28:51 -03:00
dependabot[bot] 7f4c88ad22 Bump github.com/valyala/fasthttp from 1.31.0 to 1.34.0 (#4716)
Bumps [github.com/valyala/fasthttp](https://github.com/valyala/fasthttp) from 1.31.0 to 1.34.0.
- [Release notes](https://github.com/valyala/fasthttp/releases)
- [Commits](https://github.com/valyala/fasthttp/compare/v1.31.0...v1.34.0)
2022-03-28 10:45:15 -07:00
Lucas Manuel Rodriguez 378b1dd34f Upgrade osquery-go (#4728)
* Upgrade and replace kolide/osquery-go with osquery/osquery-go

* Upgrade macadmins/osquery-extension to v0.0.7

* Upgrade kolide/launcher to latest

* go mod tidy
2022-03-23 12:15:37 -03:00
Lucas Manuel RodriguezandZach Wasserman ecdfd627b6 Fleet Desktop MVP (#4530)
* WIP

* WIP2

* Fix orbit and fleetctl tests

* Amend macos-app default

* Add some fixes

* Use fleetctl updates roots command

* Add more fixes to Updater

* Fixes to app publishing and downloading

* Add more changes to support fleetctl cross generation

* Amend comment

* Add pkg generation to ease testing

* Make more fixes

* Add changes entry

* Add legacy targets (until our TUF system exposes the new app)

* Fix fleetctl preview

* Fix bool flag

* Fix orbit logic for disabled-updates and dev-mode

* Fix TestPreview

* Remove constant and fix zip-slip attack (codeql)

* Return unknown error

* Fix updater's checkExec

* Add support for executable signing in init_tuf.sh

* Try only signing orbit

* Fix init_tuf.sh targets, macos-app only for osqueryd

* Specify GOARCH to support M1s

* Add workflow to generate osqueryd.app.tar.gz

* Use 5.2.2 on init_tuf.sh

* Add unit test for tar.gz target

* Use artifacts instead of releases

* Remove copy paste residue

* Fleet Desktop Packaging WIP

* Ignore gosec warning

* Trigger on PR too

* Install Go in workflow

* Pass url parameter to desktop app

* Fix fleetctl package

* Final set of changes for v1 of Fleet Desktop

* Add changes

* PR fixes

* Fix CI build

* add larger menu bar icon

* Add transparency item

* Delete host_device_auth entry on host deletion

* Add SetTargetChannel

* Update white logo and add desktop to update runner

* Add fleet-desktop monitoring to orbit

* Define fleet-desktop app exec name

* Fix update runner creation

* Add API test before enabling the My device menu item

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2022-03-21 14:53:53 -03:00
Benjamin Edwards 74bb559645 Add public ip to hosts & derive geolocation when rendering host (#4652)
* geoip wip
* return nil if ip is empty string or if ParseIP returns nil
* add ui component to render geolocation if available, address PR feedback
* render public ip if available
* add changes file, document geoip in deployment guide
* update rest-api docs
2022-03-21 12:29:52 -04:00
Martin Angers bb678b6b2e Add support for downloading a list of hosts in CSV format (#4596) 2022-03-15 15:14:42 -04:00
Lucas Manuel Rodriguez c641e39ef4 Allow disabling auto updates in fleetctl package and orbit (#4296)
* Add disable-updates flag to fleetctl and orbit

* Fix ruleguard execution error on make lint-go

* Introduce dev-mode for ease of development of orbit

* Add changes file
2022-02-18 15:42:39 -03:00
Zachary WinnermanandTomas Touceda c5c72ed713 Add apm for testing apm (#4053)
* Add apm for testing apm

* Testing opentracing

* testing

* Testing

* go fmt

* Add config switch for tracing.

* fixup

* Update cmd/fleet/serve.go

Co-authored-by: Tomas Touceda <chiiph@gmail.com>

* Add support for both elasticapm and opentelemetry

* Fix driver stuff and config options

* Fixup

* fixup

* Add changes file

* Add config for sql driver

* fixup

* Add doc to exported field

* testing

* fixup

* fixup

* Testing again

* fixup

* testing

* Undo

Co-authored-by: Tomas Touceda <chiiph@gmail.com>
2022-02-15 12:42:22 -05:00
Lucas Manuel Rodriguez aa9e257a2a Upgrade fleetdm/goose version to latest (#4195) 2022-02-14 17:33:37 -03:00
Lucas Manuel Rodriguez be72dc356c Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Lucas Manuel Rodriguez 9c25ea1641 Prepare LoadHostByNodeKey query once (#4128)
* Prepare LoadHostByNodeKey query once

* Use a protected map for storing statements

* Add proposed test
2022-02-14 12:13:38 -03:00
Zach Wasserman 1b1d3f8a61 Update go-tuf dependency (#3837)
Resolves the warning described in #3699 by updating to the latest
version of the dependency with the warning fixed.

The warning should go away on all clients after new metadata is 
generated with these changes.
2022-02-10 08:16:36 -08:00
Lucas Manuel Rodriguez 81672ee50e Add dev sql interceptor for logging of queries (#3815)
* Add dev sql interceptor for logging of queries

* Remove extra spaces and tabs from queries

* Make regex global
2022-01-21 14:28:21 -03:00
Tomas Touceda 9d572309ae Add sentry (#3669)
* Add sentry

* Fix gosum

* More gosum fixes

* Add missing def for config

* Enrich sentry scope a bit

* Add changes file

* Add goroutine safe scope to errors

* Encapsulate sentry logic

* Add documentation for new flag

* Add sentry capturing to crons and other background tasks

* Only send to sentry when enabled
2022-01-20 16:41:02 -03:00
Martin Angers e988d16eb3 Update the prometheus go client library (#3140) 2021-12-20 09:20:58 -05:00
Zach Wasserman 4e72ebb68f Build Orbit releases in GitHub Actions (#3413)
Complete build including macOS signing+notarization.
2021-12-19 17:06:17 -08:00
Tomas Touceda a0404b6736 Add software count API (#3105)
* Add software count API

* Fix makefile

* Fine no mock generating at this point

* Actually, one last try

* Use go install instead

* Fix go sum/mod

* Improve documentation

* Try setting node to 14
2021-12-03 10:54:17 -03:00
Tomas Touceda a36e029d95 App config caching per instance (#3071)
* Do caching of app config per instance instead of across all of them in redis

* Add changes file

* Simplify code based on review comment

* Use go-cache instead of creating our own

* Dont export consts

* Copy app config before returning it

* Fix lint

* Update go sum

* Update go sum
2021-11-29 12:51:57 -03:00
Martin Angers c997f853e5 Make creation of http.Client uniform across the codebase (#3097) 2021-11-24 15:56:54 -05:00
Victor VrantchanandZach Wasserman a760cdc26d Add MacAdmins tables in Orbit extension (#2140)
Uses the extension tables from https://github.com/macadmins/osquery-extension.

Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-11-15 18:40:53 -08:00
Lucas Manuel Rodriguez 5c1edaf527 Kill existing instances of orbit and osqueryd in fleetctl preview (#2921)
* Kill existing instances of orbit and osqueryd in fleetctl preview

* Add changes file
2021-11-15 18:12:25 -03:00
Zach Wasserman e279dc1682 Implement fleetctl updates rotate (#2831)
Add support for updating keys used in the TUF update system.
2021-11-15 10:01:48 -08:00
Martin Angers 348841c90b Add explanation output to fleetctl package (#2864) 2021-11-15 08:40:58 -05:00
Lucas Manuel Rodriguez 837bfed167 Try goqu as sql builder on listSoftwareDB (#2773)
* Try goqu as sql builder on listSoftwareDB

* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Tomas Touceda 2902da76ca Issue 2027 better error visibility (#2069) 2021-11-02 13:35:57 -04:00
Lucas Manuel Rodriguez 43d1a8c9a0 Fix races in go tests and run with -race on CI (#2712)
* Fix races in go tests and run with -race on CI

* Fix race in pubsub

* Increase timeout to 15m for go tests

* CI takes forever, try disabling race

* Remove timeout from go tests
2021-11-01 15:31:01 -03:00
Tomas Touceda 825939e3dc Improve osquery perf (#2503)
* Improve osquery perf

* Update dependencies

* Embed template so this can be run from the root of the repo

* Fix lint

* Address review comments
2021-10-14 10:09:58 -03:00
Tomas Touceda 8600d71d35 Add osquery perf (#2190)
* Add osquery perf

* Update dockerfile and gh action
2021-09-22 17:18:55 -03:00
Martin Angers 0ac4fc4c1a Update redisc and replace EachRedisNode implementation (#1949) 2021-09-07 13:25:36 -04:00
Tomas Touceda d6d53acda5 Issue 1600 fleetctl license expiration (#1800)
* Show banner in fleet and fleetctl if license expired

* Ignore if license is nil or tier is core

* Address review comments
2021-08-26 10:28:53 -03:00
Tomas Touceda c6c63ab12a Refactor app config (POC, for now) (#1685) 2021-08-20 12:27:41 -03:00
Tomas Touceda 96c0244c04 Fix redis locking receive (#1655)
* Receive redis data with timeout to not hold the connection

* Address review comments
2021-08-13 11:16:11 -03:00
Tomas Touceda 989e638cc0 Make creating dirs and files more secure by checking permissions (#1566)
* Add safe mkdirall and open

* Use secure as much as possible and merge gomodules for orbit to fleet

* Improve openfile and mkdirall to check for permissiveness instead of equality

* Don't shift

* Fix links

* Address review comments
2021-08-11 11:02:22 -03:00
Zach Wasserman 9db49a1128 Update JWT library to github.com/golang-jwt/jwt (#1568)
- The former repo has been migrated (see
  https://github.com/dgrijalva/jwt-go/issues/462).

- Dependabot alerted on a security issue in the old
  version (https://github.com/advisories/GHSA-w73w-5m7g-f7qc). This issue
  should not effect Fleet due to no use of `aud` claims.
2021-08-06 08:49:38 -07:00
Tomas Touceda f8b7a83cc6 Process stored CPEs and store found CVEs (#1533)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Add first cve db generation

* WIP but with final strategy, preparring to merge main

* Fix merge conflicts

* WIP

* wip

* Insert CVEs to the db

* Remove unused code

* Use wg instead of counting

* Call cancelFunc to avoid ctx leak

* Fix logs for better readability

* Point code to fleetdm instead of my repo
2021-08-04 18:01:39 -03:00
Tomas Touceda 46b0b7765b Issue 1435 software to cpe (#1488)
* WIP

* WIP

* Make path optional and fix tests

* Add first generate

* Move to nvd package

* remove replace

* Re-add replace

* It's path, not file name

* Change how db path is set and use etag

* Fix typos

* Make db generation faster

* Remove quotes

* Doesn't like comments

* Samitize etag and save to file

* Refactor some things and improve writing of etagenv

* Compress file and truncate amount of items for faster testing

* Remove quotes

* Try to improve performance

* Ignore truncate error if not exists

* Minor cleanup and make sqlite have cpe prefix

* Simplify code and test sync

* Add VCR for sync test

* Check for nvdRelease nil

* Add test for the actual translation

* Address review comments

* Rename generate command because we'll have a cve one too

* Move to its own dir

* Address review comments
2021-07-29 13:10:34 -03:00
Tomas Touceda d5e40f329e Issue 1324 add activity feed (#1343)
* Add activities generation

* Add activities endpoint

* Fix merge error

* Fix indentation issue

* Add changes file

* Address PR review comments

* Add mock activity func

* Address codacy warings

* Set foreign key but on delete set null

* Make user_id set to null if deleted
2021-07-13 16:54:22 -03:00
Zach Wasserman c5280c0517 Add v4 suffix in go.mod (#1224) 2021-06-25 21:46:51 -07:00
Zach Wasserman 07b89c4cbd Implement release automation (#1215)
- Use goreleaser to automate release process.
- Add new dockerfiles for fleet (with fleetctl) and fleetctl (only).
- Add GitHub Action Workflow to run goreleaser on new tag.
- Update NPM to match new archive naming.
2021-06-25 18:40:26 -07:00
Zach Wasserman 2ad557e3b3 Merge branch 'main' into teams 2021-06-18 09:42:20 -07:00
47b423ee29 Add Redis cluster support (#1045)
This should support Redis in both cluster and non-cluster modes.

Updates were made separately to github.com/throttled/throttled to support the slight changes in types.

Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
2021-06-18 08:51:47 -07:00
Zach Wasserman 9876dbe6b6 Implement license loading and validation (#908)
- Load license from ES256 signed JWT key.
- Parse license claims into LicenseInfo struct.
- Update contribution documentation with sample license key.

Closes #816.
2021-05-31 11:35:15 -07:00
Zach Wasserman 69f73b56f8 Upgrade dependencies and remove replace directive (#802)
Now that https://github.com/throttled/throttled/pull/87 is merged, we
can remove the stopgap `replace` directive in `go.mod`.
2021-05-18 17:00:20 -07:00
Zach Wasserman 1638d5e2da Update Redigo version and timeouts (#793)
- Use newer version of Redigo that properly handles TLS mismatch.
- Add timeouts for Redis TCP connection.

Fixes #792
2021-05-18 15:04:14 -07:00
WangXiang 468754f2b9 Format and clean code (#774)
1. use [staticcheck](https://staticcheck.io/) to check the code, and fix some issues.
2. use `go fmt` to format the code.
3. use `go mod tidy` clean the go mod.
2021-05-17 10:29:50 -07:00