* Remove username from UI code
* Remove username from tests
* Remove username from database
* Modify server endpoints for removing username
* Implement backend aspects of removing username
* Update API docs
* Add name to fleetctl
- Add enable_analytics column to database.
- Allow enable_analytics to be set via API.
- Add messaging in fleetctl setup.
Note that this defaults to off for existing installations, and defaults
on for newly set up installs.
No collection or sending of analytics yet exists, we are strictly
storing the preference at this time.
Part of #454
The previous strategy unintentionally cleared existing environment
values, preventing us from setting a specific version of the Fleet
container to run.
This should support Redis in both cluster and non-cluster modes.
Updates were made separately to github.com/throttled/throttled to support the slight changes in types.
Co-authored-by: Joseph Macaulay <joseph.macaulay@uber.com>
Co-authored-by: Zach Wasserman <zach@fleetdm.com>
- Add policy.rego file defining authorization policies.
- Add Go integrations to evaluate Rego policies (via OPA).
- Add middleware to ensure requests without authorization check are rejected (guard against programmer error).
- Add authorization checks to most service endpoints.
- Move team-related service methods to `ee/server/service`.
- Instantiate different service on startup based on license key.
- Refactor service errors into separate package.
- Add support for running E2E tests in both Core and Basic tiers.
- Add `team_id` field to secrets.
- Remove secret `name` and `active` fields (migration deletes inactive secrets).
- Assign hosts to Team based on secret provided.
- Add API for retrieving secrets by Team.
Some datastore and service methods would return slices of structs,
rather than slices to pointers of structs (which most methods used).
Make this more consistent.
- Add config option for license key.
- Define license details data structure.
- Include license details in app config API responses.
Currently any non-empty value for `--license_key` behaves as though the
installation is licensed for `basic`. If the license key is empty,
`core` is returned.
Still to come is the appropriate parsing for the license key.
This feature enables a new config option (redis.duplicate_results). When set to true, all Live Query results will be copied to an additional Redis pubsub channel named LQDuplicate
This is useful in a scenario that would involve shipping the Live Query results outside of Fleet, near-realtime.
- Allow agent options to be set on per-team basis.
- Move global agent options into app configs.
- Update logic for calculating agent options for hosts.
- Updates to relevant testing.
Instead of synchronously updating the seen_time column for a host on an update, batch these updates to be written together every 1 second.
This results in a ~33% reduction in MySQL CPU usage in a local test with 4,000 simulated hosts and MySQL running in Docker.
Prevent abuse of these endpoints with rate limiting backed by Redis. The
limits assigned should be appropriate for almost any Fleet deployment.
Closes#530
This PR contains the initial implementation of the fleetctl updates commands, along with documentation on using this to self-host an agent update server.
Co-authored-by: Noah Talerman <noahtal@umich.edu>
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
- Return same error in all cases for login endpoint.
- Log error details in server logs.
- Make most login errors take ~1s to prevent timing attacks.
- Don't return forgot password errors.
- Log password errors in server logs.
- Make most forgot password requests take ~1s to prevent timing attacks.
Fixes#531
Make fleetctl preview available over HTTP
Run a second copy of the Fleet server listening over HTTP on
localhost:1337 so that the UI can be used without the errors displayed
with a self-signed TLS certificate. Osquery clients and fleetctl
continue to communicate with the existing Fleet server on
https://localhost:8412.
Closes#504
Co-authored-by: Mike McNeil <mikermcneil@users.noreply.github.com>
- Download configuration on every run. This ensures that users always
get the latest version.
- Pull latest Docker containers on every run.
- Add stop and reset commands (for cleanup).
Fixes#378
Osquery now exposes more information during host enrollment than Fleet
previously handled. We can use this to provide more options to users in
problematic enrollment scenarios.
Users can configure --osquery_host_identifier in Fleet to set which
identifier is used to determine uniqueness of hosts. The
default (provided) replicates existing behavior in Fleet. For many
users, setting this to instance will provide better enrollment
stability.
Closes#373
- Better documentation and error for missing docker-compose.
- Handle case of no existing config file.
- Make logs directory world-writable (to allow writes from inside container).
Fixes#286
- Add --dev flag that will set default flag values. This simplifies the
invocation of Fleet in a development environment.
- Change defaults in docker-compose to use `fleet` in place of `kolide`.
- Skip prompt in `prepare db` when `--dev` specified.
- Update developer documentation.
Updates to MySQL configuration in docker-compose.yml may require
existing development containers and volumes to be deleted (this will
delete data in MySQL):
```shell
docker-compose rm -sf
docker volume rm fleet_mysql-persistent-volume
```
Closes#170
- Make the preview directory in the default .fleet directory.
- Check for Docker daemon installed but not running.
- Add message for Chrome users on self-signed certs.
- Display login information on later invocations of command.
- Remove "Kolide" from error messages.
Closes#190
Part of #197
Mozilla's recommended settings have changed since this was last updated.
We now link directly to the relevant revision in the Mozilla wiki to
avoid confusion if this is updated.
- Run the `fleetctl setup` command to set up the Fleet server with
default username ('admin') and password ('admin123#').
- Configures fleetctl if it has not yet been configured.
Closes#152
The current implementation of FleetDM doesn't support Docker secrets for supplying the MySQL password and JWT key. This PR provides the ability for a file path to read in secrets. The goal of this PR is to avoid storing secrets in a static config or in an environment variable.
Example config for Docker:
```yaml
mysql:
address: mysql:3306
database: fleet
username: fleet
password_path: /run/secrets/mysql-fleetdm-password
redis:
address: redis:6379
server:
address: 0.0.0.0:8080
cert: /run/secrets/fleetdm-tls-cert
key: /run/secrets/fleetdm-tls-key
auth:
jwt_key_path: /run/secrets/fleetdm-jwt-key
filesystem:
status_log_file: /var/log/osquery/status.log
result_log_file: /var/log/osquery/result.log
enable_log_rotation: true
logging:
json: true
```
This adds the option to set up an S3 bucket as the storage backend for file carving (partially solving #111).
It works by using the multipart upload capabilities of S3 to maintain compatibility with the "upload in blocks" protocol that osquery uses. It does this basically replacing the carve_blocks table while still maintaining the metadata in the original place (it would probably be possible to rely completely on S3 by using object tagging at the cost of listing performance). To make this pluggable, I created a new field in the service struct dedicated to the CarveStore which, if no configuration for S3 is set up will be just a reference to the standard datastore, otherwise it will point to the S3 one (effectively this separation will allow in the future to add more backends).
Zach Wasserman