Commit Graph
244 Commits
Author SHA1 Message Date
Konstantin Sykulev 75c05aeb3f adding gotestsum for better test output (#40753)
**Related issue:** Resolves #30896

## Testing

- [x] QA'd all new/changed functionality manually
2026-03-03 16:01:11 -06:00
Victor Lyuboslavsky 05648093f1 Added dedicated Go test workflow for activity bounded context (#40653) 2026-02-27 07:53:57 -06:00
Victor Lyuboslavsky 9a25a2b43d Fix broken lint on main. (#40147)
`make lint-go` was not flagging issues in CI.
2026-02-20 07:15:23 -06:00
Ian Littman 6570e8846a 🤖 Update goreleaser snapshot build to pull version number from branch name when applicable (#39593)
For #39522.
2026-02-10 13:51:52 -06:00
2b5eedac5e 🤖 Improve build version population based on git branch (#39523)
For #39522. Guessing this doesn't resolve goreleaser issues though?

Tested with this branch (shows a snapshot), `4.77.9-plus-sparklies`,
`v4.77.12-without-sparkles`, and `rc-patch-fleet-v4.77.7` locally to
make sure this behaves properly.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] QA'd all new/changed functionality manually

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: iansltx <472804+iansltx@users.noreply.github.com>
2026-02-09 08:43:02 -06:00
jacobshandlingandVictor Lyuboslavsky 79b7d83bf5 Use nilaway to incrementally check for unsafe nil pointer dereferences (#39030)
**Related issue:** Resolves #32999 

- Enhanced internal code quality tooling by implementing a custom
linting build configuration.
- Updated continuous integration workflow to utilize the new custom
build process for improved code analysis and consistency checks.

### Confirmed that running local custom `golangci-lint` build with
`nilaway` plugin catches lots of issues when run on `fleet/`:
<img width="1555" height="939" alt="Screenshot 2026-01-29 at 2 47 50 PM"
src="https://github.com/user-attachments/assets/c6a18400-fdf0-4104-97d8-e117efc28ed6"
/>
<img width="301" height="109" alt="Screenshot 2026-01-29 at 2 48 36 PM"
src="https://github.com/user-attachments/assets/b459ee7b-b391-457a-9191-17d56a80c783"
/>

### Confirmed that new incremental CI step using custom `golangci-lint`
build with `nilaway` plugin _does not_ check any `.go` files when none
have been modified, and so passes successfully (incremental check works
as expected):
<img width="337" height="197" alt="Screenshot 2026-01-29 at 2 45 24 PM"
src="https://github.com/user-attachments/assets/c7ae585e-2e10-4ebf-a3a3-96c26063f1e4"
/>

### Confirmed that new incremental CI step using custom `golangci-lint`
build with `nilaway` plugin _does_ check modified lines of `.go` files,
and so successfully flags a potentially unsafe dereference and fails the
job (incremental check works as expected):
<img width="825" height="491" alt="Screenshot 2026-01-29 at 5 50 01 PM"
src="https://github.com/user-attachments/assets/82bc5616-6fb9-4357-b8bc-c7eebc42c2d8"
/>

### Honorable mention:
`nilaway` agrees that `listHostSoftware` is a wild beast:
<img width="1277" height="190" alt="Screenshot 2026-01-29 at 5 52 32 PM"
src="https://github.com/user-attachments/assets/dfade2a8-fbcc-4bae-98f9-6bf1089620d2"
/>

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Fleet dev cycle reliability improvements**


<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com>
2026-02-06 08:51:17 -06:00
Nico 2a23fe1724 Fix: GLIBC version incompatibility causes orbit agent failure on Ubuntu 20.04 during fleet-desktop update (#38648)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #35413 

Applied the same fix as in
https://github.com/fleetdm/fleet/pull/29186/files:
- Added musl-tools to the container image. This provides musl-gcc, a
compiler that links against musl libc instead of glibc. We use it for
static linking as explained below.
- Added static linking flags (CGO_ENABLED=1 CC=musl-gcc -linkmode
external -extldflags "-static"). This produces a self-contained binary
with all C library code embedded, eliminating runtime dependencies on
the host system's glibc version.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

Steps:

- Started TUF server for the first time. Generated a Linux x86_64 image.

```
SYSTEMS="linux" \                                      
DEB_FLEET_URL=https://nicofleet.ngrok.io \
DEB_TUF_URL=http://nicotuf.ngrok.io \
GENERATE_DEB=1 \
ENROLL_SECRET=tm2CHBEF1I5BVuM1+4hzRRtpC5ZYV8vb \
FLEET_DESKTOP=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
```

- Installed `fleet-osquery_26.1.46030_amd64.deb` generated by the
previous command on a Kubuntu 20.04 x86_64 VM.
- Ran `sudo journalctl -u orbit` to debug logs and verified that the
GLIBC incompatibility error was raised:

```
tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 sudo[125623]: pam_unix(sudo:session): session opened for user nicolas by (uid=0)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125624]: /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125624]: /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /opt/orbit/bin/desktop/linux/stable/fleet-desktop/fleet-desktop)

tammi 22 15:36:53 nicolas-Standard-PC-i440FX-PIIX-1996 sudo[125623]: pam_unix(sudo:session): session closed for user nicolas

tammi 22 15:37:08 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:08+02:00 INF killing any pre-existing fleet-desktop instances

tammi 22 15:37:09 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:09+02:00 INF attempting to get user session type and display id=1000 user=nicolas

tammi 22 15:37:09 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[125199]: 2026-01-22T15:37:09+02:00 ERR failed to get X11 display, using default :0 error="display not found on who output"
```

- Built a new version of the agent after applying the fixes on this PR
and pushed it to the TUF server:

```
# 1. Hardcode orbit to a higher version
export ORBIT_VERSION=26.1.46099

#2. Generate new package
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-linux

#3. Update to TUF server
./tools/tuf/test/push_target.sh linux desktop desktop.tar.gz $ORBIT_VERSION
```

<img width="396" height="179" alt="Screenshot 2026-01-22 at 5 18 25 PM"
src="https://github.com/user-attachments/assets/56182580-1d54-4945-af03-98762e7795e9"
/>


- In the VM, verified that an update for fleet desktop was detected by
running `sudo journalctl -u orbit -g "update detected"`:

```
tammi 22 20:33:32 nicolas-Standard-PC-i440FX-PIIX-1996 orbit[4114]: 2026-01-22T20:33:32+02:00 INF update detected target=desktop
```

- Verified the new version is shown both on the desktop icon and the
Fleet UI:

<img width="1373" height="248" alt="Screenshot 2026-01-22 at 5 22 17 PM"
src="https://github.com/user-attachments/assets/9fac73ac-a3a6-49ed-82c7-261abec43798"
/>

<img width="1420" height="496" alt="Screenshot 2026-01-22 at 5 22 46 PM"
src="https://github.com/user-attachments/assets/55321108-9233-4799-b5e5-0713172c0138"
/>


## fleetd/orbit/Fleet Desktop

- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
2026-01-23 12:02:03 -03:00
Ian Littman fe2a9a867e Swap minio to rustfs (#36851)
Resolves #36909.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-12-10 10:03:48 -06:00
Jonathan Katz b8d2ba371e Update replica db setup and guide (#36918)
**Related issue:** Resolves #35937
Changes:
- Fix `make db-replica-setup`
- mention the tool in
`docs/contributing/getting-started/testing-and-local-development.md`.
2025-12-08 17:07:04 -05:00
Victor Lyuboslavsky 40022c5537 Add incremental lint run, with modernize as the linter. (#36711)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #32999

The `modernize` linter was discussed some time ago in the backend sync.
We wanted to add it but it was not possible.

Now that it has been added to golangci-lint, we are adding it.

golangci-lint has incremental mode, where only changes vs the base
branch are linted. This is nice when adding new linters without needing
to fix the whole codebase. That said, it would be nice to `modernize`
the whole codebase.
2025-12-08 09:06:05 -06:00
Lucas Manuel Rodriguez 93bea23859 Add CVE scanning for fleetdm/wix and split out workflows (#33687)
Attempting to fix
https://github.com/fleetdm/fleet/actions/runs/18120473187/job/51564073671#step:11:38

Changes:
- The docker images are too big so they started to cause issues in
Github runners. Thus I'm splitting the one workflow into three separate
workflows, one for each image.
- While we are at it: start tracking vulnerabilities in fleetdm/wix and
fleetdm/bomutils.

New runs:
-
https://github.com/fleetdm/fleet/actions/runs/18161326970/job/51692559418
-
https://github.com/fleetdm/fleet/actions/runs/18161326953/job/51692559257
-
https://github.com/fleetdm/fleet/actions/runs/18161326952/job/51692559172
2025-10-01 19:37:22 -03:00
Victor Lyuboslavsky e872f9a984 Update golangci-lint to v2.4.0 (#33251)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #33250

Waived most new failures. Planning to come back and fix some of them in
subsequent PRs.
2025-09-22 13:17:11 -05:00
Jordan Montgomery 596fb70399 Update Makefile swift dialog versions and add github workflow (#32511)
For #31675
For #32099 

Adds a Github workflow to generate our packaged build of Swift
Dialog(following existing Nudge packager), updates the version to 2.5.6
and modifies the Migration dialog to render properly with the new Swift
Dialog version(it previously rendered it just didn't format as expected
due to changes in the markdown formatter)

Makefile changes are necessary not only because of the version bump but
because the latest package includes xattrs for some strange reason.
Extracting it verbatim on a system(at least with our Go implementation)
creates files that cause Gatekeeper to stop execution of swift dialog.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] QA'd all new/changed functionality manually

## fleetd/orbit/Fleet Desktop

- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
2025-09-05 10:49:21 -04:00
Carlo 8bc8d01f0a Merge Android datastore into main Fleet datastore (#32233)
Resolves #31218
2025-08-25 11:41:28 -04:00
Victor Lyuboslavsky 98954e8113 Make MySQL test port configurable (#31782)
Fixes #31781 

Only test-related changes.

I found this useful for agentic AI workflows. For example, you have an
AI agent debugging/rerunning a test. Meanwhile, you can spin up another
`mysql_test` instance in another workarea and work there in parallel.
2025-08-12 15:45:43 +02:00
Lucas Manuel Rodriguez 4265bed7f1 Make sure VEX report is up-to-date with a CI check (#31759) 2025-08-11 14:55:31 -05:00
Victor Lyuboslavsky 0180cc8086 Add SCEP endpoint for host identity. (#30589)
Fixes #30458 

Contributor docs PR: https://github.com/fleetdm/fleet/pull/30651

# Checklist for submitter

- We will add changes file later.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- Did not do manual QA since the SCEP client I have doesn't support ECC.
Will rely on next subtasks for manual QA.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Introduced Host Identity SCEP (Simple Certificate Enrollment Protocol)
support, enabling secure host identity certificate enrollment and
management.
* Added new API endpoints for Host Identity SCEP, including certificate
issuance and retrieval.
* Implemented MySQL-backed storage and management for host identity SCEP
certificates and serials.
* Added new database tables for storing host identity SCEP certificates
and serial numbers.
* Provided utilities for encoding certificates and keys, and handling
ECDSA public keys.

* **Bug Fixes**
  * None.

* **Tests**
* Added comprehensive integration and unit tests for Host Identity SCEP
functionality, including certificate issuance, validation, and error
scenarios.

* **Chores**
* Updated test utilities to support unique test names and new SCEP
storage options.
* Extended mock datastore and interfaces for new host identity
certificate methods.

* **Documentation**
* Added comments and documentation for new SCEP-related interfaces,
methods, and database schema changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-11 11:44:07 -03:00
Lucas Manuel Rodriguez 5646062c85 Update go to 1.24.4 and add some automation (#29954)
Fixes CVE-2025-22874 reported by
https://github.com/fleetdm/fleet/actions/runs/15601368321/job/43941793647.

(IMO not a critical CVE, so it doesn't need to be cherry-picked into
v4.69.0.)

Added automation to make this easier next time.
2025-06-13 13:08:14 -05:00
Luke Heath 63596c761e Update changelog command (#29382) 2025-05-27 15:17:47 -05:00
Lucas Manuel RodriguezandScott Gress 7f85e529be Iterate status.md for reporting vulnerability updates (#29062)
For #28805.

See scenario we want to support in the linked issue.

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
2025-05-15 21:15:37 -03:00
Scott Gress 7225625bbb Fix make so that it builds both binaries again (#28538)
Previous work on the makefile broke the default `make` action. This
restores it by setting default binaries to be built.
2025-04-24 16:03:21 -05:00
Lucas Manuel RodriguezandSharon Katz 895194d63b Add scanning to released images and process to track vulnerabilities (#28087)
For #25902.

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2025-04-16 11:50:10 -03:00
Dante CatalfamoandLucas Rodriguez 94f6127edc Orbit for Windows ARM64 (#27882)
#27275 and #27274

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2025-04-11 10:18:28 -04:00
Victor Lyuboslavsky 7558d5afa1 Improvements to test-go CI job (#27916)
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.

Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
https://github.com/fleetdm/fleet/issues/27927

## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>

## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
2025-04-07 12:52:26 -05:00
Lucas Manuel Rodriguez 8003227554 Use go 1.24 new tool feature (#27765)
Release notes: https://tip.golang.org/doc/go1.24

> Go modules can now track executable dependencies using tool directives
in go.mod. This removes the need for the previous workaround of adding
tools as blank imports to a file conventionally named “tools.go”. The go
tool command can now run these tools in addition to tools shipped with
the Go distribution. For more information see [the
documentation](https://tip.golang.org/doc/modules/managing-dependencies#tools).
The new -tool flag for go get causes a tool directive to be added to the
current module for named packages in addition to adding require
directives.

I ran:
```
go get -tool github.com/fleetdm/fleet/v4/server/goose
go get -tool github.com/kevinburke/go-bindata
go get -tool github.com/quasilyte/go-ruleguard/dsl
go rm tools.go
go mod tidy
```

`make deps-go` was failing in CI because of the removal of `tools.go`
(my guess is that `go get .` was a nop because there was nothing in `.`
to download).
So, taking the chance of removing `deps-go` because `go` will download
packages during the build process. AFAICS there's no need to download
everything beforehand.
2025-04-07 11:12:05 -03:00
Scott Gress d51f2815ad FDM updates: fdm serve, snapshot/restore improvements (#27890)
For #27889 

This PR introduces several improvements to the Makefile/`fdm` tool for
development:
 
### `fdm serve` (alias `fdm up`)

Starts a local Fleet server (building the binary first). The first time
this is called, it will start the server on `localhost:8080` with the
`--dev` and `--dev_license` flags, but the command accepts all of the
options that you can pass to `fleet serve`. If you pass options to `fdm
serve`, then subsequent invocations _without_ options will replay your
last command. Additionally, `fdm serve` supports the following:

- `--use-ip`: start the local server on your system's local IP address
rather than `localhost`. This makes it easier to point VMs on your
system to the fleet server to act as hosts.
- `--no-build`: don't rebuild the fleet binary before starting the
server.
- `--no-save`: don't save the current command for future invocations
(useful for scripting)
- `--show`: show options for the last-invoked `fdm serve` command
- `--reset`: reset the options for `fdm serve`. The next time `fdm
serve` is invoked, it will use the default options.
- `--help`: show all of the Fleet server options

### `fdm snapshot` improvements

* Added `fdm snap` alias
* Tracks the name of the last snapshot saved, to use as the default for
`fdm restore`
* Suppresses the "don't use password in CLI" warning when saving the
snapshot

### `fdm restore` improvements

* Added `--prep` / `--prepare` option to run db migrations after
restoring snapshot.
* Improved UI (more options displayed, and clearer indicator for
selected option)
* Now defaults to last snapshot restored
2025-04-07 09:10:15 -05:00
Victor Lyuboslavsky 527428f0b4 Speeding up Go tests in CI. (#27434)
Using `go get .` (~3s) is faster than `go mod download` (~13s) in CI.
2025-03-25 12:03:24 -05:00
Victor Lyuboslavsky 46719c9fb2 Auto-generate and check Android schema.sql (#26720)
For #26218 

This is an infrastructure change. No functional feature changes.
2025-02-28 16:30:40 -06:00
Scott Gress f7e4a39915 Add help system to Makefile + FDM command (#25028) 2025-02-28 07:42:32 -06:00
Victor Lyuboslavsky 3d5666d4c6 Added GET enterprise API endpoint. (#26555)
For #26218 

- Added `GET /api/_version_/fleet/android_enterprise` andpoint and tests
- Set up some testing infrastructure for Android service tests -- see
new README.md

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-26 10:47:05 -06:00
Lucas Manuel Rodriguez f9b08468d7 Fix automation for fleetd orbit/TUF.md (#26406)
Scheduled workflow to update the orbit/TUF.md has been failing since I
changed the tooling to use the new TUF:
https://github.com/fleetdm/fleet/actions/workflows/fleetd-tuf.yml.
2025-02-18 10:34:52 -03:00
Lucas Manuel Rodriguez 9114d0405b Update TUF status generation to use new TUF repository (#26099)
For #25853.
2025-02-07 08:30:07 -03:00
Scott Gress 94eb573736 Add Fleet dev snapshot tool (#25909)
For #23750 

# Overview

This PR adds a basic tool for creating and restoring Fleet dev
snapshots. In this first iteration a snapshot is just a folder
containing a MySQL db dump made using the existing backup/restore
scripts, and the tool allows you to easily save and restore snapshots
interactively.

## Usage

* `make snapshot` to create a new snapshot
* `make restore` to select and restore a snapshot 

## Future plans

Future iterations can add metadata to snapshots to integrate things
like:

* node keys from osquery-perf, so you can easily reconnect to hosts
created in a previous session
* env vars from when the snapshot was made
* the branch from when the snapshot was made, to allow switching to that
branch and restarting the server as part of the restore process
*
![image](https://github.com/user-attachments/assets/6dc86581-5c12-4b57-b900-5034e00bc496)

## Demo


https://github.com/user-attachments/assets/1590c37a-3df9-4201-a42b-ccd1a36cb6cf
2025-02-05 09:52:10 -06:00
Lucas Manuel Rodriguez a4db139e82 Remove fleetctl binary from fleetdm/fleet image and remove unused Dockerfile (#25749)
For #25748.

Manually tested by:
1. Building a `fleet` executable for Linux on a Ubuntu VM (with
`-extldflags "-static"`) for Alpine.
2. Placing the executable in `tools/fleet-docker`.
3. Building a local docker image using `docker build -t
fleetdm/fleet:v42.42.42 --platform=linux/amd64 .`.
4. Running the docker image (using `docker run`) and use Fleet on the
browser.
```
docker run -v $(pwd)/../osquery:/run -p 8412:8412 -e FLEET_MYSQL_ADDRESS=host.docker.internal:3306 -e FLEET_MYSQL_DATABASE=fleet -e FLEET_MYSQL_USERNAME=fleet -e FLEET_MYSQL_PASSWORD=insecure -e FLEET_REDIS_ADDRESS=host.docker.internal:6379 -e FLEET_SERVER_ADDRESS=0.0.0.0:8412 -e FLEET_SERVER_CERT=/run/fleet.crt -e FLEET_SERVER_KEY=/run/fleet.key -e FLEET_LOGGING_JSON='true' -e FLEET_VULNERABILITIES_CURRENT_INSTANCE_CHECKS="yes" -e FLEET_VULNERABILITIES_DATABASES_PATH=/vulndb -e FLEET_LOGGING_DEBUG='true' -it fleetdm/fleet:v42.42.42
```

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2025-01-28 19:58:20 -03:00
Dante Catalfamo 8c338a1d77 Try splitting up integration tests (#25312)
Follow up to #25271 and #21774

Integration test failures will happen much faster of they occur, but now
the bottleneck is the `fleetctl` test suite.

It's trivial to continue splitting tests up now. We should look into
creating an action that checks that mock generation is up-to-date, run
it before all the tests, and then remove the mock generation step from
each test step. That would save about a minute and a half of runtime
from each test and help offset the cost of splitting the tests up.

![ci runtime
breakdown](https://github.com/user-attachments/assets/057b8ee1-782c-4e1f-9486-42c7d1169c81)
![ci runtime
max](https://github.com/user-attachments/assets/3a26995f-d9cb-490b-84d9-1a7fbb3cd6b3)

![image](https://github.com/user-attachments/assets/b4c888c8-867f-4bdd-9b69-0dc20d0d202a)
2025-01-10 12:52:13 -05:00
Dante Catalfamo e6fb647d95 Run CI tests in parallel (#25271)
#21774

Improves run time by about 30%.

Things have been arranged in such a way that splitting modules out
further will be trivial in the future, such as breaking the different
integration test suited into their own units.


![image](https://github.com/user-attachments/assets/ead46e4c-6f14-406d-a29b-b25abc79c384)


![image](https://github.com/user-attachments/assets/3f7fd7f3-d7a8-4ff8-a184-646a72f1d015)
2025-01-09 13:38:24 -05:00
Scott Gress bf7876af54 Add make commands to standardize running/debugging Go tests (#24606)
# Checklist for submitter

- [X] Manual QA for all new/changed functionality

# Details

This PR adds two new user-facing `make` targets:

* `run-go-tests`: run Go tests for one or more packages, optionally
filtering to specific tests
* `debug-go-tests`: debug (using Delve) Go tests for one or more
packages, optionally filtering to specific tests

Example usage:

```
# Run all tests in the mysql and gdmf packages
make run-go-tests PKG_TO_TEST="server/mdm/apple/gdmf server/datastore/mysql"
```

```
# Run all the TestMDMApple tests in the mysql package
make run-go-tests PKG_TO_TEST=server/datastore/mysql TESTS_TO_RUN="^TestMDMApple\$$" 
```

```
# Run only the TestMDMAppleProfileLabels test in the mysql package
make run-go-tests PKG_TO_TEST=server/datastore/mysql TESTS_TO_RUN="^TestMDMApple\$$/^TestMDMAppleProfileLabels\$$" 
```

# Notes

Two new "private" targets `.run-go-tests` and `.debug-go-tests` were
created as base commands for both `test-go` (used in CI) and the new
user-facing commands.
2024-12-11 11:06:56 -06:00
Lucas Manuel Rodriguez c993e5a485 Add workflow to build fleetd_tables (#23873)
Workflow to build `fleetd_tables` in Github.
2024-11-15 19:05:07 -03:00
Victor Lyuboslavsky d4525d8b94 Allow reusing docker container for building desktop-linux. (#23424) 2024-10-31 09:59:04 -05:00
Lucas Manuel Rodriguez 2e5bf75b6d fleetd to start up when TUF signatures are expired (#23102)
#22740

Full QA is still a WIP but this is ready for review.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-10-28 20:40:19 -03:00
Victor Lyuboslavsky f85b6f776f Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
Ian Littman 92bb7ec666 Add policy ID and name to activity for automated software installs, list Fleet as author rather than installer uploader (#22747)
#22424, #22705

TODO: integration test updates

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-10-09 18:15:56 -05:00
e4df7abb67 Backend build for script automation (#22472)
#22115, #22116

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

No changes file, as FE changes file covers the entire feature

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
2024-10-03 20:03:40 -05:00
Ian Littman 518a4a1f98 Use previous package filename for activity if installer edit doesn't change the package (#22177)
Also adds a line in the makefile help for generate-doc, as it took me
way too long to find that command this time.

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
2024-09-17 15:21:25 -05:00
Lucas Manuel Rodriguez 18f010f228 Update fleetdm/fleetctl, fleetdm/wix and fleetdm/bomutils docker images (#21063)
#20571

## Summary of changes

We have a few moving parts in fleetctl land (`fleetdm/wix` is used to
build `msi`s and `fleetdm/bomutils` is used to build `pkg`s, and
`fleetdm/fleetctl` can be used to build packages using docker, no need
for fleetctl executable):
```mermaid
graph LR

fleetctl_exec[fleetctl<br>executable];
wix_image[fleetdm/wix<br>docker image];
bomutils_image[fleetdm/bomutils<br>docker image];
fleetctl_image[fleetdm/fleetctl<br>docker image];

fleetctl_exec -- uses --> wix_image;

fleetctl_image -- COPY dependencies<br>FROM --> wix_image;

fleetctl_exec -- uses --> bomutils_image;

fleetctl_image -- COPY dependencies<br>FROM --> bomutils_image;
```
So, we'll need to update the three images: `fleetdm/bomutils`,
`fleetdm/wix` & `fleetdm/fleetctl`.

- `tools/bomutils-docker/Dockerfile`, `tools/wix-docker/Dockerfile` and
`tools/fleetctl-docker/Dockerfile`: Updating the base image to fix the
CRITICAL vulnerabilities.
- Modified existing+unused
`.github/workflows/build-and-check-fleetctl-docker-and-deps.yml` to run
every day to check for CRITICAL vulnerabilities in `fleetdm/wix`,
`fleetdm/bomutils` and `fleetdm/fleetctl`.
- `.github/workflows/goreleaser-fleetctl-docker-deps.yaml`:
`fleetdm/bomutils` and `fleetdm/wix` were pushed manually a few years
ago (most likely by Zach), so I've added a new action to release them
when we have changes to release (like now). It will basically release
`fleetctl/bomutils` and `fleetdm/wix` when pushing a tag of the form
`fleetctl-docker-deps-*` (we'll need to protect such tag prefix).
- Changes in `.github/workflows/test-native-tooling-packaging.yml` to
build `fleetdm/bomutils` and `fleetdm/wix` for `fleetdm/fleetctl` to use
them instead of the ones in docker hub.

--

Build before upgrading `debian:stable-slim`:

https://github.com/fleetdm/fleet/actions/runs/10255391418/job/28372231837
![Screenshot 2024-08-05 at 5 24
25 PM](https://github.com/user-attachments/assets/8a7d3576-3eb6-474f-989a-079873fca4fa)

Build after upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255550034

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-08-20 14:07:59 -03:00
Lucas Manuel Rodriguez ab7df5155d Use docker compose on CI instead of docker-compose (#21017)
After this is merged I'll cherry pick to `minor-fleet-4.55.0`.
2024-08-02 18:12:36 -03:00
Roberto Dip 7a080a9b36 use Escrow Buddy to rotate FileVault keys on macOS (#20842)
back-end and agent part of #13157

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-07-31 16:59:30 -03:00
ecf2346ace Add support for Linux ARM64 (#19931)
#1845

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-17 16:07:59 -04:00
Victor Lyuboslavsky 8d253fe19a Updating user now updates activity feed. (#18962)
#18766
Fixed a bug where updating user via `/api/v1/fleet/users/:id` endpoint
sometimes did not update the activity feed and returned the un-updated
user object.

You must use a DB configuration with a replica to reproduce the issue.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-14 15:06:23 -05:00
Victor Lyuboslavsky 51f8e1de67 fleetd-chrome 1.3.0 release (#18546)
#18466 

The beta release is already available at:
https://chrome-beta.fleetdm.com/updates.xml
2024-04-29 10:55:37 -05:00