For #27284
This PR:
- Adds SCIM as a fallback for username during macOS end user
authentication during setup experience
- Adds SCIM/endUsers details to host details
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
I created and tested the fix before seeing the [comment in this
issue](https://github.com/fleetdm/fleet/issues/24417).
Feel free to merge the fix or close this out but I made this pull
request since we had another customer report this issue today.
For #27601
## Details
A couple of updates to the save/edit Query screens to bring them in line
with how save/edit Policy screens work, as described in [the
Figma](https://www.figma.com/design/LzGmucdJQgbQCuHMZhGKCM/-24097-Custom-targets--labels--for-policies?node-id=2-130&p=f&t=9iFB2FRnIkn98NSg-0):
* Moved platform selector out from under Advanced
* Use checkboxes for selecting platforms
* Disable Save button when no platforms are selected
* In the "Save new query" modal, pre-select the platforms that are
compatible with the query
## Testing
1. Add a new query with the SQL "SELECT * FROM privacy_preferences;".
Verify that only ChromeOS is checked in the modal when you click Save.
2. Verify that if you deselect ChromeOS (so that no platforms are
selected), the save button is disabled in the modal.
3. Select one or more platforms and save the query.
4. After saving the query, edit it and verify that the platforms you
selected are checked.
5. Verify that deselecting all platforms disables the edit button on the
Edit Query screen.
6. Select different platforms, save, and refresh to verify that the
platforms you selected were persisted.
For #27235.
This is updating the dependency after fixing the bug in
https://github.com/elastic/apm-agent-go/pull/1707.
The issue with the upload of medium/big sized packages can be reproduced
by running `fleet serve` with `FLEET_LOGGING_TRACING_ENABLED=1
FLEET_LOGGING_TRACING_TYPE=elasticapm`. We have reproduced this issue in
Dogfood only because it's the only production environment where APM
tracing is enabled. We also have APM enabled in our internal
load-testing to collect data during troubleshooting.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.
Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
https://github.com/fleetdm/fleet/issues/27927
## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>
## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
Release notes: https://tip.golang.org/doc/go1.24
> Go modules can now track executable dependencies using tool directives
in go.mod. This removes the need for the previous workaround of adding
tools as blank imports to a file conventionally named “tools.go”. The go
tool command can now run these tools in addition to tools shipped with
the Go distribution. For more information see [the
documentation](https://tip.golang.org/doc/modules/managing-dependencies#tools).
The new -tool flag for go get causes a tool directive to be added to the
current module for named packages in addition to adding require
directives.
I ran:
```
go get -tool github.com/fleetdm/fleet/v4/server/goose
go get -tool github.com/kevinburke/go-bindata
go get -tool github.com/quasilyte/go-ruleguard/dsl
go rm tools.go
go mod tidy
```
`make deps-go` was failing in CI because of the removal of `tools.go`
(my guess is that `go get .` was a nop because there was nothing in `.`
to download).
So, taking the chance of removing `deps-go` because `go` will download
packages during the build process. AFAICS there's no need to download
everything beforehand.
For #27889
This PR introduces several improvements to the Makefile/`fdm` tool for
development:
### `fdm serve` (alias `fdm up`)
Starts a local Fleet server (building the binary first). The first time
this is called, it will start the server on `localhost:8080` with the
`--dev` and `--dev_license` flags, but the command accepts all of the
options that you can pass to `fleet serve`. If you pass options to `fdm
serve`, then subsequent invocations _without_ options will replay your
last command. Additionally, `fdm serve` supports the following:
- `--use-ip`: start the local server on your system's local IP address
rather than `localhost`. This makes it easier to point VMs on your
system to the fleet server to act as hosts.
- `--no-build`: don't rebuild the fleet binary before starting the
server.
- `--no-save`: don't save the current command for future invocations
(useful for scripting)
- `--show`: show options for the last-invoked `fdm serve` command
- `--reset`: reset the options for `fdm serve`. The next time `fdm
serve` is invoked, it will use the default options.
- `--help`: show all of the Fleet server options
### `fdm snapshot` improvements
* Added `fdm snap` alias
* Tracks the name of the last snapshot saved, to use as the default for
`fdm restore`
* Suppresses the "don't use password in CLI" warning when saving the
snapshot
### `fdm restore` improvements
* Added `--prep` / `--prepare` option to run db migrations after
restoring snapshot.
* Improved UI (more options displayed, and clearer indicator for
selected option)
* Now defaults to last snapshot restored
Closes: https://github.com/fleetdm/confidential/issues/9974
Changes:
- Removed the calculated max height of the sidebar on osquery schema
table pages
- Update the platform filters on osquery schema table pages, vitals
pages, query library, and policy library to scroll with the page.
- Updated /vitals/* and /tables/* pages to scroll users past the page
headline when they switch platforms.
- Updated the query library and policy library pages to scroll users to
the top of the list of policies/queries when they switch platforms (If
they have scrolled the list)
- updated the script and policy names to reflect naming convention and
look better in Fleet Desktop
- removed script from Servers (canary); since Fleet Desktop is not
enabled on Servers, it leads to awkward conversations when demo'ing with
customers.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
---------
Co-authored-by: Brock Walters <153771548+nonpunctual@users.noreply.github.com>
For #26716.
Also moves the article to an OS-independent URL, updates links from
elsewhere, and adds a bit more internal-link juice for install
automation.
---------
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Notarization from the fleetctl-docker image is broken actually:
```
fleetctl package --type=pkg --fleet-url=myurl --enroll-secret=mysecret --macos-devid-pem-content=XYZ --notarize --app-store-connect-api-key-id=XYZ --app-store-connect-api-key-issuer=XYZ --app-store-connect-api-key-content=XYZ
[..]
transporter error> Package Summary:
transporter error>
transporter error> 1 package(s) were not uploaded because they had problems:
transporter error> /tmp/apple-codesign-QAsKT8/17081d03-fdc8-46cd-873a-2970f7be9c7c.itmsp - Error Messages:
transporter error> Notarization of MacOS applications using altool has been decommissioned. Please use notarytool. See: https://developer.apple.com/documentation/technotes/tn3147-migrating-to-the-latest-notarization-tool (4200)
transporter error> [2024-11-15 13:35:47 UTC] <main> DBG-X: Returning 1
Error: I/O error: command ["/usr/local/bin/iTMSTransporter", "-m", "upload", "-apiIssuer", "XYZ", "-apiKey", "XYZ", "-f", "/tmp/apple-codesign-QAsKT8/17081d03-fdc8-46cd-873a-2970f7be9c7c.itmsp", "-vp", "json"] exited with code 1
Error: rcodesign notarize: exit status 1
```
Luckily, bumping `rcodesign` version is enough to make it work again.
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Jordan Montgomery