Scott Gress
1cd37ef966
For #25555 # Checklist for submitter - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) This PR updates the `NewLabel` service to use the `UpdateLabelMembershipByHostIDs` method previously added by @jacobshandling rather than using `ApplyLabels`. The latter method has performance issues when adding large numbers of hosts at once to a manual label (see #25555) because it does an expensive lookup of host names before transforming those into Fleet host IDs. The new code skips the middleman and transforms host identifiers directly to Fleet host IDs, and does so using a batching strategy to ensure the queries don't get too large. This PR does update `UpdateLabelMembershipByHostIDs` slightly to return an updated Label object and host IDs array, as this is the expected return value for `NewLabel`. I update the method's tests accordingly. I don't think any new tests for `NewLabel` are needed as it should have the same functionality and return values. ## Manual Testing On the main branch, I launched my local MySQL with the thread stack size set to the minimal allowed, and used the API to try and create a new label with 5,000 hosts attached, and received a 422 response from the server. Server logs showed: ``` level=error ts=2025-01-28T15:08:20.465401Z component=http user=scott@fleetdm.com method=POST uri=/api/latest/fleet/labels took=16.610292ms err="get hostnames by identifiers: Error 1436 (HY000): Thread stack overrun: 111136 bytes used of a 131072 byte stack, and 20000 bytes needed. Use 'mysqld --thread_stack=#' to specify a bigger stack." ``` On this branch, I kept the same MySQL settings and tried my API request again and it was successful: <img width="776" alt="image" src="https://github.com/user-attachments/assets/c4f0f52b-4d09-457b-8096-4dd3a747b1f4" /> ## QA The script I used to create a new manual label with lots of hosts is at: https://gist.github.com/sgress454/84f12064c437da456c456e25c26d9069 To run it, first grab a bearer token from any API request by opening the network tab, clicking a Fleet API request, and in the headers tab scrolling down to Authorization: <img width="892" alt="image" src="https://github.com/user-attachments/assets/5680f3bf-8db8-469a-9f03-000b86622c04" /> (only take the part _after_ "Bearer") Then download the script from that gist and in its folder run: ``` NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js <the bearer token> "<a label name>" ``` e.g. ``` NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js U3HpbdtadmJXGKYSB0U/PbwfOpHbBt7FpkWmGKKYolOO1moLNZA6XxP+QO5LVukvAotZ7d+JbNUEEhYHZtxoqg== "some test label" ``` This will invoke the API on https://localhost:8080 and try to add 5000 hosts a new label "some test label". If you need to change the # of hosts or the url of the server, there are additional arguments: ``` NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js <the bearer token> "<a label name>" <number of hosts> <url> ``` e.g. ``` NODE_TLS_REJECT_UNAUTHORIZED=0 node ./add_hosts_to_label.js U3HpbdtadmJXGKYSB0U/PbwfOpHbBt7FpkWmGKKYolOO1moLNZA6XxP+QO5LVukvAotZ7d+JbNUEEhYHZtxoqg== "some test label" 10000 https://foo.bar ```
News · Report a bug · Handbook · Why open source? · Art
Open-source platform for IT and security teams with thousands of computers. Designed for APIs, GitOps, webhooks, YAML, and humans.
What's it for?
Organizations like Fastly and Gusto use Fleet for vulnerability reporting, detection engineering, device management (MDM), device health monitoring, posture-based access control, managing unused software licenses, and more.
Explore data
To see what kind of data you can use Fleet to gather, check out the table reference documentation.
Out-of-the-box policies
Fleet includes out-of-the box support for all CIS benchmarks for macOS and Windows, as well as many simpler queries.
Take as much or as little as you need for your organization.
Supported platforms
Here are the platforms Fleet currently supports:
- Linux (all distros)
- macOS
- Windows
- Chromebooks
- Amazon Web Services (AWS)
- Google Cloud (GCP)
- Azure (Microsoft cloud)
- Data centers
- Containers (kube, etc)
- Linux-based IoT devices
Lighter than air
Fleet is lightweight and modular. You can use it for security without using it for MDM, and vice versa. You can turn off features you are not using.
Openness
Fleet is dedicated to flexibility, accessibility, and clarity. We think everyone can contribute and that tools should be as easy as possible for everyone to understand.
Good neighbors
Fleet has no ambition to replace all of your other tools. (Though it might replace some, if you want it to.) Ready-to-use, enterprise-friendly integrations exist for Snowflake, Splunk, GitHub Actions, Vanta, Elastic Jira, Zendesk, and more.
Fleet plays well with Munki, Chef, Puppet, and Ansible, as well as with security tools like Crowdstrike and SentinelOne. For example, you can use the free version of Fleet to quickly report on what hosts are actually running your EDR agent.
Free as in free
The free version of Fleet will always be free. Fleet is independently backed and actively maintained with the help of many amazing contributors.
Longevity
The company behind Fleet is founded (and majority-owned) by true believers in open source. The company's business model is influenced by GitLab (NYSE: GTLB), with great investors, happy customers, and the capacity to become profitable at any time.
In keeping with Fleet's value of openness, Fleet Device Management's company handbook is public and open source. You can read about the history of Fleet and osquery and our commitment to improving the product.
Is it any good?
Fleet is used in production by IT and security teams with thousands of laptops and servers. Many deployments support tens of thousands of hosts, and a few large organizations manage deployments as large as 400,000+ hosts.
Chat
Please join us in MacAdmins Slack or in osquery Slack.
The Fleet community is full of kind and helpful people. Whether or not you are a paying customer, if you need help, just ask.
Contributing 
The landscape of cybersecurity and IT is too complex. Let's open it up.
Contributions are welcome, whether you answer questions on Slack / GitHub / StackOverflow / LinkedIn / Twitter, improve the documentation or website, write a tutorial, give a talk at a conference or local meetup, give an interview on a podcast, troubleshoot reported issues, or submit a patch. The Fleet code of conduct is on GitHub.
What's next?
To see what Fleet can do, head over to fleetdm.com and try it out for yourself, grab time with one of the maintainers to discuss, or visit the docs and roll it out to your organization.
Production deployment
Fleet is simple enough to spin up for yourself. Or you can have us host it for you. Premium features are available either way.
Documentation
Complete documentation for Fleet can be found at https://fleetdm.com/docs.
License
The free version of Fleet is available under the MIT license. The commercial license is also designed to allow contributions to paid features for users whose employment agreements allow them to contribute to open source projects. (See LICENSE.md for details.)
Fleet is built on osquery, nanoMDM, Nudge, and swiftDialog.