Allen HouchinsandEric
19af21dd1a
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** N/A ## What this does The `/query-generator` page's osquery-SQL-generation step ([get-llm-generated-sql.js](website/api/controllers/query-generator/get-llm-generated-sql.js)) was on `claude-sonnet-4-6`, which is now one generation behind. This PR: - Bumps that call to `claude-sonnet-5`. The schema-filtration step stays on `claude-haiku-4-5`, which is already the latest Haiku release, so no change needed there. - Adds `effort` support to the shared [`ai.prompt` helper](website/api/helpers/ai/prompt.js), forwarded as `output_config.effort` on Anthropic requests, and sets it to `"low"` for the SQL-generation call. Effort controls how much the model deliberates (and how many tokens/how much latency that costs). `"low"` was chosen because the Haiku pre-filtering step already narrows the osquery schema down to relevant tables, so the Sonnet step isn't starting from scratch and doesn't need to spend much effort re-deriving that context. - Bumps `max_tokens` in the Anthropic branch of the helper from 4096 to 8192. Claude Sonnet 5 turns on adaptive thinking by default when the `thinking` param is omitted (which this helper does), and `max_tokens` is a hard cap on *total* output including thinking tokens — at 4096 there was a real risk of thinking tokens eating into the budget and truncating the JSON response the SQL step needs to return. - **Fixes a pre-existing bug found while making the above changes:** the `sqlReport` call passed the system prompt as a bare object-shorthand key named `systemPromptForQueryGeneration`, but the `ai.prompt` helper's declared input is `systemPrompt`. Sails silently drops unrecognized keys passed to `.with(...)`, so the "Return ONLY a raw JSON object..." system prompt was never actually reaching the model for this call. This has been broken since the query generator was switched to Anthropic (`f7c20c4731`); the sibling `filteredTables` call above it was unaffected since it passes `systemPrompt` positionally. Now fixed to `systemPrompt: systemPromptForQueryGeneration`. ## Why Claude Sonnet 5 follows structured/constrained instructions (don't alias tables, use `LIKE` with wildcards, only reference documented columns, etc.) more literally than 4.6, which should make the generated SQL more reliable. It's priced the same or cheaper than 4.6 during the current introductory period. ## Trade-offs called out for review - Thinking being on by default adds some latency versus the old (thinking-off) behavior on 4.6. This call is not currently streamed (`sails.helpers.http.post`, single blocking call over a socket), so any added thinking time is invisible wait time for the user rather than a visible "thinking" indicator. `effort: "low"` should keep this modest, but worth confirming with a manual QA pass on a few representative questions before merging. - Only the SQL-generation call was migrated. The schema-filtration call also runs on an Anthropic model, but Haiku 4.5 doesn't support `output_config.effort` (added `effort` is a no-op if passed to it), so it was left as-is. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. - [x] Timeouts are implemented and retries are limited to avoid infinite loops - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [ ] Added/updated automated tests - [ ] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * Improved AI-generated SQL responses with an updated language model. * Added adaptive effort controls for supported AI requests. * Increased response capacity to support more detailed generated results. * Improved handling of AI responses to provide more reliable results when content includes different response formats. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Eric <eashaw@sailsjs.com>
fleetdm.com
This is where the code for the public https://fleetdm.com website lives.
Bugs
To report a bug or make a suggestion for the website, create an issue in the fleet GitHub repository.
Testing locally
See https://fleetdm.com/handbook/engineering#test-fleetdm-com-locally
Deploying the website
To deploy changes to the website to production, merge changes to the main branch. If the changes affect the website's code, or touch any files that the website relies on to build content, such as the query library, osquery schema, docs, handbook, articles, etc., then the website will be redeployed.
Wondering how this works? This is implemented in a GitHub action in this repo. Check out the code there to see how it works! For help understanding what
sails runandnpm runcommands in there do, check the scripts inwebsite/package.jsonand inwebsite/scripts/.
Changing the database schema
To deploy new code to production that relies on changes to the database schema or other external systems (e.g. Stripe), first put the website in "maintenance mode" in Heroku. Then, make your changes in the database schema. Next, if you have a script to fix/migrate existing data, go ahead and run it now. (e.g. sails run fix-or-migrate-existing-data). Then, merge your changes and wait for the deploy to finish. Finally, switch off "maintenance mode" in Heroku.
Note that entering maintenance mode prevents visitors from using the website, so it should be used sparingly, and ideally at low-traffic times of day.
Warning: Doing an especially sensitive schema migration? There is a potential timing issue to consider, thanks to an infrastructure change that eliminated downtime during deploys by using Heroku's built-in support for hot-swapping. Read more in https://github.com/fleetdm/fleet/issues/6568#issuecomment-1211503881
Wiping the production database
I hope you know what you're doing. The "easiest" kind of database schema migration:
sails_datastores__default__url='REAL_DB_URI_HERE' sails run wipe
Then when you see the sailboat, hit CTRL+C to exit. All done!