Victor Lyuboslavsky
6fc7132350
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #35239 Docs PR: #39770 ## Remote trigger approach When FLEET_VULNERABILITIES_DISABLE_SCHEDULE=true, the main Fleet server registers a RemoteTriggerSchedule instead of the real vulnerability schedule. When a user runs fleetctl trigger --name=vulnerabilities: 1. Main server: RemoteTriggerSchedule.Trigger() inserts a cron_stats record with status=queued. 2. Worker server: The vulnerability schedule runs with WithTriggerPollInterval(60s), which starts a poll goroutine that checks the DB every 60s for queued records. 3. Pickup: When the poll goroutine finds a queued record, it sends the stats ID on the trigger channel (non-blocking). 4. Execution: The trigger handler acquires the lock, claims the record via ClaimCronStats (updating status to pending and instance to the actual worker ID), runs all jobs, and marks it completed. Key details: - The trigger channel carries an int: 0 for in-process triggers, >0 for DB-polled stats IDs. This lets runWithStats reuse the existing record instead of inserting a new one. - Both Schedule.Trigger() and RemoteTriggerSchedule.Trigger() treat pending and queued as conflicts to prevent duplicate runs. - Queued records expire after 2 hours via CleanupCronStats, same as pending records. - The poll goroutine only signals; it doesn't modify DB state. The handler claims when ready. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for remote trigger execution in vulnerability scheduling workflows. * Implemented periodic polling mechanism to detect and process externally triggered vulnerability scans. * **Bug Fixes** * Enhanced trigger status tracking to properly handle queued scan jobs. * **Improvements** * Strengthened scheduling system with improved timeout and cancellation management capabilities. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
schedule: the Fleet cron job machinery
Fleet has several pieces of functionality that are implemented as cron jobs, which run on a
schedule. Package schedule implements the machinery needed for queueing and running these jobs.
List of cron jobs
See server/fleet/cron_schedules.go for a list of the currently implemented cron jobs and information about what they do.
Cron jobs are created and registered in the cmd/fleet package because they have to be run at
server start. The actual implementation of the cron job logic is usually elsewhere however,
typically in a service layer method (and related datastore methods).
How to add a new cron job
See this PR for a nice example of how to add a simple cron job.
- Do you need a new cron job? You can add sub-jobs to an existing cron job; for example, if
you're adding some functionality for cleaning up unused data, you might want to implement it as a
sub-job in the
cleanups_then_aggregationcron. - Add a cron job name. If you determine that you do need a new cron job, create a descriptive name in cron_schedules.go. Make sure you leave a comment explaining what the job does.
- Implement your functionality. Do this wherever it makes sense. In the example PR, the
functionality exists in the
server/mdm/maintainedapps/ingest.gofile. However, you'll most likely implement a service layer method and related datastore layer methods. - Add a function that returns a
*schedule.Scheduleincmd/fleet/cron.go. This function will be used to register your cron job so it can actually run. This function should call whatever you implemented in step 3. This is also where you can set the interval on which your cron job will run. - Register the cron job in
cmd/fleet/serve.go. You'll usecronSchedules.StartCronScheduleto register the cron job by passing it an anonymous function that calls the function you wrote in step 3.