Juan Fernandez
7023c5be9a
Fixes #48497 When a cron run's context was cancelled mid-flight (e.g. the instance received SIGTERM during graceful shutdown), the stats row was left "pending" because the terminal-status write failed on the cancelled context. CleanupCronStats would later reap it to "expired", hiding the fact that the run was interrupted and discarding the captured job errors. Record the terminal status on a context detached from cancellation (context.WithoutCancel with a bounded timeout) so an interrupted run persists its outcome. The run is marked "canceled" only when the context was cancelled AND a job actually reported an error, so a run whose jobs all finished cleanly is still "completed" even if cancellation merely raced the end of the run.
schedule: the Fleet cron job machinery
Fleet has several pieces of functionality that are implemented as cron jobs, which run on a
schedule. Package schedule implements the machinery needed for queueing and running these jobs.
List of cron jobs
See server/fleet/cron_schedules.go for a list of the currently implemented cron jobs and information about what they do.
Cron jobs are created and registered in the cmd/fleet package because they have to be run at
server start. The actual implementation of the cron job logic is usually elsewhere however,
typically in a service layer method (and related datastore methods).
How to add a new cron job
See this PR for a nice example of how to add a simple cron job.
- Do you need a new cron job? You can add sub-jobs to an existing cron job; for example, if
you're adding some functionality for cleaning up unused data, you might want to implement it as a
sub-job in the
cleanups_then_aggregationcron. - Add a cron job name. If you determine that you do need a new cron job, create a descriptive name in cron_schedules.go. Make sure you leave a comment explaining what the job does.
- Implement your functionality. Do this wherever it makes sense. In the example PR, the
functionality exists in the
server/mdm/maintainedapps/ingest.gofile. However, you'll most likely implement a service layer method and related datastore layer methods. - Add a function that returns a
*schedule.Scheduleincmd/fleet/cron.go. This function will be used to register your cron job so it can actually run. This function should call whatever you implemented in step 3. This is also where you can set the interval on which your cron job will run. - Register the cron job in
cmd/fleet/serve.go. You'll usecronSchedules.StartCronScheduleto register the cron job by passing it an anonymous function that calls the function you wrote in step 3.