Files
fleet/.github/workflows/dependabot-qa-ticket.yml
Sharon Katz 0fd5739514 Auto-create QA tickets for Dependabot PRs (#48008)
**Related issue:** Resolves #48007

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## What

When Dependabot opens a code-dependency PR (Go or npm), a GitHub Actions
workflow now auto-creates a QA tracking issue assigned to
@AndreyKizimenko and places it on the `:help-qa` board in "Ready"
status. Also exempts `dependabot[bot]` from the PR template check.

### Files changed

- **`.github/workflows/dependabot-qa-ticket.yml`** (new) - The core
workflow. Triggers on `pull_request_target[opened]`, filters for
code-dependency Dependabot PRs (Go modules + npm, excluding
website/slackbot), creates a QA tracking issue, comments on the PR, and
adds the issue to the `:help-qa` project board.
- **`.github/workflows/check-pr-template.yml`** - Exempts
`dependabot[bot]` from the PR description template check (Dependabot PRs
have auto-generated descriptions).
- **`.github/zizmor-gate.yml`** - Suppresses two expected zizmor
findings: `dangerous-triggers` (uses `pull_request_target` but never
checks out PR code) and `bot-conditions` (gates on `user.login` which is
not spoofable for same-repo Dependabot PRs).
- **`changes/48007-dependabot-qa-ticket`** - Release note.

## Security review

This workflow uses `pull_request_target`, a known attack surface for
GitHub Actions. A dedicated security audit was performed covering all
standard threat vectors:

| Area | Status | Details |
|------|--------|---------|
| **Code checkout injection** | SAFE | No `actions/checkout` or `git
clone`. The workflow only calls the GitHub API via
`actions/github-script`. The inline JavaScript is embedded in the
base-branch YAML, not loaded from PR code. |
| **Script/template injection** | SAFE | PR data (`pr.title`,
`pr.head.ref`) is used in JavaScript template literals passed to the
GitHub API as strings. There is no shell interpolation, no `eval()`, no
`${{ }}` expressions in `run:` blocks. |
| **Bot identity spoofing** | SAFE | Gates on
`github.event.pull_request.user.login == 'dependabot[bot]'`, which is
server-set by GitHub and cannot be spoofed. Uses `user.login`
(immutable) rather than `github.actor` (which can differ on re-runs).
The `[bot]` suffix is reserved for GitHub App accounts. |
| **Secret exposure** | SAFE | `FLEET_GITHUB_TOKEN_PROJECTS` is passed
via `env:` block, accessed as `process.env.PROJECT_TOKEN`, and used only
as an argument to `getOctokit()`. Never logged, never interpolated into
output. |
| **Permission scope** | MINIMAL | Top-level: `contents: read`.
Job-level: `contents: read` + `issues: write`. No `pull-requests: write`
(PR comments use the Issues API). |
| **Idempotency** | SAFE | Before creating an issue, searches for an
existing open issue with the same title and `dependabot` label. Prevents
duplicates on workflow re-runs. |
| **Markdown injection** | LOW (theoretical) | `pr.title` is
interpolated into issue title/body. Dependabot controls the title format
("Bump X from Y to Z") and package names are registry-constrained. Not
exploitable without compromising Dependabot itself. |
| **Denial of service** | LOW (theoretical) | Bounded by Dependabot's
own rate limits (default 5 PRs/ecosystem/day) plus the idempotency
guard. |
| **Zizmor waivers** | JUSTIFIED | `dangerous-triggers`: no code
checkout, API-only. `bot-conditions`: uses `user.login` not
`github.actor`. Both include inline rationale comments. |
| **check-pr-template.yml** | SAFE | `AUTHOR` is passed via `env:` (not
`${{ }}` in shell), double-quoted in comparison. `user.login` is
GitHub-constrained to alphanumeric + hyphens. |

**Actions are SHA-pinned** with version comments:
`step-security/harden-runner@9af8...` (v2.19.4) and
`actions/github-script@60a0...` (v7.0.1).

## Testing

- [x] QA'd all new/changed functionality manually

Since this is a GitHub Actions workflow that triggers on Dependabot PR
opens, it cannot be fully end-to-end tested without a real Dependabot
PR. The following offline validations were performed:

### 1. YAML syntax validation
```
$ ruby -e "require 'yaml'; YAML.safe_load(File.read('.github/workflows/dependabot-qa-ticket.yml')); puts 'YAML: valid'"
YAML: valid
```

### 2. Verified hardcoded GraphQL IDs against the live GitHub API

**Project board ID** (`PVT_kwDOBDAnic4A5q11`):
```
$ gh api graphql -f query='query { node(id: "PVT_kwDOBDAnic4A5q11") { ... on ProjectV2 { title number url } } }'
-> title: ":help-qa", number: 85, url: "https://github.com/orgs/fleetdm/projects/85"
```

**Status field ID** (`PVTSSF_lADOBDAnic4A5q11zgubZ6Y`) and option ID
(`f75ad846`):
```
$ gh api graphql -f query='query { node(id: "PVTSSF_lADOBDAnic4A5q11zgubZ6Y") { ... on ProjectV2SingleSelectField { name options { id name } } } }'
-> field name: "Status", option f75ad846 = "Ready" 
```

### 3. Verified assignee and label exist
```
$ gh api users/AndreyKizimenko --jq '.login'   -> AndreyKizimenko 
$ gh label list --search dependabot             -> dependabot 
```

### 4. Branch filtering logic unit tested

Extracted the JavaScript filter logic and ran it against 8 test cases
covering single-package updates, grouped updates, and exclusions:

```
$ node -e "<test script>"
PASS | dependabot/go_modules/golang.org/x/crypto-0.24.0      | match (Go single)
PASS | dependabot-go_modules-google.golang.org-grpc-1.82.1    | match (Go grouped)
PASS | dependabot/npm_and_yarn/typescript-5.5.0               | match (npm single)
PASS | dependabot-npm_and_yarn-postcss-8.5.18                 | match (npm grouped)
PASS | dependabot/npm_and_yarn/website/next-14.0.0            | skip (website single)
PASS | dependabot-npm_and_yarn-website-next-14.0.0            | skip (website grouped)
PASS | dependabot/npm_and_yarn/fleet-slackbot/axios-1.7.0     | skip (slackbot single)
PASS | dependabot/github_actions/actions/checkout-4            | skip (GitHub Actions)
8/8 passed
```

### 5. Validated filter against real Dependabot PRs

Checked the 20 most recent Dependabot PRs (`gh pr list --author
'app/dependabot' --state all --limit 20`) to confirm real branch names
match the filter patterns. This revealed three bugs that were fixed:

1. **Go branch prefix was wrong**: Real branches use
`dependabot/go_modules/`, not `dependabot/gomod/`. Fixed.
2. **Grouped updates use dashes**: Real grouped branches like
`dependabot-go_modules-...` and `dependabot-npm_and_yarn-...` were not
matched. Added dash-variant prefix checks and exclusions.
3. **Status label mismatch**: The constant was named `AWAITING_QA` but
the option ID maps to "Ready" on the board (there is no "Awaiting QA"
status). Renamed to `READY` and updated comments.

### 6. CI checks
All CI checks pass (CodeQL, zizmor, dependency-review, build-binaries,
publish).

**Note:** The project board integration requires a
`FLEET_GITHUB_TOKEN_PROJECTS` repo secret with Organization Projects
read/write scope. Without it, the issue is still created but not added
to the board (a warning is logged).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an automated GitHub Actions workflow for eligible Dependabot
update pull requests.
* Creates (and deduplicates) labeled QA issues with pull request details
and checklist content, and posts the issue back to the pull request.
* Optionally links new QA issues to project tracking with an “Awaiting
QA” status when configured.

* **Chores**
  * Hardened the automation workflow’s runner and permissions.
* Updated safety-gate configuration to waive checks for the new
Dependabot QA workflow.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-28 13:43:52 -04:00

174 lines
6.9 KiB
YAML

name: Create QA ticket for Dependabot PRs
# When Dependabot opens a code-dependency PR, create a QA issue on the
# #g-orchestration board so the team can verify that the bump introduces no
# behavioral regressions (see postmortem for #45763).
on:
pull_request_target:
types: [opened]
permissions:
contents: read
jobs:
create-qa-ticket:
if: >
github.event.repository.owner.login == 'fleetdm' &&
github.event.pull_request.user.login == 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@9af89fc71515a100421586dfdb3dc9c984fbf411 # v2.19.4
with:
egress-policy: audit
- name: Create QA issue and add to project board
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
env:
PROJECT_TOKEN: ${{ secrets.FLEET_GITHUB_TOKEN_PROJECTS }}
with:
script: |
const pr = context.payload.pull_request;
const branch = pr.head.ref;
// Only process code dependency updates (Go modules, npm root).
// Skip: github-actions, docker, website npm, slackbot npm.
// Dependabot uses slashes for single-package updates and dashes for grouped updates:
// single: dependabot/go_modules/golang.org/x/crypto-0.35.0
// grouped: dependabot-go_modules-google.golang.org-grpc-1.82.1
const isGomod = branch.startsWith('dependabot/go_modules/')
|| branch.startsWith('dependabot-go_modules-');
const isNpm = (branch.startsWith('dependabot/npm_and_yarn/')
|| branch.startsWith('dependabot-npm_and_yarn-'))
&& !branch.includes('/website/')
&& !branch.includes('/fleet-slackbot/')
&& !branch.includes('-website-')
&& !branch.includes('-fleet-slackbot-');
if (!isGomod && !isNpm) {
console.log(`Skipping non-code dependency PR (branch: ${branch})`);
return;
}
const ecosystem = branch.includes('go_modules') ? 'Go' : 'npm';
const issueTitle = `QA: ${pr.title}`;
// Idempotency: check if a QA issue already exists for this PR
// (guards against duplicate tickets on workflow re-runs).
const existing = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
labels: 'dependabot',
state: 'open',
per_page: 100,
});
const duplicate = existing.data.find(i => i.title === issueTitle);
if (duplicate) {
console.log(`QA issue already exists: #${duplicate.number} — skipping.`);
return;
}
// Create the QA tracking issue.
const issue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: issueTitle,
body: [
`## Dependabot dependency update — QA review needed`,
``,
`| Field | Value |`,
`|-------|-------|`,
`| **PR** | #${pr.number} |`,
`| **Ecosystem** | ${ecosystem} |`,
`| **Branch** | \`${branch}\` |`,
``,
`### Why this needs QA`,
``,
`Dependabot PRs update third-party dependencies. Even minor/patch bumps can`,
`change runtime behavior (see #45763 for a real example where a dependency`,
`bump silently broke SCEP certificate issuance). Please verify that the`,
`updated dependency does not cause regressions in related functionality.`,
``,
`### What to check`,
``,
`1. Review the PR diff and the dependency's changelog/release notes`,
`2. Identify which Fleet features use this dependency`,
`3. Test those features for behavioral changes`,
``,
`---`,
`*This issue was automatically created by the dependabot-qa-ticket workflow.*`,
].join('\n'),
labels: ['dependabot'],
assignees: ['AndreyKizimenko'],
});
console.log(`Created issue #${issue.data.number}`);
// Comment on the PR linking to the QA issue.
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
body: `QA ticket created: #${issue.data.number}`,
});
// --- Project board: add to #g-orchestration and set status to "Inbox" ---
// Requires a PAT stored as FLEET_GITHUB_TOKEN_PROJECTS with
// read/write access to organization projects.
const projectToken = process.env.PROJECT_TOKEN;
if (!projectToken) {
core.warning(
'FLEET_GITHUB_TOKEN_PROJECTS secret is not configured — ' +
'issue was created but not added to the project board. ' +
'Create a fine-grained PAT with Organization Projects read/write scope.'
);
return;
}
const { getOctokit } = require('@actions/github');
const projectOctokit = getOctokit(projectToken);
const PROJECT_ID = 'PVT_kwDOBDAnic4A4BEx'; // #g-orchestration (#71)
const STATUS_FIELD = 'PVTSSF_lADOBDAnic4A4BExzgtDlHw';
const INBOX = 'f33a1ec5';
// Add the issue to the project.
const addResult = await projectOctokit.graphql(`
mutation($projectId: ID!, $contentId: ID!) {
addProjectV2ItemById(input: {projectId: $projectId, contentId: $contentId}) {
item { id }
}
}
`, {
projectId: PROJECT_ID,
contentId: issue.data.node_id,
});
const itemId = addResult.addProjectV2ItemById.item.id;
console.log(`Added to #g-orchestration board (item ${itemId})`);
// Set status to "Inbox".
await projectOctokit.graphql(`
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId
itemId: $itemId
fieldId: $fieldId
value: {singleSelectOptionId: $optionId}
}) {
projectV2Item { id }
}
}
`, {
projectId: PROJECT_ID,
itemId: itemId,
fieldId: STATUS_FIELD,
optionId: INBOX,
});
console.log('Status set to "Inbox"');