**Related issue:** Resolves #48007 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/` - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## What When Dependabot opens a code-dependency PR (Go or npm), a GitHub Actions workflow now auto-creates a QA tracking issue assigned to @AndreyKizimenko and places it on the `:help-qa` board in "Ready" status. Also exempts `dependabot[bot]` from the PR template check. ### Files changed - **`.github/workflows/dependabot-qa-ticket.yml`** (new) - The core workflow. Triggers on `pull_request_target[opened]`, filters for code-dependency Dependabot PRs (Go modules + npm, excluding website/slackbot), creates a QA tracking issue, comments on the PR, and adds the issue to the `:help-qa` project board. - **`.github/workflows/check-pr-template.yml`** - Exempts `dependabot[bot]` from the PR description template check (Dependabot PRs have auto-generated descriptions). - **`.github/zizmor-gate.yml`** - Suppresses two expected zizmor findings: `dangerous-triggers` (uses `pull_request_target` but never checks out PR code) and `bot-conditions` (gates on `user.login` which is not spoofable for same-repo Dependabot PRs). - **`changes/48007-dependabot-qa-ticket`** - Release note. ## Security review This workflow uses `pull_request_target`, a known attack surface for GitHub Actions. A dedicated security audit was performed covering all standard threat vectors: | Area | Status | Details | |------|--------|---------| | **Code checkout injection** | SAFE | No `actions/checkout` or `git clone`. The workflow only calls the GitHub API via `actions/github-script`. The inline JavaScript is embedded in the base-branch YAML, not loaded from PR code. | | **Script/template injection** | SAFE | PR data (`pr.title`, `pr.head.ref`) is used in JavaScript template literals passed to the GitHub API as strings. There is no shell interpolation, no `eval()`, no `${{ }}` expressions in `run:` blocks. | | **Bot identity spoofing** | SAFE | Gates on `github.event.pull_request.user.login == 'dependabot[bot]'`, which is server-set by GitHub and cannot be spoofed. Uses `user.login` (immutable) rather than `github.actor` (which can differ on re-runs). The `[bot]` suffix is reserved for GitHub App accounts. | | **Secret exposure** | SAFE | `FLEET_GITHUB_TOKEN_PROJECTS` is passed via `env:` block, accessed as `process.env.PROJECT_TOKEN`, and used only as an argument to `getOctokit()`. Never logged, never interpolated into output. | | **Permission scope** | MINIMAL | Top-level: `contents: read`. Job-level: `contents: read` + `issues: write`. No `pull-requests: write` (PR comments use the Issues API). | | **Idempotency** | SAFE | Before creating an issue, searches for an existing open issue with the same title and `dependabot` label. Prevents duplicates on workflow re-runs. | | **Markdown injection** | LOW (theoretical) | `pr.title` is interpolated into issue title/body. Dependabot controls the title format ("Bump X from Y to Z") and package names are registry-constrained. Not exploitable without compromising Dependabot itself. | | **Denial of service** | LOW (theoretical) | Bounded by Dependabot's own rate limits (default 5 PRs/ecosystem/day) plus the idempotency guard. | | **Zizmor waivers** | JUSTIFIED | `dangerous-triggers`: no code checkout, API-only. `bot-conditions`: uses `user.login` not `github.actor`. Both include inline rationale comments. | | **check-pr-template.yml** | SAFE | `AUTHOR` is passed via `env:` (not `${{ }}` in shell), double-quoted in comparison. `user.login` is GitHub-constrained to alphanumeric + hyphens. | **Actions are SHA-pinned** with version comments: `step-security/harden-runner@9af8...` (v2.19.4) and `actions/github-script@60a0...` (v7.0.1). ## Testing - [x] QA'd all new/changed functionality manually Since this is a GitHub Actions workflow that triggers on Dependabot PR opens, it cannot be fully end-to-end tested without a real Dependabot PR. The following offline validations were performed: ### 1. YAML syntax validation ``` $ ruby -e "require 'yaml'; YAML.safe_load(File.read('.github/workflows/dependabot-qa-ticket.yml')); puts 'YAML: valid'" YAML: valid ``` ### 2. Verified hardcoded GraphQL IDs against the live GitHub API **Project board ID** (`PVT_kwDOBDAnic4A5q11`): ``` $ gh api graphql -f query='query { node(id: "PVT_kwDOBDAnic4A5q11") { ... on ProjectV2 { title number url } } }' -> title: ":help-qa", number: 85, url: "https://github.com/orgs/fleetdm/projects/85" ``` **Status field ID** (`PVTSSF_lADOBDAnic4A5q11zgubZ6Y`) and option ID (`f75ad846`): ``` $ gh api graphql -f query='query { node(id: "PVTSSF_lADOBDAnic4A5q11zgubZ6Y") { ... on ProjectV2SingleSelectField { name options { id name } } } }' -> field name: "Status", option f75ad846 = "Ready" ✅ ``` ### 3. Verified assignee and label exist ``` $ gh api users/AndreyKizimenko --jq '.login' -> AndreyKizimenko ✅ $ gh label list --search dependabot -> dependabot ✅ ``` ### 4. Branch filtering logic unit tested Extracted the JavaScript filter logic and ran it against 8 test cases covering single-package updates, grouped updates, and exclusions: ``` $ node -e "<test script>" PASS | dependabot/go_modules/golang.org/x/crypto-0.24.0 | match (Go single) PASS | dependabot-go_modules-google.golang.org-grpc-1.82.1 | match (Go grouped) PASS | dependabot/npm_and_yarn/typescript-5.5.0 | match (npm single) PASS | dependabot-npm_and_yarn-postcss-8.5.18 | match (npm grouped) PASS | dependabot/npm_and_yarn/website/next-14.0.0 | skip (website single) PASS | dependabot-npm_and_yarn-website-next-14.0.0 | skip (website grouped) PASS | dependabot/npm_and_yarn/fleet-slackbot/axios-1.7.0 | skip (slackbot single) PASS | dependabot/github_actions/actions/checkout-4 | skip (GitHub Actions) 8/8 passed ``` ### 5. Validated filter against real Dependabot PRs Checked the 20 most recent Dependabot PRs (`gh pr list --author 'app/dependabot' --state all --limit 20`) to confirm real branch names match the filter patterns. This revealed three bugs that were fixed: 1. **Go branch prefix was wrong**: Real branches use `dependabot/go_modules/`, not `dependabot/gomod/`. Fixed. 2. **Grouped updates use dashes**: Real grouped branches like `dependabot-go_modules-...` and `dependabot-npm_and_yarn-...` were not matched. Added dash-variant prefix checks and exclusions. 3. **Status label mismatch**: The constant was named `AWAITING_QA` but the option ID maps to "Ready" on the board (there is no "Awaiting QA" status). Renamed to `READY` and updated comments. ### 6. CI checks All CI checks pass (CodeQL, zizmor, dependency-review, build-binaries, publish). **Note:** The project board integration requires a `FLEET_GITHUB_TOKEN_PROJECTS` repo secret with Organization Projects read/write scope. Without it, the issue is still created but not added to the board (a warning is logged). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an automated GitHub Actions workflow for eligible Dependabot update pull requests. * Creates (and deduplicates) labeled QA issues with pull request details and checklist content, and posts the issue back to the pull request. * Optionally links new QA issues to project tracking with an “Awaiting QA” status when configured. * **Chores** * Hardened the automation workflow’s runner and permissions. * Updated safety-gate configuration to waive checks for the new Dependabot QA workflow. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
174 lines
6.9 KiB
YAML
174 lines
6.9 KiB
YAML
name: Create QA ticket for Dependabot PRs
|
|
|
|
# When Dependabot opens a code-dependency PR, create a QA issue on the
|
|
# #g-orchestration board so the team can verify that the bump introduces no
|
|
# behavioral regressions (see postmortem for #45763).
|
|
|
|
on:
|
|
pull_request_target:
|
|
types: [opened]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
create-qa-ticket:
|
|
if: >
|
|
github.event.repository.owner.login == 'fleetdm' &&
|
|
github.event.pull_request.user.login == 'dependabot[bot]'
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: read
|
|
issues: write
|
|
steps:
|
|
- name: Harden Runner
|
|
uses: step-security/harden-runner@9af89fc71515a100421586dfdb3dc9c984fbf411 # v2.19.4
|
|
with:
|
|
egress-policy: audit
|
|
|
|
- name: Create QA issue and add to project board
|
|
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
|
|
env:
|
|
PROJECT_TOKEN: ${{ secrets.FLEET_GITHUB_TOKEN_PROJECTS }}
|
|
with:
|
|
script: |
|
|
const pr = context.payload.pull_request;
|
|
const branch = pr.head.ref;
|
|
|
|
// Only process code dependency updates (Go modules, npm root).
|
|
// Skip: github-actions, docker, website npm, slackbot npm.
|
|
// Dependabot uses slashes for single-package updates and dashes for grouped updates:
|
|
// single: dependabot/go_modules/golang.org/x/crypto-0.35.0
|
|
// grouped: dependabot-go_modules-google.golang.org-grpc-1.82.1
|
|
const isGomod = branch.startsWith('dependabot/go_modules/')
|
|
|| branch.startsWith('dependabot-go_modules-');
|
|
const isNpm = (branch.startsWith('dependabot/npm_and_yarn/')
|
|
|| branch.startsWith('dependabot-npm_and_yarn-'))
|
|
&& !branch.includes('/website/')
|
|
&& !branch.includes('/fleet-slackbot/')
|
|
&& !branch.includes('-website-')
|
|
&& !branch.includes('-fleet-slackbot-');
|
|
|
|
if (!isGomod && !isNpm) {
|
|
console.log(`Skipping non-code dependency PR (branch: ${branch})`);
|
|
return;
|
|
}
|
|
|
|
const ecosystem = branch.includes('go_modules') ? 'Go' : 'npm';
|
|
const issueTitle = `QA: ${pr.title}`;
|
|
|
|
// Idempotency: check if a QA issue already exists for this PR
|
|
// (guards against duplicate tickets on workflow re-runs).
|
|
const existing = await github.rest.issues.listForRepo({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
labels: 'dependabot',
|
|
state: 'open',
|
|
per_page: 100,
|
|
});
|
|
const duplicate = existing.data.find(i => i.title === issueTitle);
|
|
if (duplicate) {
|
|
console.log(`QA issue already exists: #${duplicate.number} — skipping.`);
|
|
return;
|
|
}
|
|
|
|
// Create the QA tracking issue.
|
|
const issue = await github.rest.issues.create({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
title: issueTitle,
|
|
body: [
|
|
`## Dependabot dependency update — QA review needed`,
|
|
``,
|
|
`| Field | Value |`,
|
|
`|-------|-------|`,
|
|
`| **PR** | #${pr.number} |`,
|
|
`| **Ecosystem** | ${ecosystem} |`,
|
|
`| **Branch** | \`${branch}\` |`,
|
|
``,
|
|
`### Why this needs QA`,
|
|
``,
|
|
`Dependabot PRs update third-party dependencies. Even minor/patch bumps can`,
|
|
`change runtime behavior (see #45763 for a real example where a dependency`,
|
|
`bump silently broke SCEP certificate issuance). Please verify that the`,
|
|
`updated dependency does not cause regressions in related functionality.`,
|
|
``,
|
|
`### What to check`,
|
|
``,
|
|
`1. Review the PR diff and the dependency's changelog/release notes`,
|
|
`2. Identify which Fleet features use this dependency`,
|
|
`3. Test those features for behavioral changes`,
|
|
``,
|
|
`---`,
|
|
`*This issue was automatically created by the dependabot-qa-ticket workflow.*`,
|
|
].join('\n'),
|
|
labels: ['dependabot'],
|
|
assignees: ['AndreyKizimenko'],
|
|
});
|
|
|
|
console.log(`Created issue #${issue.data.number}`);
|
|
|
|
// Comment on the PR linking to the QA issue.
|
|
await github.rest.issues.createComment({
|
|
owner: context.repo.owner,
|
|
repo: context.repo.repo,
|
|
issue_number: pr.number,
|
|
body: `QA ticket created: #${issue.data.number}`,
|
|
});
|
|
|
|
// --- Project board: add to #g-orchestration and set status to "Inbox" ---
|
|
// Requires a PAT stored as FLEET_GITHUB_TOKEN_PROJECTS with
|
|
// read/write access to organization projects.
|
|
const projectToken = process.env.PROJECT_TOKEN;
|
|
if (!projectToken) {
|
|
core.warning(
|
|
'FLEET_GITHUB_TOKEN_PROJECTS secret is not configured — ' +
|
|
'issue was created but not added to the project board. ' +
|
|
'Create a fine-grained PAT with Organization Projects read/write scope.'
|
|
);
|
|
return;
|
|
}
|
|
|
|
const { getOctokit } = require('@actions/github');
|
|
const projectOctokit = getOctokit(projectToken);
|
|
|
|
const PROJECT_ID = 'PVT_kwDOBDAnic4A4BEx'; // #g-orchestration (#71)
|
|
const STATUS_FIELD = 'PVTSSF_lADOBDAnic4A4BExzgtDlHw';
|
|
const INBOX = 'f33a1ec5';
|
|
|
|
// Add the issue to the project.
|
|
const addResult = await projectOctokit.graphql(`
|
|
mutation($projectId: ID!, $contentId: ID!) {
|
|
addProjectV2ItemById(input: {projectId: $projectId, contentId: $contentId}) {
|
|
item { id }
|
|
}
|
|
}
|
|
`, {
|
|
projectId: PROJECT_ID,
|
|
contentId: issue.data.node_id,
|
|
});
|
|
|
|
const itemId = addResult.addProjectV2ItemById.item.id;
|
|
console.log(`Added to #g-orchestration board (item ${itemId})`);
|
|
|
|
// Set status to "Inbox".
|
|
await projectOctokit.graphql(`
|
|
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
|
|
updateProjectV2ItemFieldValue(input: {
|
|
projectId: $projectId
|
|
itemId: $itemId
|
|
fieldId: $fieldId
|
|
value: {singleSelectOptionId: $optionId}
|
|
}) {
|
|
projectV2Item { id }
|
|
}
|
|
}
|
|
`, {
|
|
projectId: PROJECT_ID,
|
|
itemId: itemId,
|
|
fieldId: STATUS_FIELD,
|
|
optionId: INBOX,
|
|
});
|
|
|
|
console.log('Status set to "Inbox"');
|