Discharge7443/fleet
1
0
Fork 0
Code Issues Pull Requests Actions 24 Packages Projects Releases Wiki Activity

Exclude migration tests from migration timestamp/ordering check (#22496)

Browse Source 
This fix applies to cases (e.g.
00ec402f18) where order of files being
added is e.g.:

1. Migration A
2. Migration B
3. Test for migration A

This also reorders workflow steps so the ones that don't require setting
up Go + compiling happen first, so if we have a migration issue it gets
reported sooner.

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
This commit is contained in:
Ian Littman
2024-09-30 09:53:19 -05:00
committed by GitHub
parent 0bd0dd8fb7
commit 59160c2f30
1 changed files with 46 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test-db-changes.yml
+46 -45
View File
@@ -40,6 +40,51 @@ jobs:
with:
fetch-depth: 0
# TODO: This doesn't cover all scenarios since other PRs might
# be merged into `main` after this check has passed.
#
# We should add a Slack notification or something similar for
# when this check fails on `main`.
#
# TODO: This only checks for added files, we should also check for renames,
# which should be more of an edge case, but they might still happen
- name: Check migration order
run: |
# if the workflow is run during a push event (on merges to main and
# tags,) use the latest created tag as a reference
base_ref=origin/${{github.base_ref}}
if [ "${{github.event_name}}" == "push" ]; then
base_ref=$(git tag --list "fleet-v*" --sort=-creatordate | head -n 1)
fi
all_migrations=($(ls server/datastore/mysql/migrations/tables/20*_*.go | sort -r | grep -v '_test.go'))
new_migrations=($(git diff --find-renames --name-only --diff-filter=A $base_ref -- server/datastore/mysql/migrations/tables/20\*_\*.go ':(exclude,glob)server/datastore/mysql/migrations/tables/20*_*_test.go' | sort -r))
index=0
for migration in "${new_migrations[@]}"; do
if [ "$migration" != "${all_migrations[$index]}" ]; then
echo "❌ fail: $migration has an older timestamp than ${all_migrations[$index]}"
echo "this might cause problems if this change is merged"
echo "please update the timestamp of $migration"
exit 1
fi
index=$((index+1))
done
- name: Prevent hosts foreign keys
run: |
# grep exits with an error code if it doesn't find a match, so this condition
# is only true if it a) finds a matching migrations file in the diff, and b)
# finds an FK to hosts in one of the migrations files.
#
# grep prints the matches, which will help figure out where those references are.
if git diff --name-only origin/main | grep "migrations/" | xargs grep -i -E 'references\s*hosts\s*\(\s*id\s*\)' ; then
echo "❌ fail: hosts foreign keys are not allowed"
echo "Ref: https://github.com/fleetdm/fleet/blob/main/handbook/engineering/scaling-fleet.md#foreign-keys-and-locking"
exit 1
fi
- name: Install Go
uses: actions/setup-go@93397bea11091df50f3d7e59dc26a7711a8bcfbe # v4.1.0
with:
@@ -65,48 +110,4 @@ jobs:
echo "❌ fail: uncommited changes in schema.sql"
echo "please run `make dump-test-schema` and commit the changes"
exit 1
fi
# TODO: This doesn't cover all scenarios since other PRs might
# be merged into `main` after this check has passed.
#
# We should add a Slack notification or something similar for
# when this check fails on `main`.
#
# TODO: This only checks for added files, we should also check for renames,
# which should be more of an edge case, but they might still happen
- name: Check migration order
run: |
# if the workflow is run during a push event (on merges to main and
# tags,) use the latest created tag as a reference
base_ref=origin/${{github.base_ref}}
if [ "${{github.event_name}}" == "push" ]; then
base_ref=$(git tag --list "fleet-v*" --sort=-creatordate | head -n 1)
fi
all_migrations=($(ls server/datastore/mysql/migrations/tables/20*_*.go | sort -r))
new_migrations=($(git diff --find-renames --name-only --diff-filter=A $base_ref -- server/datastore/mysql/migrations/tables/20\*_\*.go | sort -r))
index=0
for migration in "${new_migrations[@]}"; do
if [ "$migration" != "${all_migrations[$index]}" ]; then
echo "❌ fail: $migration has an older timestamp than ${all_migrations[$index]}"
echo "this might cause problems if this change is merged"
echo "please update the timestamp of $migration"
exit 1
fi
index=$((index+1))
done
- name: Prevent hosts foreign keys
run: |
# grep exits with an error code if it doesn't find a match, so this condition
# is only true if it a) finds a matching migrations file in the diff, and b)
# finds an FK to hosts in one of the migrations files.
#
# grep prints the matches, which will help figure out where those references are.
if git diff --name-only origin/main | grep "migrations/" | xargs grep -i -E 'references\s*hosts\s*\(\s*id\s*\)' ; then
echo "❌ fail: hosts foreign keys are not allowed"
echo "Ref: https://github.com/fleetdm/fleet/blob/main/handbook/engineering/scaling-fleet.md#foreign-keys-and-locking"
exit 1
fi
fi