diff --git a/tools/rds-db-restore/README.md b/tools/rds-db-restore/README.md new file mode 100644 index 0000000..1e81f2b --- /dev/null +++ b/tools/rds-db-restore/README.md @@ -0,0 +1,233 @@ +# Fleet database restore on AWS + +`db-restore.sh` automates the restoration of a Fleet-managed Aurora database cluster using Terraform and AWS RDS APIs. It supports point-in-time recovery (PITR), RDS snapshot restoration, dry-runs, ECS scale-down handling, optional Fleet image rollback, RDS master username updates, and delayed cleanup of old DB resources. + +This script is designed for self-hosted Fleet deployments provisioned via `fleet-terraform`. + +> **Note:** Fleet built and tested `db-restore.sh` against the [`fleet-terraform/example`](https://github.com/fleetdm/fleet-terraform/tree/main/example) (Standard) deployment layout. If your Fleet deployment is not based on `fleet-terraform/example`, this script will not support your restore. + +## Prerequisites + +- `terraform`, `aws` CLI, `jq`, `perl`, and `python3` on your `PATH` +- AWS credentials configured with permissions to manage RDS, ECS, IAM, Secrets Manager, and EC2 security groups in the target region +- The target `fleet-terraform` environment directory checked out locally + +## Quick start + +The script does not need to live in your environment directory, but you should run it from there. It defaults to `$PWD` for Terraform state and config. `cd` into your environment directory before running it, whether the script lives inside that directory or somewhere else on disk. + +Every restore command needs exactly one restore source. Use `--restore-time ` for point-in-time recovery (PITR), or `--restore-snapshot ` to restore from an RDS DB cluster snapshot. Each step below shows both forms. + +1. **Run from inside the environment directory:** + ```bash + cd fleet-terraform/example + AWS_PROFILE= /path/to/db-restore.sh --list + ``` +2. **Dry-run the restore:** + + PITR: + ```bash + AWS_PROFILE= /path/to/db-restore.sh \ + --restore-time 2026-05-05T11:00:00Z \ + --dry-run + ``` + Snapshot: + ```bash + AWS_PROFILE= /path/to/db-restore.sh \ + --restore-snapshot arn:aws:rds:us-east-2:123456789012:cluster-snapshot:fleet-prod-manual-2026-04-06 \ + --dry-run + ``` +3. **Execute the restore:** + + PITR: + ```bash + AWS_PROFILE= /path/to/db-restore.sh \ + --restore-time 2026-05-05T11:00:00Z \ + --confirm + ``` + Snapshot: + ```bash + AWS_PROFILE= /path/to/db-restore.sh \ + --restore-snapshot arn:aws:rds:us-east-2:123456789012:cluster-snapshot:fleet-prod-manual-2026-04-06 \ + --confirm + ``` + +## Supported Terraform layouts + +The script works with `fleet-terraform` layouts by auto-detecting the parent module address from Terraform state. It searches for `module.rds.aws_rds_cluster.this` and uses the path preceding it as the base. + +| Layout | Environment Directory | Auto-detected `MODULE_ADDRESS` | +|---|---|---| +| Standard | `fleet-terraform/example` | `module.fleet.module.byo-vpc` | +| BYO-VPC *(not verified; contact Fleet customer support before use)* | `fleet-terraform/byo-vpc/example` | `module.byo-vpc.module.byo-db` | + +## Execution path + +When running a standard restore with migrations enabled, the script performs the following steps: + +1. Captures a full copy of Terraform state and resource metadata into `.db-restore-/`. +2. Optionally updates `fleet_config.image` if `--rollback` and `--fleet-image` are provided. Supports literal values and `local.*` references whose definitions are literals in the same file. `var.*` expressions are rejected. +3. Optionally adds/updates `rds_config.master_username` if `--master-username` is provided. +4. Updates the `rds_config` block to point to the restored database. +5. Scales Fleet ECS services to `0` so no tasks connect during the restore. +6. Removes old RDS resources from Terraform state. +7. Creates a new Aurora cluster from the chosen restore point. +8. Restores the original `rds_config` and re-applies monitoring/observability settings. +9. Applies ECS services and runs database migrations (`module.migrations`). +10. Scales ECS services back up. +11. Keeps old DB resources intact for safe cleanup later. + +## Configuration & flags + +| Flag | Required | Description | +|---|---|---| +| `--cleanup-only` | No | Cleanup mode. Requires `--manifest` pointing to the `manifest.json` written under `.db-restore-/` by a previous run. Deletes old DB resources from that restore. | +| `--cleanup-old-resources` | No | Delete old DB resources immediately after restore. Default keeps them for safe later cleanup. | +| `--confirm` | No | Skips the interactive typed confirmation prompt. Without it, you will be prompted to type the environment name. | +| `--config-file ` | No | Terraform file containing `rds_config`. Default: `/main.tf`. | +| `--destination-name ` | No | Override restored DB name. Default increments from current cluster name. | +| `--dry-run` | No | Print the execution path and planned mutations. Creates a manifest without applying changes. | +| `--env-dir ` | No | Root directory containing the environment. Default: `$PWD`. | +| `--fleet-image ` | No | Specifies the Fleet image version when using `--rollback`. | +| `--help` | No | Show help text. | +| `--master-username ` | No | Set `rds_config.master_username`. Adds if absent, updates if present. Requires fleet-terraform root module tag >= `tf-mod-root-v1.28.0` or byo-vpc tag >= `tf-mod-byo-vpc-v1.29.0`. | +| `--manifest ` | Yes (with `--cleanup-only`) | Path to the `manifest.json` JSON file produced by a prior restore (written under `.db-restore-/`). Accepts an absolute or relative path; absolute is convenient when invoking the script from outside the environment directory. Used by `--cleanup-only`. | +| `--module-address ` | No | Terraform address of the parent module. Auto-detected when possible. | +| `--no-ecs-apply` | No | Do not apply ECS targets or scale services back up. Services remain at `0` for manual validation. | +| `--old-final-snapshot-id ` | No | Final snapshot ID for old cluster deletion. Auto-generated if omitted, in the form `-pre-restore-retirement-`. | +| `--region ` | No | AWS region. Default: `AWS_REGION` / `AWS_DEFAULT_REGION` / `us-east-2`. | +| `--restore-snapshot ` | No | Restore from an RDS DB cluster snapshot identifier or ARN. | +| `--restore-time