Replace the hardcoded FLEET_MYSQL_MAX_OPEN_CONNS=10 in the loadtest
terraform (both the root ecs.tf stack and the infra/ stack) with a
mysql_max_open_conns variable, applied to both the writer and
read-replica pools.
Also document per-tier connection pool sizing in the AWS reference
architecture: 20 for R-class instances (25k+ hosts), 10 for the smaller
T-class instances (5k/10k hosts) whose low default max_connections
(~90/135) leaves little headroom.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44802
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configurable setting for MySQL connection limits in the load
testing environment.
* Connection settings now use a shared input value instead of fixed
defaults, making it easier to tune database load behavior across
container types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Document the scenarios for load testing Windows MDM at scale.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44188
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#47573
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated observability and telemetry configuration for improved
deployment monitoring.
* Enhanced infrastructure environment variables including logging,
database connection management, and Fleet service settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Bump osquery-perf module version to v1.2.2
- Increases ulimits from 9999 to 999999
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated performance testing infrastructure dependency to the latest
patch version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#47159.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the Go toolchain to 1.26.4 across modules, Docker build
stages, and build/configuration records to standardize the toolchain
version used for builds and tooling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35405
- fixes broken osquery_perf
- lower max connection lifetime so that loadtest can spread load across
readers faster (we often cause issues and need to recover)
- fixed broken signoz `terraform destroy`
- added README.md for pmm
# Checklist for submitter
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved load-testing image build process for more reliable binary
creation.
* Reduced MySQL connection pool timeouts to recycle connections more
frequently.
* Added robust, idempotent infrastructure cleanup steps during teardown
to avoid resource finalizer hangs and release cloud load balancers and
volumes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Prevents `terraform destroy` from running against load test
infrastructure, for a given workspace, if an identical osquery perf
workspace exists.
- Adds a check to the osquery perf workflow, to fail fast, if the
specified load test infrastructure workspace is not detected.
- Adds ability for users to configure the osquery perf (increment -
`input.loadtest_containers_increment`) batch size that get deployed to
an environment during every loop. Default: 4.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configurable load-test container increment input (default: 4)
to control scaling increments.
* **Improvements**
* Prevented destructive runs when an existing infra workspace matches
the requested name.
* Tightened workspace-existence checks to require exact name matching.
* **Chores**
* Default load-test increment reduced from 8 to 4 and input validated as
a positive integer.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Enable `standard` RDS database performance insights
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Enhanced database monitoring capabilities by enabling Database
Insights for load testing infrastructure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Disable performance insights
- Allow redis instance count >=1
- Properly set ecs_cluster logging config path
- Targeted apply with auto approve for pre-creating fleet and execution
roles
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Enhanced ECS cluster logging with CloudWatch integration and extended
log retention to 365 days.
* Adjusted RDS monitoring configuration and disabled performance
insights for operational optimization.
* Reduced minimum Redis instance requirement from 3 to 1 for greater
deployment flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#43671.
Bumps the Alpine base image from 3.23.3 to 3.23.4 in the Dockerfiles
that produce published images, picking up patched openssl, musl, and
zlib packages. Follows the same pattern as #38977.
### CVEs resolved
- HIGH: CVE-2026-28388, CVE-2026-28389, CVE-2026-28390, CVE-2026-31790,
CVE-2026-2673, CVE-2026-40200
- MEDIUM: CVE-2026-27171, CVE-2026-6042, CVE-2026-22184
### Test plan
- CI image build passes.
- Trivy/ECR scan on the resulting fleetdm/fleet image confirms the nine
listed CVEs are gone.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Docker base images to Alpine 3.23.4 across infrastructure and
deployment components for improved stability and security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Golang 1.26.2 has been released. It fixes some CVEs:
https://github.com/golang/go/issues?q=milestone%3AGo1.26.2+label%3ACherryPickApproved
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Go toolchain to 1.26.2 across the repository and build
configs.
* Updated Docker build images to use Go 1.26.2.
* Expanded the set of tracked modules for the Go version update so
additional module files are included in automated updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Updates wording in `.github/workflows/loadtest-osquery-perf.yml`
- `4098` -> `4096`
- Removes: `(should be a multiple of 8, if setting
loadtest_containers_starting_index)`
- Updates `infrastructure/loadtesting/terraform/osquery_perf/enroll.sh`
to handle values that are not multiples of 8. If the value is not a
multiple of 8, logic has been added to apply the remainder.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Documentation**
* Updated load testing workflow configuration input descriptions for
improved clarity of parameters and their usage examples.
* **Bug Fixes**
* Fixed container count allocation logic in the load testing process to
ensure the final target count is always properly applied, even when
using increment values that don't divide evenly into the specified total
range.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Configures internal alb to log to the same bucket as the public alb
- Adds support for osquery-perf task size (cpu/memory) configuration
- Updates defaults for osquery-perf extra_flags
- Updates default enroll.sh loop sleep_time from 60s -> 300s
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves # N/A
- Resolves an issue that prevents some locally pulled docker images from
being pushed to ECR.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41749
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Adds require_secure_transport for mysql connections to the db_cluster
parameter group for dogfood and loadtest environments.
```
db_cluster_parameters = {
require_secure_transport = "ON"
}
```
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34677 and #35932
Adding ~450K software to the loadtest, including scripts to add more
software in the future.
Software is held in a `software.sql` file, which is used to create a
sqlite DB during osquery perf run/deployment.
# Checklist for submitter
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for loading software data from an external SQLite
database via a new `--software_db_path` command-line flag for more
realistic simulation scenarios.
* Added import and SQL generation tools to build and manage custom
software libraries.
* **Documentation**
* Added comprehensive README with setup instructions, tool usage, and
end-to-end workflow guidance for the software library.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34677
Changing OTEL set up to group exceptions by type, which is an
OTEL/industry best practice.
- Removes timestamp from osquery_perf image
- Adds `default: 0` to loadtest osquery_perf workflow, `variable:
loadtest_containers_starting_index`
- Adds `variable: sleep_time` to loadtest osquery_perf workflow
- Adds osquery_perf docker repository in ECR
- Adds support for `sleep_time` to `enroll.sh`
- Updates terraform variables to enforce `git_branch` or `git_tag` for
osquery_perf
- Removes `firehose` logging from loadtesting environment
- Sets `filesystem` logging in loadtesting environment
- Updates fleet image to 4.76.0 as the default value
- Updates `migrations` and `logging_alb` modules with latest versions
- Removes `(Coming Soon)` from
`infrastructure/loadtesting/infra/README.md` with regards to deployment
via Github Actions
- Moves Signoz steps to `.header.md` to preserve steps in generated
`README.md`
- Adds support for `enroll.sh`, to deploy osquery_perf in batches
- Merges variables `tag` and `git_branch` into `git_tag_branch`. Only
one tag or git_branch should be specified.
- Still used for osquery_perf to check out the correct tag/branch.
- Removes fleet_image requirement for cutting osquery_perf images
---------
Co-authored-by: Robert Fairburn <8029478+rfairburn@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34500
Terraform changes after my latest loadtest.
VPC consolidation: updated (and deployed) shared VPC so that Signoz
backend can now use it
- Removed eks-vpc/ directory
- Moved VPC management to shared/vpc.tf
- Updated shared/init.tf to reflect VPC changes
Infra improvements
- infra/internal_alb.tf - changed suffix from -internal to -int since I
hit max 32 characters issue
OTEL
- OTEL Collector configuration overrides for production stability
untested end-to-end but works as a replacement for plans and doesn't
require a local arm64 build to work.
Co-authored-by: Jorge Falcon <22119513+BCTBB@users.noreply.github.com>
Victor Lyuboslavsky
