feat(runner-binaries-syncer): add s3_tags variable for additional S3 bucket tagging

[damianrekosz]

This pull request introduces an option to provide additional tags for S3 buckets created by the binaries-syncer module.

S3 tags can be specified using the runner_binaries_s3_tags variable at the module level to apply tags to all S3 buckets at once, or at the individual runner configuration level to define different tags for specific runners. Tags defined at both the module and runner configuration levels are merged when their OS and architecture match.

[Copilot]

Pull Request Overview

This PR adds support for additional S3 bucket tagging in the runner-binaries-syncer module. Users can now specify custom tags for S3 buckets at both the module level and individual runner configuration level.

• Introduces runner_binaries_s3_tags variable for module-level S3 bucket tagging
• Adds runner_binaries_s3_tags field to multi-runner configuration for runner-specific tagging
• Implements tag merging logic that combines module-level and runner-specific tags

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
variables.tf	Adds module-level runner_binaries_s3_tags variable
modules/runner-binaries-syncer/variables.tf	Adds s3_tags variable to syncer module
modules/runner-binaries-syncer/main.tf	Updates S3 bucket resource to merge default and additional tags
modules/multi-runner/variables.tf	Adds runner_binaries_s3_tags to runner config and module variables
modules/multi-runner/runner-binaries.tf	Passes merged tags to runner binaries module
modules/multi-runner/main.tf	Implements tag merging logic for different OS/architecture combinations
main.tf	Passes S3 tags variable to runner binaries module

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[damianrekosz]

@npalm could I get your input on this?

[npalm]

Sorry won't be able to review any PR till the end of the month.

[npalm]

@damianrekosz can you check the commen from copilot. Can you also quickly explain the use case for different tags on the bucket.

[damianrekosz]

can you check the commen from copilot.

The comment from Copilot isn't accurate. Removing flatten will cause an error because the arguments form a tuple. Also, runner_binaries_s3_tags will not produce a list of maps, as its type is set to map(string).

Can you also quickly explain the use case for different tags on the bucket.

Yeah, sure. Some organizations may tag their S3 buckets differently - eg each bucket might be tagged with its name or department to provide better cost management visibility when custom tags in cost allocation are enabled.

[npalm]

@damianrekosz thx for the PR, Sorry for the waiting time. PR looks in general ok. Made some suggestion comments on the multi runner. No need create a complex local object. Simply pass the tags down the mdule multi-runner module.

[npalm]

I think this local for s3_tags is not needed. The multi runner crates a bucket per github dist. Which means not a bucket er multi runner. So would simple pass the s3_tags down to the module. And no cmplext merge.

[npalm]

This variable is ok, lets remove the one on runner config lvel.

[npalm]

Suggested change

	s3_tags = local.s3_tags[each.key]
	s3_tags = var.runner_binaries_s3_tags

[damianrekosz]

Thanks for the suggestions. Unfortunately, without configuration at the runner level, we won't be able to assign different tags to different runner pools, such as arm64 and x64. This PR was meant to introduce that capability. The approach you suggested would only allow managing additional tags for S3 buckets, which wouldn't be sufficient, at least not in my organization.
For example with my changes, I could tag the arm64 bucket with architecture: arm64 and the x64 bucket with architecture: x64, while also defining global S3 tags at the module level to apply a common set of tags to all buckets - making the setup much more flexible.

[npalm]

Thanks for the suggestions. Unfortunately, without configuration at the runner level, we won't be able to assign different tags to different runner pools, such as arm64 and x64. This PR was meant to introduce that capability. The approach you suggested would only allow managing additional tags for S3 buckets, which wouldn't be sufficient, at least not in my organization. For example with my changes, I could tag the arm64 bucket with architecture: arm64 and the x64 bucket with architecture: x64, while also defining global S3 tags at the module level to apply a common set of tags to all buckets - making the setup much more flexible.

But the bucket is shared with multiple runners in case the architecture and os is the same. So I think mapping per runner config is confusing. Costs related to those buckets should be smalle. Since it is not about much data.

[damianrekosz]

You're right, the cost isn't a major concern here, but organizations may have an AWS Config rule with auto-remediation enabled to tag all S3 buckets with the bucket's name. Such a configuration will cause drifts in the runner module, because without specifying the tags at the runner level, the drifts won't be resolved, causing the tags to be removed at every apply and re-applied by AWS Config.

[npalm]

You're right, the cost isn't a major concern here, but organizations may have an AWS Config rule with auto-remediation enabled to tag all S3 buckets with the bucket's name. Such a configuration will cause drifts in the runner module, because without specifying the tags at the runner level, the drifts won't be resolved, causing the tags to be removed at every apply and re-applied by AWS Config.

Yup, so what main is to apply the s3_tags directly to all bukcets created by the runner-binary module. And for multi runner not pass the tags in the runner-config. But as top-level variable in the multi-runner module. It was now on two places in that module.

[damianrekosz]

Okay, let's investigate this case. I have a multi-runner configuration with 2 pools:

1. arm64
2. x64

The 2 pools will create separate S3 buckets - one for each architecture - and now I'd like to tag the created buckets with specific tags. Let's assume the following buckets were created:

1. multi-runner-linux-arm64-dist
2. multi-runner-linux-x64-dist

Now each bucket should have its own BucketName tag to eliminate the drifts caused by my AWS Config auto-remediation rule that automatically tags new buckets.

I can now add the tags at the runner configuration level:

• for multi-runner-linux-arm64-dist:

  runner_binaries_s3_tags:
    BucketName: multi-runner-linux-arm64-dist

• for multi-runner-linux-x64-dist

  runner_binaries_s3_tags:
    BucketName: multi-runner-linux-x64-dist

which gives me the option to eliminate the drifts.
Defining s3_tags as a top-level variable would only allow assigning common tags to all buckets, not individual ones. That's why I'd like to add the option to define tags at the runner module level, in addition to the global s3_tags included in this PR.