Setting up S3 cross-account replication for AWS cost and usage reports (CURs) with Terraform.

Photo by Vijay Sekhar Sadineni on Unsplash

AWS cost and usage reports (CUR) enabled from the billing dashboard provide one of the most detailed insights into AWS costs for an account or organization.

In this, article I’m going to talk about setting everything up, starting with the report itself and configuring s3 to receive those reports. Furthermore, since usually CUR is set up in the payer account, which is usually also the master account for most AWS organizations, we will also set up replication from the s3 bucket in the source (payer) account to a child account (destination) where we would actually like to utilize those report.

These reports can be used to setup the CUDOS cost insights dashboards from AWS. This article focuses on setting things up in accordance with the CUDOS workshop documentation.

Let us first set some definitions.

• Source Account: Where we set up the cost and usage reports and our main CUR S3 bucket.
• Destination Account: Where we set up our destination s3 bucket as a replication target of our main CUR S3 bucket located in our source account. This is the account where we actually want to set up the infrastructure to consume those reports.

We assume here that both source and destination S3 buckets are in the same region.

Source Account

First, let us create our cost and usage report as shown below.

The above format is recommended for the CUDOS dashboard. See documentation here.

Next, create the s3 bucket as a target for the reports.

Now we must create the role and configure its permissions so as to actually allow the replication to happen. (Role referenced in line 17 above)

Now, we’re good to go from the source account. Halfway there! Time to set up our destination account.

Destination Account

Setting up the destination account is relatively straightforward. We create the s3 bucket and give the necessary permissions for replication to take place on the bucket. We authorize the replication role to take replication actions on our destination bucket.

And we are done. Now you should be ready to run terraform planand then terraform applyto apply your changes.

One point I would mention here, since we are essentially creating resources in two separate accounts from the same terraform workspace we would need two aws providers blocks here. One main (source account) and a second one with an alias and a profile or role to the second account. All resources created under the destination account would essentially use the second provider provider = aws.destination_account (depending on convention you follow ofcourse) in all resource blocks. We leave it out here and focus on the actual deployment.

As an example the terraform provider config for the above deployment can look like this.

# default for the source account resources
provider "aws" {
  profile = var.aws_profile
  region  = var.aws_region
}
# We use this one with the CUR report definition resource.
provider "aws" {
  alias   = "cur_reports"
  profile = var.aws_profile
  region  = "us-east-1"
}
# For the destination account resources.
provider "aws" {
  profile = var.aws_profile # detination account profile
  region  = var.aws_region
  alias   = "destination_account"
}

Test out your replication by manually uploading a file to the source account bucket and wait for it to get replicated to the destination account. You can have a look at the replication status by clicking on the object itself and looking under the replication rules. It will transition from PENDING to COMPLETE once done. (It can take a few minutes for the object to be replicated). See below

Pending…

.. A few moments later…Completed…

You will see FAILED if the replication fails for any reason. In that case, I recommend going through this guide here.

…And that’s it, folks. :)