Managing Cloud Expenses with Infracost
Learn how to manage the cloud costs of the Cloud Infrastructure
When we integrate cloud services with Infrastructure as Code (IaC) tools like Terraform and continuous deployment, we gain the nearly magical ability to create resources on demand. Despite its many benefits, the cloud presents challenges, such as accurately estimating costs. Can this tedious task be automated?
Yes, it can, thanks to the open-source tool Infracost.
Why Cloud Costs Are Hard
Ever tried to navigate the cost structures of cloud providers?
It’s like trying to solve a constantly changing puzzle. Take AWS, for example. Did you know they offer over 700 types of Linux machines on EC2? Many of them have similar names and features. Consider “m6g.2xlarge” and “m6gd.2xlarge”: one includes an SSD, while the other does not. A simple mistake in your Terraform file can cause your bill to skyrocket unexpectedly. Sounds familiar?
What is Infracost?
Infracost is an open-source project designed to help us understand our cloud spending. It provides a detailed breakdown of our infrastructure costs and calculates the financial impact of any changes. In essence, Infracost acts as a "git diff" for cloud billing.
Infracost comes in two flavors: a handy VSCode extension and a versatile command-line program. Curious about what they do? Both versions parse your Terraform or Terragrunt files, fetch the latest price points from the Infracost cloud pricing API, and give you an estimate of your costs. Whether you prefer a graphical interface or the command line, Infracost has you covered!
You have the option to use the Infracost pricing API for free, or you can host it yourself. If you opt for the paid tier, you'll also gain access to a cloud dashboard that helps you track cost changes over time.
Using this extension, the estimates for the cost are seen right in the VS Code console.
Installing Infracost
So let us begin with the installation of Infracost on a linux machine:
This tool does not come with multiple commands to be executes, just a simple single line command run brings the tool to your environment.
$ curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | shTo use Infracost, you will require an API Key which can be easily retrieved using the below command. Just execute on the shell or CLI.
$ infracost auth loginAnd also have some Terraform configuration files ready to be analyze.
Ready to get started? Let's dive in with the infracost breakdown command. This command analyzes your Terraform plans and provides a cost estimate. Make sure the --path variable points to the folder containing your Terraform files.
Let us consider, you want to provision an "a1.medium" EC2 instance, here's what you need to do:
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "my_instance" {
ami = "ami-674cbc1e"
instance_type = "a1.medium"
root_block_device {
volume_size = 100
}
}When we run infracost breakdown --path ., we receive a cost estimate for the specified resources in our Terraform configuration.
$ infracost breakdown --path .
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs
Project: Shivanshu-Sharma/searce-infracost/ec2
Name Monthly Qty Unit Monthly Cost
aws_instance.my_instance
├─ Instance usage (Linux/UNIX, on-demand, a1.medium) 730 hours $18.62
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 100 GB $10.00
OVERALL TOTAL $28.62
──────────────────────────────────
1 cloud resource was detected:
∙ 1 was estimated
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ Shivanshu-Sharma/searce-infracost/ec2 ┃ $29 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛
If we add an additional 600GB of EBS storage, the cost increases to $89, as shown below:
$ infracost breakdown --path .
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs
Project: Shivanshu-Sharma/searce-infracost/ec2
Name Monthly Qty Unit Monthly Cost
aws_ebs_volume.extra_storage
└─ Storage (general purpose SSD, gp2) 600 GB $60.00
aws_instance.my_instance
├─ Instance usage (Linux/UNIX, on-demand, a1.medium) 730 hours $18.62
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 100 GB $10.00
OVERALL TOTAL $88.62
──────────────────────────────────
3 cloud resources were detected:
∙ 2 were estimated
∙ 1 was free
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ Shivanshu-Sharma/searce-infracost/ec2 ┃ $89 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛
Infracost can calculate costs for usage-based resources like AWS Lambda. Let's explore what happens when we replace the EC2 instance with serverless functions, but running infracost breakdown yields a total cost of 0 dollars in this case. As there are no requests coming to the Lambda function, not incurring and cost for the usage if this Lambda function.
To understand costing of a AWS Lambda function, we can provide usage estimates for our Lambda function using a usage file. Here's the command to create and configure the usage file.
$ infracost breakdown --sync-usage-file --usage-file usage.yml --path .We can now provide estimates by editing usage.yml. Here's an example where we estimate 5 million requests with an average runtime of 300 milliseconds:
# usage.yml
resource_usage:
aws_lambda_function.my_lambda:
monthly_requests: 5000000
request_duration_ms: 300 Now we are required to use this usage.yml file to understand the right estimates for our requirements. The below command helps us achieve this.
$ infracost breakdown --path . --usage-file usage.ymlNow if you compare the results with an empty run for Lambda, the difference is evident.
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Syncing usage data from cloud
└─ Synced 0 of 1 resource
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs
Project: Shivanshu-Sharma/searce-infracost/lambda
Name Monthly Qty Unit Monthly Cost
aws_lambda_function.serverless_lambda
├─ Requests 5 1M requests $1.00
└─ Duration (first 6B) 1,500,000 GB-seconds $25.00
OVERALL TOTAL $26.00
──────────────────────────────────
1 cloud resource was detected:
∙ 1 was estimated
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project ┃ Monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ Shivanshu-Sharma/searce-infracost/lambda ┃ $26 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛Much better!
Just remember, the accuracy hinges on the precision of our usage file. If you're not entirely sure, why not spice things up by integrating Infracost with your cloud provider? That way, you can fetch utilization metrics straight from the source for more dependable estimates!
Dealing with changes in Infrastructure Configuration
To estimate the impact of changes, Infracost allows us to save results in JSON format using the --format json and --out-file options. This provides a baseline file that we can include in our source control:
$ infracost breakdown --path . --format json --usage-file usage.yml --out-file baseline.jsonNow, we can compare changes using infracost diff. Let's observe what happens if the Lambda execution time increases from 300 to 350 milliseconds.
$ infracost diff --path . --compare-to baseline.json --usage-file usage.yml
Evaluating Terraform directory at .
✔ Downloading Terraform modules
✔ Evaluating Terraform directory
✔ Retrieving cloud prices to calculate costs
Key: * usage cost, ~ changed, + added, - removed
──────────────────────────────────
Project: Shivanshu/infracost-demo/lambda
~ aws_lambda_function.serverless_lambda
+$4 ($26 → $30)
~ Duration (first 6B)
+$4 ($25 → $29), +250,000 GB-seconds (1,500,000 → 1,750,000)*
Monthly cost change for Shivanshu-Sharma/searce-infracost/lambda
Amount: +$4 ($26 → $30)
Percent: +16%
──────────────────────────────────
Key: * usage cost, ~ changed, + added, - removed
1 cloud resource was detected:
∙ 1 was estimated
Infracost estimate: Monthly cost will increase by $4 ↑
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Project ┃ Cost change ┃ New monthly cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━┫
┃ Shivanshu-Sharma/searce-infracost/lambda ┃ +$4 (+16%) ┃ $30 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━┛Summing it all up
The ability to provision resources instantly is a double-edged sword: a simple typo in a Terraform file can lead to expensive mistakes. If you're already automating deployments and managing services with Terraform, integrating Infracost into your workflow can provide valuable insights and prevent unexpected costs. Setting it up is quick and easy, taking just a few minutes, yet it can potentially save thousands of dollars in the long run.
Article by - Shivanshu Sharma
Credits - Infracost: https://www.infracost.io/docs/ https://github.com/infracost/infracost Spacelift: https://spacelift.io/blog/terraform-cost-estimation-using-infracost
Last updated