Terraform by HashiCorp#

General information#

Terraform is an advanced tool for automated cloud infrastructure management. It uses a simple and expressive language similar to plain English. The code in this language is written in a declarative manner: you describe what you want to get and do not think about how to get it.

Once you have written such a code, you can reuse it many times: enter a couple of short commands in the terminal. And every time you will get a predictable result: the requested number of VMs will be created in the cloud from the specified images, the required number of external IP addresses will be allocated, security groups will be configured, and all the other actions described in the code will be performed. Performing the same actions in the web interface will take more time, especially if you need to repeat them. In addition, when doing this manually, you face a dramatically higher risk of making a mistake and getting something different from what you planned, with much time being spent than trying to understand why it went wrong.

This approach to infrastructure deployment is called “Infrastructure as a Code” (IaaC) and allows you to:

  • use version control systems;

  • comment the code to document what it does;

  • test the code before using it in an actual infrastructure to identify possible negative consequences;

  • hand over the code to other developers to evaluate its quality and finally get the best solution.

Installation and configuration#

Note

The guide was written and tested using Terraform v1.0.8 for the croccloud v4.14.0-CROC1 and AWS v3.63.0 providers, and the information below is relevant for these versions. We have frozen the provider version by embedding it in the configuration code to ensure stability and compatibility.

Terraform is distributed as an executable file with versions for Linux, Windows, macOS, etc. You can download the version you need from the official download page. If the official page is unavailable, download the installation package here. After downloading and extracting the archive, we recommend moving the extracted file to any folder specified in the current environment variable PATH or adding the target folder to this variable. For Linux, it can be /usr/local/bin/, while for Windows — C:\Windows\system32 (OS administrator rights are required to access system folders). Thus, you will not have to specify the full path to the file each time.

Local mirrors for common providers#

Providers from CROC Cloud’s local mirrors are identical to those from the original repositories. You can use them to avoid installation problems.

Note

For up-to-date information, follow the link.

Provider

Available versions

Link to the mirror

AWS

2.64.0 3.15.0 3.63.0 4.8.0

https://hc-releases.website.cloud.croc.ru/terraform-provider-aws/

Kubernetes

2.10.0 2.11.0

https://hc-releases.website.cloud.croc.ru/terraform-provider-kubernetes/

Random

3.3.2

https://hc-releases.website.cloud.croc.ru/terraform-provider-random/

Template

2.2.0

https://hc-releases.website.cloud.croc.ru/terraform-provider-template/

TLS

3.1.0

https://hc-releases.website.cloud.croc.ru/terraform-provider-tls/

Provider from CROC Cloud#

To work with CROC Cloud, you can use croccloud provider from C2Devel. It was created on the basis of AWS 4.14.0 provider and excludes Terraform resources not supported in CROC Cloud. At the same time, it has added functionality specific to CROC Cloud, such as st2 volumes.

For information about croccloud provider releases, visit the official repository.

The croccloud provider is included in the official Terraform register . The provider page offers documentation on resources supported in CROC Cloud.

Note

The documentation contains a unified list of resources with a prefix that is automatically generated for each resource and matches the croccloud name. For compatibility with configurations for the AWS provider, we retained the aws prefix in resource description and usage examples.

Resource list by category#

Category

Terraform Resource

Terraform Datasource

Auto Scaling

EBS (EC2)

EC2 (Elastic Compute Cloud)

EKS (Elastic Kubernetes)

ELB (Elastic Load Balancing)

PaaS

Route53

Transit Gateway

S3 (Simple Storage)

VPC (Virtual Private Cloud)

VPN (Site-to-Site)

Describing Terraform configuration#

The ready-to-use code described below is stored in our official repository terraform-examples on GitHub in the quick_start folder. You can download and start using it right away with minimal edits. However, for a better code understanding, we recommend that you follow this guide’s steps and operations one by one.

Warning

When using Terraform, run commands only if you have a good idea of what you are doing and why. Terraform warns of potentially destructive operations and requires additional confirmation in these cases. Pay attention to these warnings, since otherwise, you may inadvertently lose part or even all of your project’s infrastructure and data. And if there are no backups, the data will be lost forever.

As an example, let’s consider the description of the Terraform configuration to automatically create an infrastructure consisting of:

  • 1 VPC (to isolate the project infrastructure at the network layer);

  • 1 subnet with prefix /24;

  • 2 VMs: one for a web application, and the other for a database server;

  • 1 Elastic IP is an address assigned to the VM with the web application so that it (and the application) can be accessed from the Internet;

  • 2 security groups: one group allows inbound traffic from the interfaces to which it is assigned (so that the VMs interact only with each other within the created subnet), while the other allows access from the outside over TCP ports 22, 80, and 443. All outbound traffic is allowed for each of the groups;

  • 1 bucket to store project files.

Provider description — providers.tf#

Terraform deals with various cloud platforms and services, using special plugins, which are called providers. To work with CROC Cloud, you can use CROC Cloud provider (c2devel/croccloud) or AWS provider (hashicorp/aws), since CROC Cloud API is AWS-compatible.

Create a providers.tf file to describe the required providers and their settings:

providers.tf
# Select a specific provider version to ensure compatibility
# and stable operation of the developed configuration
terraform {
  required_providers {
    aws = {
      # Use the CROC Cloud local mirror
      # to download c2devel/croccloud provider
      source  = "hc-registry.website.cloud.croc.ru/c2devel/croccloud"
      version = "4.14.0-CROC1"
    }
  }
}

# Connect and configure the provider to work
# with all CROC Cloud services except for object storage
provider "aws" {
  endpoints {
    ec2 = "https://api.cloud.croc.ru"
  }

  skip_credentials_validation = true
  skip_requesting_account_id  = true
  skip_region_validation      = true

  insecure   = false
  access_key = var.access_key
  secret_key = var.secret_key
  region     = "croc"
}

# Connect and configure the provider to work
# with the CROC Cloud object storage
provider "aws" {
  alias = "noregion"
  endpoints {
    s3 = "https://storage.cloud.croc.ru"
  }

  skip_credentials_validation = true
  skip_requesting_account_id  = true
  skip_region_validation      = true

  insecure   = false
  access_key = var.access_key
  secret_key = var.secret_key
  region     = "us-east-1"
}

The first provider block allows you to interact with all CROC Cloud services except for object storage, while the second is just responsible for interacting with the object storage. If you plan to work with CROC Cloud only, you can reuse this part of the code without changes.

Note that access_key and secret_key do not contain the data itself but rather point to variable values. This is done on purpose so that a ready-to-use configuration can be handed over to other people without revealing the key values. In addition, this allows you to quickly define all keys in one place and avoid multiple edits in the code itself when they change.

Variable description — variables.tf#

Information about all variables in use is stored in the variables.tf file, where you can specify a description and the default value for each variable.

variables.tf
variable "secret_key" {
  description = "Enter the secret key"
}

variable "access_key" {
  description = "Enter the access key"
}

variable "public_key" {
  description = "Enter the public SSH key"
}

variable "pubkey_name" {
  description = "Enter the name of the public SSH key"
}

variable "bucket_name" {
  description = "Enter the bucket name"
}

variable "az" {
  description = "Enter availability zone (ru-msk-comp1p by default)"
  default     = "ru-msk-comp1p"
}

variable "eips_count" {
  description = "Enter the number of Elastic IP addresses to create (1 by default)"
  default     = 1
}

variable "vms_count" {
  description = "Enter the number of virtual machines to create (2 by default)"
  default     = 2
}

variable "hostnames" {
  description = "Enter hostnames of VMs"
}

variable "allow_tcp_ports" {
  description = "Enter TCP ports to allow connections to (22, 80, 443 by default)"
  default     = [22, 80, 443]
}

variable "vm_template" {
  description = "Enter the template ID to create a VM from (cmi-AC76609F [CentOS 8.2] by default)"
  default     = "cmi-AC76609F"
}

variable "vm_instance_type" {
  description = "Enter the instance type for a VM (m5.2small by default)"
  default     = "m5.2small"
}

variable "vm_volume_type" {
  description = "Enter the volume type for VM disks (gp2 by default)"
  default     = "gp2"
}

variable "vm_volume_size" {
  # Default size and increment are specified for the gp2 volume type
  # For other volume types, they may differ (for details, see the volumes documentation)
  description = "Enter the volume size for VM disks (32 by default, in GiB, must be multiple of 32)"
  default     = 32
}

The variables.tf file contains only a list of all variables used in the configuration (and default values for some of them). The actual values are set in the terraform.tfvars file.

Actual variable values — terraform.tfvars#

Which values to apply in each case is specified in the file terraform.tfvars. Its content takes precedence over the default values, making it easy to override the default configuration behaviour.

terraform.tfvars
secret_key       = "ENTER_YOUR_SECRET_KEY_HERE"
access_key       = "ENTER_YOUR_ACCESS_KEY_HERE"
public_key       = "ENTER_YOUR_PUBLIC_KEY_HERE"
pubkey_name      = "My-project-SSH-key"
bucket_name      = "My-project-bucket"
az               = "ru-msk-comp1p"
eips_count       = 1
vms_count        = 2
hostnames        = ["webapp", "db"]
allow_tcp_ports  = [22, 80, 443]
vm_template      = "cmi-AC76609F"
vm_instance_type = "m5.2small"
vm_volume_type   = "gp2"
vm_volume_size   = 32

The template with all variables and their values is in the terraform.tfvars.example file. To set variables faster, copy the file content to the terraform.tfvars file and then change values as required:

cp terraform.tfvars.example terraform.tfvars

Warning

Remember that the terraform.tfvars file may contain sensitive data such as your key values that should not be exposed to third parties. If you are using the Git system for storing and versioning Terraform configurations, make sure that the file is not committed to the repository. To avoid this, include an appropriate exclusion in .gitignore. Also, if you share your Terraform configuration with other people, make sure you don’t share terraform.tfvars. Leaking keys can result in third parties gaining control over your infrastructure.

You can obtain your secret_key and access_key values in the Cloud management console. Сlick the user login in the top right corner, select Profile Get API access settings.

CROC Cloud supports 2084-bit RSA keys. An SSH key can be generated, for example, by the command:

ssh-keygen -b 2048 -t rsa

Set the public key as the public_key value.

pubkey_name must include letters and digits only. bucket_name may additionally include dots and hyphens (see bucket naming conventions).

When all variables are described, and their values are set, you can start describing the main configuration.

Main configuration — main.tf#

The code is written in the main configuration file main.tf and ensures the future automatic performance of all critical actions on the infrastructure.

The configuration consists of code blocks, each of which, as a rule, is responsible for actions on objects of a particular type, for example, VMs or security groups. In Terraform, such blocks are called resources. Next, one by one, we consider all resource blocks required to describe the above configuration. Each block has comments explaining the changes being made.

First, create a VPC to isolate the project resources at the network layer:

Create VPC
resource "aws_vpc" "vpc" {
  # Specify an IP address for the VPC network in CIDR notation (IP/Prefix)
  cidr_block         = "172.16.8.0/24"
  # Enable support for the domain name resolution using CROC Cloud DNS servers
  enable_dns_support = true

  # Assign the Name tag to the created resource
  tags = {
    Name = "My project"
  }
}

Next, define a subnet in the VPC we have just created (the CIDR block of the subnet must belong to the address space allocated to the VPC):

Creating a subnet
resource "aws_subnet" "subnet" {
  # Specify the availability zone, in which the subnet will be created
  # Take its value from the az variable
  availability_zone = var.az
  # Use the same CIDR block of IP addresses for the subnet as for the VPC
  cidr_block        = aws_vpc.vpc.cidr_block
  # Specify the VPC where the subnet will be created
  vpc_id            = aws_vpc.vpc.id
  # Create a subnet only after creating a VPC
  depends_on        = [aws_vpc.vpc]

  # Include the az variable value and the Name tag for the VPC in the Name tag for the subnet
  tags = {
    Name = "Subnet in ${var.az} for ${lookup(aws_vpc.vpc.tags, "Name")}"
  }
}

Next, add a public SSH key, which will later be used to access the VM:

Add SSH key
resource "aws_key_pair" "pubkey" {
  # Specify the SSH key name (the value is taken from the pubkey_name variable)
  key_name   = var.pubkey_name
  # and public key content
  public_key = var.public_key
}

Create a bucket in the object storage to store website data and backups:

Create bucket
resource "aws_s3_bucket" "bucket" {
  provider = aws.noregion
  # Specify the storage name from the bucket_name variable
  bucket = var.bucket_name
  # Specify access permissions
  acl    = "private"
}

Allocate an Elastic IP to enable access to the web application server from the outside:

Allocate Elastic IP
resource "aws_eip" "eips" {
  # Specify the number of allocated EIPs in the eips_count variable –
  # this allows you to immediately allocate the required number of EIPs.
  # In our case, the address is allocated to the first server only
  count = var.eips_count
  # Allocate within our VPC
  vpc = true
  # and only after the VPC creation
  depends_on = [aws_vpc.vpc]

  # Take the host name of the future VM from the hostnames variable with the array index
  # as the value of the Name tag
  tags = {
    Name = "${var.hostnames[count.index]}"
  }
}

Then create two security groups: one to allow access from all addresses over ports 22, 80 and 443, and the second to allow full access within itself. Later, add a VM with a web application to the first group and place both our servers in the second so that they can interact with each other:

Creating security groups
# Create a security group to enable access from the outside
resource "aws_security_group" "ext" {
  # Within our VPC
  vpc_id = aws_vpc.vpc.id
  # specify the security group name
  name = "ext"
  # and description
  description = "External SG"

  # Define inbound rules
  dynamic "ingress" {
    # Specify the name of the variable, which will be used
    # to iterate over all given ports
    iterator = port
    # Iterate over ports from the allow_tcp_ports port list
    for_each = var.allow_tcp_ports
    content {
      # Set the range of ports (in our case, it consists of one port),
      from_port = port.value
      to_port   = port.value
      # protocol,
      protocol = "tcp"
      # and source IP address in CIDR notation (IP/Prefix)
      cidr_blocks = ["0.0.0.0/0"]
    }
  }

  # Define an outbound rule to enable all outbound IPv4 traffic
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  depends_on = [aws_vpc.vpc]

  tags = {
    Name = "External SG"
  }
}

# Create an internal security group,
# within which all traffic between its members will be allowed
resource "aws_security_group" "int" {
  vpc_id      = aws_vpc.vpc.id
  name        = "int"
  description = "Internal SG"

  ingress {
    from_port = 0
    to_port   = 0
    protocol  = "-1"
    self      = true
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  depends_on = [aws_vpc.vpc]

  tags = {
    Name = "Internal SG"
  }
}

Now write a block of code to create VMs:

Creating instances
resource "aws_instance" "vms" {
  # Take the number of VMs to create from the vms_count variable
  count = var.vms_count
  # the image ID to create the instance is taken from the vm_template variable
  ami = var.vm_template
  # the instance type of the VM to be created is taken from the vm_instance_type variable
  instance_type = var.vm_instance_type
  # Assign the instance an internal IP address from the previously created subnet in the VPC
  subnet_id = aws_subnet.subnet.id
  # Connect the internal security group to the created instance
  vpc_security_group_ids = [aws_security_group.int.id]
  # Add the previously created public SSH key to the server
  key_name = var.pubkey_name
  # Do not allocate or assign an external Elastic IP to the instance
  associate_public_ip_address = false
  # Activate monitoring of the instance
  monitoring = true

  # Create an instance only after the creation of:
  # — subnet
  # — internal security group
  # — public SSH key
  depends_on = [
    aws_subnet.subnet,
    aws_security_group.int,
    aws_key_pair.pubkey,
  ]

  tags = {
    Name = "VM for ${var.hostnames[count.index]}"
  }

  # Create a volume to be attached to an instance
  ebs_block_device {
    # Instruct the system to delete the volume along with the instance
    delete_on_termination = true
    # Specify the device name in the format "disk<N>",
    device_name = "disk1"
    # its type
    volume_type = var.vm_volume_type
    # and size
    volume_size = var.vm_volume_size

    tags = {
      Name = "Disk for ${var.hostnames[count.index]}"
    }
  }
}

After creating instances, assign an external security group to the first one:

Assigning security groups
resource "aws_network_interface_sg_attachment" "sg_attachment" {
  # Get the external security group ID
  security_group_id    = aws_security_group.ext.id
  # and the network interface ID of the first instance
  network_interface_id = aws_instance.vms[0].primary_network_interface_id
  # Assign a security group only after the creation of
  # respective instance and security group
  depends_on = [
    aws_instance.vms,
    aws_security_group.ext,
  ]
}

And an external Elastic IP:

Elastic IP association
resource "aws_eip_association" "eips_association" {
  # Get the number of created EIPs
  count         = var.eips_count
  # and assign each of them to instances one by one
  instance_id   = element(aws_instance.vms.*.id, count.index)
  allocation_id = element(aws_eip.eips.*.id, count.index)
}

Output values — outputs.tf#

The outputs.tf file describes all values whose result becomes known after applying the configuration plan, as consecutive output blocks.

The configuration is completed with a single output block in our case. This block outputs the Elastic IP address of the web application server to the terminal so that the user does not need to look for it in the cloud web interface:

outputs.tf
output "ip_of_webapp" {
  description = "IP of webapp"
  # Take the value of the public IP address of the first instance
  # and output it when Terraform finished its work
  value       = aws_eip.eips[0].public_ip
}

Thus, we can right away copy the IP address for the server connection and continue working with it.

Use of a ready-to-use configuration#

The described actions result in a Terraform configuration consisting of five files:

  • providers.tf — a file with the settings for connecting to, and interacting with, services or platforms that will be used to deploy the infrastructure;

  • variables.tf — a description file of all used variables and their default values;

  • terraform.tfvars— a file with variable values, including secret and access keys, which is why it should be stored in a secure place inaccessible to third parties;

  • main.tf — the main configuration file describes the entire project infrastructure that Terraform manages;

  • outputs.tf — file with description of output values.

To deploy the infrastructure from this configuration, follow the steps below:

  1. Clone the repository and navigate to the folder with configuration files:

    git clone https://github.com/C2Devel/terraform-examples.git && cd terraform-examples/quick_start
    
  2. Copy the environment variable template with their values from the example file:

    cp terraform.tfvars.example terraform.tfvars
    

    Be sure to make the necessary changes to the new file. To get the minimum working configuration, specify your secret_key and access_key in it to work with the CROC Cloud API.

  3. Run the initialization command:

    terraform init
    

    Terraform uses this command to initialize the configuration, download all the necessary plugins and get ready to work with the infrastructure.

  4. Run the command to generate a plan for the changes to make:

    terraform plan
    

    The terminal will display all the changes Terraform plans to make to the infrastructure.

  5. Study the output carefully. If the proposed changes are the same as expected, apply them:

    terraform apply
    

    The plan will be displayed again. Carefully double-check it. To execute the plan, type yes and press Enter.

After some time, the entire infrastructure you have described will be created in CROC Cloud. If you need to make further changes to it, you should change the current Terraform configuration and reapply the plan.

To display the values of the output variables in the terminal again, enter the command:

terraform output

To remove the infrastructure created with Terraform, you can run the following command:

terraform destroy

The terminal will display the infrastructure deletion plan. To confirm the deletion, type yes and press Enter.

Important

Be extremely careful when running this command since the entire infrastructure described in the configuration is deleted.

To sum up, the main Terraform configuration, which is directly responsible for actions on the infrastructure, consists of blocks called resources. By changing the block sequence and type, you can create that very infrastructure your project requires, like in Lego.

For additional examples of how to use Terraform, as well as supported and non-supported parameters for each resource, see the cases folder in our official terraform-examples repository on GitHub. The examples are compiled for AWS provider v3.63.0 (Terraform v0.14.0).