Running the agent on AWS

Deploy a self-hosted Airplane agent on AWS with ECS and Fargate
Airplane supports easy installation via Terraform or AWS CloudFormation.
To follow the installation instructions below, you'll need the following values:
  • YOUR_API_TOKEN: generate a new token by running airplane apikeys create <token name> from the Airplane CLI.
  • YOUR_TEAM_ID: get your team ID via airplane auth info or visit the Team Settings page.
If your team already uses Terraform, we recommend using the Terraform module: Install with Terraform.
Otherwise, for a faster setup you can also use AWS CloudFormation: Install with AWS CloudFormation.

Install with Terraform
If you're already using Terraform, the fastest way to set up a cluster of agents is to use the airplane-agents Terraform module.
See the Terraform module page for the full module documentation.
To deploy into an existing VPC, simply specify the subnet_ids:
hcl
Copied
1
module "airplane_agent" {
2
  source  = "airplanedev/airplane-agents/aws"
3


4
  api_token              = "YOUR_API_TOKEN"
5
  team_id                = "YOUR_TEAM_ID"
6


7
  # Set which VPC / subnets agents should live in
8
  subnet_ids = ["subnet-000", "subnet-111"]
9


10
  # Optional: Attach labels to agents for constraints
11
  agent_labels = {
12
    vpc = "123"
13
    env = "test"
14
  }
15


16
  # Optional: If set, only allows agents to execute runs from
17
  # the environment with the provided slug
18
  env_slug = ""
19
}
For details on using labels, see Execute rules & constraints.
For details on using environments, see Environments.
You should be able to terraform apply—that's it! Visit app.airplane.dev/settings/team and confirm that an agent has now appeared:

Self-hosted storage with Terraform
To also enable self-hosted storage, include the following module settings:
hcl
Copied
1
module "airplane_agent" {
2
  # ... settings from above ...
3


4
  enable_self_hosted_storage = true
5


6
  # Optional: See below for more details
7
  agent_storage_zone_slug             = "aws"
8
  agent_storage_alb_public_subnet_ids = ["subnet-000", "subnet-111"]
9
}
The zone slug, which is used to identify your storage zone in the Airplane API and UI, defaults to aws. You can optionally configure a custom slug by setting the agent_storage_zone_slug variable. Note that the slug value should contain lowercase letters and numbers only (no spaces, dashes, underscores, etc.).
If your agents are running in private subnets, set the agent_storage_alb_public_subnet_ids variable to a list of public subnet IDs that will be used by the external agent server load balancer. Otherwise, you can leave this variable unset.
Once you've applied the module, follow the verification procedure described on the self-hosted storage page.

Install with AWS CloudFormation
As an alternative to Terraform, you can use AWS CloudFormation as a simpler but less flexible method of installation.
The latest version of the CloudFormation stack is available at https://airplane-aws-stack.s3.amazonaws.com/agentv2-stack-fargate.yaml—for your convenience, you can use the following link to launch the stack: Create CloudFormation stack.
Upon opening the link above, you should see a screen with a pre-filled Amazon S3 template URL. First, if desired, change the region using the region switcher at the top of the page. Then, click Next.
On the Specify stack details screen, fill in the parameters as follows:
  • Airplane Team ID: retrieve this by running airplane auth info.
  • Airplane API Token/Airplane API Token Secret: generate this by running airplane apikeys create "Cloudformation" from the CLI.
    • Exactly one of Airplane API Token or Airplane API Token Secret should be used
    • Airplane API Token should be used if you just want to pass the token directly to the Airplane agent via environment variable.
    • Airplane API Token Secret should be used if you wish to use AWS secret manager to manage your Airplane API tokens instead.
  • KMS Key for API Token Secret: optional, if your API Token Secret requires a KMS Key, you can set this to be the required key.
  • Cluster ARN: optional. ARN for the ECS cluster to run the Airplane agents in. If left blank, creates a new cluster for the agents.
  • Subnet IDs: choose the VPC subnet(s) you want to deploy the agent into.
  • VPC ID: choose the VPC that matches the subnets you selected.
  • Service Name: fill in with the name to assign to the ECS service, it can be left as airplane-agent if it doesn't conflict with any other name.
Click Next once you've filled in the Parameters.
On the Configure stack options page, you can optionally set additional tags and settings, but you can also leave the defaults alone.
Click Next.
On the final Review page, check "I acknowledge that AWS CloudFormation might create IAM resources." towards the bottom and click Create stack.
Once the stack has been created, it'll take 2-3 minutes for the agents to boot up. Visit https://app.airplane.dev/agents and confirm that an agent has now appeared.

Self-hosted storage with CloudFormation
To also enable self-hosted storage, use the storage-enabled variant of our CloudFormation template instead of the base one: https://airplane-aws-stack.s3.amazonaws.com/agentv2-stack-fargate-storage.yaml
In addition to the parameters documented above, you'll also need to set the StorageALBPublicSubnetIDs parameter to a list of public subnet IDs for the external agent server load balancer. If you're already using public subnets for your agents, you can use the same values that are set for the SubnetIDs parameter.
The zone slug, which is used to identify your storage zone in the Airplane API and UI, defaults to aws. You can optionally configure a custom slug by setting the AgentStorageZoneSlug parameter. Note that the slug value should contain lowercase letters and numbers only (no spaces, dashes, underscores, etc.).
Once you've created the stack, follow the verification procedure described on the self-hosted storage page.

Allowing access to AWS API or private network
Both the Terraform and CloudFormation configs will create a default run IAM role and a tasks security group. Task runners created by the ECS agent will be ECS tasks with this IAM role and belonging to this security group.
If a task needs to access an AWS resource via the AWS API, you should make sure that the run IAM role has the correct policy attached.
If a task needs to access internal resouces via the network (e.g. an internal API endpoint or a RDS database), you should make sure that the VPC is selected correctly, and that the security group for the internal resource has an ingress rule that references the tasks security group to allow Airplane tasks to access that resource.

Custom tasks security group
If you need to use custom security groups for your tasks, you can set the VpcSecurityGroupIDs parameter in the CloudFormation stack, or the vpc_security_group_ids variable in the Terraform module. Task runners will be created with the provided security groups instead of the default tasks security group.

Custom IAM roles
If you need tasks to run with custom IAM roles, you can replace the default IAM role that the AWS ECS agent will use for a specific task. To do this, add an environment variable on the task with name ECS_TASK_ROLE, and set its value to the ARN of the role you want the task to execute with.

Configuring task CPU and memory
By default, each task runner is allocated 0.5 vCPU and 1GB of memory. To change this for all task runners, you can set the DefaultTaskCPU and DefaultTaskMemory (Cloud Formation) or default_task_cpu and default_task_memory (Terraform) agent config parameters.
Alternatively, to adjust this on a task-by-task basis, you can set the AIRPLANE_TASK_CPU and AIRPLANE_TASK_MEMORY environment variables in the associated tasks.
Note that these resource limits, whether set in the agent configs or the environment, are interpreted as strings in Kubernetes quantity format. The latter supports several different units, but it's usually easiest to express CPU limits in millicores (e.g., 500m for 0.5 cores, 2000m for 2 cores, etc.) and memory limits in either megabytes (e.g., 256Mi) or gigabytes (e.g., 2Gi).
Also note that Fargate only allows certain combinations of CPU and memory; see the AWS documentation for more details.

Fargate capacity
By default, AWS only allows a total of 6 Fargate vCPU across all running workloads per account region (source). If you're running more than a handful of Airplane tasks simultaneously, you'll want to request a quota increase via the AWS console.
If you hit this limit, tasks will take a long time to start, and the agent logs will show errors like capacity is unavailable at this time. Please try again later or in a different availability zone.

Image cache
The Airplane Terraform and CloudFormation configs will create a private ECR repository in your account as a pull-through cache for Airplane-hosted images. This helps improve task start performance and can also reduce your organization's network ingress costs.
The image caching feature is enabled by default, but if you want to turn it off you can either set ecr_cache = false (if using the Terraform module) or ECRCache to false (if using the CloudFormation template).
PrivacyTermsSupport