Self-hosted agent security model
Airplane's self-hosted agents use a hybrid hosting model to allow sensitive compute to run on your
own infrastructure, while leaving the complicated aspects of scaling and maintaining the core
service to the Airplane engineering team.
When self-hosting agents, you'll typically use Terraform, CloudFormation, or a similar tool to
deploy a set of resources to your cloud provider. Under the hood, these resources will typically
include containers running the Airplane agent, plus use of a scheduler like Amazon ECS or Kubernetes
to scale and run the various tasks your team creates. The tasks are executed through containers that
run within VPC subnets that you specify.
Knowledge of Docker, Kubernetes, etc. is not necessary to install and operate Airplane agents! These
technologies are used under the hood, but once deployed agents are self-managing and require close
to zero maintenance.
Authentication
The Airplane platform and API continues to serve as a central coordination hub for self-hosted
agents. Agents are configured with an API key, which allows agents and task runners to communicate
with the Airplane API.
Task execution
The agent creates zero or more runners for each task based on signals from the API, including the
number of active executions ("runs"), page loads in the Airplane UI, and predictions of future run
activity based on recently finished task executions. As these signals change, the agent modifies the
runner state in real-time.
When a task begins execution, the Airplane API assigns the run to a runner for the associated task.
The API provides the runner with the information necessary to execute the run, including the
parameters for the run and configured environment variables. As the run proceeds, the runner sends
status changes, logs, and outputs back to the Airplane API.
After the run finishes, the task runner remains up in an idle state so it can quickly handle
follow-up runs for the same task. If no more runs are started for the associated task, then the
agent may remove the runner based on the signalling framework described above.
By default, logs and outputs are sent back to Airplane's servers. These can be stored in your own
infrastructure by using the self-hosted storage feature.
Network access
In order to operate correctly, the agent and task runner instances need outbound, TCP network access
to the addresses
api.airplane.dev:443, otel-collector.airplane.dev:443, and
temporal-api.airplane.dev:443. No inbound access is required unless
self-hosted storage is enabled.Secrets management
Airplane supports config variables as a way to securely store secrets like
database passwords and API keys. When used in a run, these values are passed from the API to the
agent to start the run.
If you would like to keep secrets separate from Airplane servers, you can use your own secret store
(e.g. HashiCorp Vault or AWS Secrets Manager). From a JavaScript or Python task, you can install and
use the SDK and connect to your secret store like normal.
See Managing secrets for more details.