Self-hosted agent security model
Airplane's self-hosted agents use a hybrid hosting model to allow sensitive compute to run on your own infrastructure, while leaving the complicated aspects of scaling and maintaining the core service to the Airplane engineering team.
When self-hosting agents, you'll typically use Terraform, CloudFormation, or a similar tool to deploy a set of resources to your cloud provider. Under the hood, these resources will typically include containers running the Airplane agent, plus use of a scheduler like Amazon ECS or Kubernetes to scale and run the various tasks your team creates. The tasks are executed through containers that run within VPC subnets that you specify.
Knowledge of Docker, Kubernetes, etc. is not necessary to install and operate Airplane agents! These technologies are used under the hood, but once deployed agents are self-managing and require close to zero maintenance.
The Airplane platform and API continues to serve as a central coordination hub for self-hosted agents. Agents are configured with an API key, which allows agents and task runners to communicate with the Airplane API.
The agent creates zero or more runners for each task based on signals from the API, including the number of active executions ("runs"), page loads in the Airplane UI, and predictions of future run activity based on recently finished task executions. As these signals change, the agent modifies the runner state in real-time.
When a task begins execution, the Airplane API assigns the run to a runner for the associated task. The API provides the runner with the information necessary to execute the run, including the parameters for the run and configured environment variables. As the run proceeds, the runner sends status changes, logs, and outputs back to the Airplane API.
After the run finishes, the task runner remains up in an idle state so it can quickly handle follow-up runs for the same task. If no more runs are started for the associated task, then the agent may remove the runner based on the signalling framework described above.
By default, logs and outputs are sent back to Airplane's servers. These can be stored in your own infrastructure by using the self-hosted storage feature.
In order to operate correctly, the agent and task runner instances need outbound, TCP network access to the addresses
temporal-api.airplane.dev:443. No inbound access is required unless self-hosted storage is enabled.
Airplane supports config variables as a way to securely store secrets like database passwords and API keys. When used in a run, these values are passed from the API to the agent to start the run.
See Managing secrets for more details.