Self-hosted agent security model
Airplane's self-hosted agents use a hybrid hosting model to allow sensitive compute to run on your own infrastructure, while leaving the complicated aspects of scaling and maintaining the core service to the Airplane engineering team.
When self-hosting agents, you'll typically use Terraform, CloudFormation, or a similar tool to deploy a set of resources to your cloud provider. Under the hood, these resources will typically include containers running the Airplane agent, plus use of a scheduler like Amazon ECS or Kubernetes to scale and run the various tasks your team creates. The tasks are executed through containers that run within VPC subnets that you specify.
Knowledge of Docker, Kubernetes, etc. is not necessary to install and operate Airplane agents! These technologies are used under the hood, but once deployed agents are self-managing and require close to zero maintenance.
The Airplane platform and API continues to serve as a central coordination hub for self-hosted agents. Agents are configured with an API key, which allow agents to communicate with the Airplane API.
When a task begins execution, the Airplane API assigns the execution ("run") to an agent. The API provides the agent with the information necessary to execute the run, including the parameters to the run and configured environment variables.
The agent executes the run, and as the run proceeds the agent sends back status changes, logs, and output back to the Airplane API.
Today, logs and outputs are sent back to Airplane's servers. A future release will allow customers to store logs/outputs in their own infrastructure and completely avoid Airplane servers. If you'd like to be a beta tester, send an email to email@example.com.
Airplane supports config variables as a way to securely store secrets like database passwords and API keys. When used in a run, these values are passed from the API to the agent to start the run.