Shell
Write shell scripts in fully customizable Docker environments.
Shell tasks allow you to write highly customized scripts that run within Airplane. You can use
shell tasks to run one-off bash scripts, or even run Ruby scripts within a fully customized
environment.
By default, shell tasks run in an Ubuntu-based image. However, with a shell task you can use your
own Dockerfile, which opens the door to running different operating systems, custom packages, and
custom runtimes.
Getting started
Getting started
This guide will get you up and running with an example shell task.
We'll use the Airplane CLI
airplane
to demonstrate how to get started with a shell task.If you haven't yet, first install the Airplane CLI by following the instructions at
Installing the CLI.
Create your task
Create your task
Once you've installed and logged in to the CLI, navigate to the folder where you want to create your
task and run:
bashCopied1airplane tasks init
You'll be prompted for some information:
- Enter a name for the task.
- Select
Shell
as the kind. - Enter a name for the script and the definition file. The script contains the task code (e.g.
delete_user_by_email.sh
) while the definition file contains task metadata (e.g.delete_user_by_email.task.yaml
).
Parameters
Parameters
Parameters allow you to prompt users for input before triggering your task. See
Parameters for more information.
Open up the definition file (the file with extension
.task.yaml
) to add parameters to the
task.Example:
yamlCopied1parameters:2- name: User email3slug: user_email4type: shorttext5description: The email address of the user
Params are passed into your script as environment variables of form
PARAM_{SLUG}
, e.g.
PARAM_USER_EMAIL
.Environment variables
Environment variables
Environment variables can be entered directly ("from value"), or you can reference
Config variables ("from config variable"). Use config variables for secrets
and/or values you want to share across multiple tasks.
yamlCopied1shell:2envVars:3# From value4MY_ENV:5value: production6# From config variable7MY_SECRET:8config: MY_SECRET_CONFIG_VAR
Access environment variables in your script using standard patterns. For example:
bashCopied1echo $MY_ENV
Develop your script
Develop your script
By default, Airplane runs your scripts on Ubuntu 22.04.
A minimal shell script looks like this (and gets generated by
airplane tasks init
):bashCopied1#!/bin/bash2# Params are in environment variables as PARAM_{SLUG}, e.g. PARAM_USER_ID3echo "Hello World!"4echo "Printing env for debugging purposes:"5env
Airplane executes your script as an executable, looking for the
shebang at the top of the file (e.g.
#!/bin/bash
) to determine how to run the script.The parameters you've specified earlier are available as environment variables named after the
parameter slug. For example, parameter with slug
user_email
is passed in as environment
variable PARAM_USER_EMAIL
.Customize the environment with Dockerfile
Customize the environment with Dockerfile
By default, Airplane will run your shell script inside an Ubuntu-based image with common tools
installed. You may want to customize this if you need a specific operating system, need to install a
specific library, etc.
To do this, you can define a custom Dockerfile that Airplane will use to build and configure
your environment. Airplane looks for the first file named
Dockerfile.airplane
in the same
directory as your shell script, or in a parent directory. If not found, Airplane will look for a
file named Dockerfile
instead.The directory that we find Dockerfile.airplane / Dockerfile in becomes the root directory of your
task. Airplane will include all the files in your root directory (minus what is
ignored by .airplaneignore) when building your task.
For example, you might have the following layout:
Copied1/2Dockerfile.airplane3prod_tasks/4task_a.sh5task_a.task.yaml6task_b.sh7task_b.task.yaml8other_tasks/9task_c.sh10task_c.task.yaml
When running
airplane deploy prod_tasks/task_a.task.yaml
, Dockerfile.airplane
is used to build
the image that task_a.sh
is run in. By organizing your files like this, you can share the same
Dockerfile among multiple tasks.Airplane will
exec
the script you deploy, so you should ensure your script is a valid executable.
Typically you just need to include a shebang at
the top of your file, e.g. #!/bin/bash
to run a bash script or even #!/usr/bin/env python
to run
a Python script:pythonCopied1# python_script.py2#!/usr/bin/env python3print("Hello world!")
Deploy to Airplane
Deploy to Airplane
Finally, run
deploy
to push your script to Airplane:bashCopied1airplane deploy ./your_task_file.sh
Once deployed, go to the Tasks page to see and run your task!
Examples
Examples
Redis
Redis
Here's a simple example with an Ubuntu image and Redis, where we'll build a task that runs
DEL <key>
:First, let's configure a shell task with a
Key
parameter:yamlCopied1# clear_redis_key.task.yaml2name: Clear Redis key3slug: clear_redis_key4description: Deletes a key from Redis5parameters:6- name: Key7slug: key8type: shorttext9description: The key to delete10shell:11entrypoint: clear_redis_key.sh12envVars:13REDIS_URI:14value: "redis://user:pass@host:6379/0"
And ensure you've configured a
REDIS_URI
environment variable the server host and password:
Above, we're using a hardcoded value for
REDIS_URI
. You can also use Config
Variables to store secrets.Once you've created the task, you can use the following Dockerfile and shell script to connect to
Redis and run a command:
dockerfileCopied1# Dockerfile.airplane2FROM ubuntu:22.0434# Install redis-cli via the redis-server package5RUN apt-get update -q && apt-get -y install redis-server
bashCopied1#!/bin/bash2# clear_redis_key.sh34# All you have to do is run redis-cli:5redis-cli -u $REDIS_URI del $PARAM_KEY
Ruby
Ruby
By using a custom Dockerfile, you can set up a full Ruby environment and run Ruby scripts:
dockerfileCopied1# Dockerfile.airplane2FROM ruby:2.634WORKDIR /app5COPY Gemfile /app/Gemfile6COPY Gemfile.lock /app/Gemfile.lock7RUN bundle install --deployment89# Assumes there are other files necessary to run your script.10COPY . .
And your script might look like this, if you're using Ruby on Rails and wanted to run it via
rails runner
:rubyCopied1#!/usr/bin/env -S bundle exec rails runner2# ruby_on_rails.sh34puts "Running!"5article = Article.new(title: "Hello Rails", body: "I am on Rails!")6puts "Saving..."7article.save