Shell

Write shell scripts in fully customizable Docker environments.

Shell tasks allow you to write highly customized scripts that run within Airplane. You can use shell tasks to run one-off bash scripts, or even run Ruby scripts within a fully customized environment.

By default, shell tasks run in an Ubuntu-based image. However, with a shell task you can use your own Dockerfile, which opens the door to running different operating systems, custom packages, and custom runtimes.

Getting started

This guide will get you up and running with an example shell task.

We'll use the Airplane CLI airplane to demonstrate how to get started with a shell task.

If you haven't yet, first install the Airplane CLI by following the instructions at Installing the CLI.

Create your task

Once you've installed and logged in to the CLI, navigate to the folder where you want to create your task and run:

bash
Copied
1
airplane tasks init

You'll be prompted for some information:

Enter a name for the task.
Select Shell as the kind.
Enter a name for the script and the definition file. The script contains the task code (e.g. delete_user_by_email.sh) while the definition file contains task metadata (e.g. delete_user_by_email.task.yaml).

Parameters

Parameters allow you to prompt users for input before triggering your task. See Parameters for more information.

Open up the definition file (the file with extension .task.yaml) to add parameters to the task.

Example:

yaml
Copied
1
parameters:
2
  - name: User email
3
    slug: user_email
4
    type: shorttext
5
    description: The email address of the user

Params are passed into your script as environment variables of form PARAM_{SLUG}, e.g. PARAM_USER_EMAIL.

Environment variables

Environment variables can be entered directly ("from value"), or you can reference Config variables ("from config variable"). Use config variables for secrets and/or values you want to share across multiple tasks.

yaml
Copied
1
shell:
2
  envVars:
3
    # From value
4
    MY_ENV:
5
      value: production
6
    # From config variable
7
    MY_SECRET:
8
      config: MY_SECRET_CONFIG_VAR

Access environment variables in your script using standard patterns. For example:

bash
Copied
1
echo $MY_ENV

Develop your script

By default, Airplane runs your scripts on Ubuntu 22.04.

A minimal shell script looks like this (and gets generated by airplane tasks init):

bash
Copied
1
#!/bin/bash
2
# Params are in environment variables as PARAM_{SLUG}, e.g. PARAM_USER_ID
3
echo "Hello World!"
4
echo "Printing env for debugging purposes:"
5
env

Airplane executes your script as an executable, looking for the shebang at the top of the file (e.g. #!/bin/bash) to determine how to run the script.

The parameters you've specified earlier are available as environment variables named after the parameter slug. For example, parameter with slug user_email is passed in as environment variable PARAM_USER_EMAIL.

Customize the environment with Dockerfile

By default, Airplane will run your shell script inside an Ubuntu-based image with common tools installed. You may want to customize this if you need a specific operating system, need to install a specific library, etc.

To do this, you can define a custom Dockerfile that Airplane will use to build and configure your environment. Airplane looks for the first file named Dockerfile.airplane in the same directory as your shell script, or in a parent directory. If not found, Airplane will look for a file named Dockerfile instead.

The directory that we find Dockerfile.airplane / Dockerfile in becomes the root directory of your task. Airplane will include all the files in your root directory (minus what is ignored by .airplaneignore) when building your task.

For example, you might have the following layout:


Copied
1
/
2
  Dockerfile.airplane
3
  prod_tasks/
4
    task_a.sh
5
    task_a.task.yaml
6
    task_b.sh
7
    task_b.task.yaml
8
  other_tasks/
9
    task_c.sh
10
    task_c.task.yaml

When running airplane deploy prod_tasks/task_a.task.yaml, Dockerfile.airplane is used to build the image that task_a.sh is run in. By organizing your files like this, you can share the same Dockerfile among multiple tasks.

Airplane will exec the script you deploy, so you should ensure your script is a valid executable. Typically you just need to include a shebang at the top of your file, e.g. #!/bin/bash to run a bash script or even #!/usr/bin/env python to run a Python script:

python
Copied
1
# python_script.py
2
#!/usr/bin/env python
3
print("Hello world!")

Deploy to Airplane

Finally, run deploy to push your script to Airplane:

bash
Copied
1
airplane deploy ./your_task_file.sh

Once deployed, go to the Tasks page to see and run your task!

Examples

Redis

Here's a simple example with an Ubuntu image and Redis, where we'll build a task that runs DEL <key>:

First, let's configure a shell task with a Key parameter:

yaml
Copied
1
# clear_redis_key.task.yaml
2
name: Clear Redis key
3
slug: clear_redis_key
4
description: Deletes a key from Redis
5
parameters:
6
  - name: Key
7
    slug: key
8
    type: shorttext
9
    description: The key to delete
10
shell:
11
  entrypoint: clear_redis_key.sh
12
  envVars:
13
    REDIS_URI:
14
      value: "redis://user:pass@host:6379/0"

And ensure you've configured a REDIS_URI environment variable the server host and password:

Above, we're using a hardcoded value for REDIS_URI. You can also use Config Variables to store secrets.

Once you've created the task, you can use the following Dockerfile and shell script to connect to Redis and run a command:

dockerfile
Copied
1
# Dockerfile.airplane
2
FROM ubuntu:22.04
3

4
# Install redis-cli via the redis-server package
5
RUN apt-get update -q && apt-get -y install redis-server

bash
Copied
1
#!/bin/bash
2
# clear_redis_key.sh
3

4
# All you have to do is run redis-cli:
5
redis-cli -u $REDIS_URI del $PARAM_KEY

Ruby

By using a custom Dockerfile, you can set up a full Ruby environment and run Ruby scripts:

dockerfile
Copied
1
# Dockerfile.airplane
2
FROM ruby:2.6
3

4
WORKDIR /app
5
COPY Gemfile /app/Gemfile
6
COPY Gemfile.lock /app/Gemfile.lock
7
RUN bundle install --deployment
8

9
# Assumes there are other files necessary to run your script.
10
COPY . .

And your script might look like this, if you're using Ruby on Rails and wanted to run it via rails runner:

ruby
Copied
1
#!/usr/bin/env -S bundle exec rails runner
2
# ruby_on_rails.sh
3

4
puts "Running!"
5
article = Article.new(title: "Hello Rails", body: "I am on Rails!")
6
puts "Saving..."
7
article.save