Python build hooks

Run custom Python build scripts
Airplane allows you to run custom build scripts before and after dependencies are installed. This lets you to customize your task's environment and perform additional setup steps your tasks may require.

Build hooks in airplane.yaml configuration file

The simplest place to specify build hooks is in the airplane.yaml configuration file.
yaml
Copied
1
# airplane.yaml
2


3
python:
4
  version: "3.11"
5
  preinstall: echo "preinstall"
6
  postinstall: echo "postinstall"

Build hook files
You can also specify build hooks in their own files. This is useful for writing more complex build hooks that span multiple lines.
  • To run a script before dependency installation, create a file with the name airplane_preinstall.sh in the root directory of your project.
  • To run a script after dependency installation, create a file with the name airplane_postinstall.sh in the root directory of your project.
Airplane runs your custom build scripts as executables.

Example: Web scraping with selenium
Let's walk through an example of how we can use custom build scripts to set up selenium for web scraping in Python.
  1. We first need a requirements.txt file that specifies the Python selenium dependency.
Copied
1
selenium==4.4.3
  1. Then we add our custom build script called airplane_postinstall.sh that installs Firefox and geckodriver. Both are needed for selenium.
bash
Copied
1
#!/bin/bash
2


3
set -euo pipefail
4


5
GECKODRIVER_VER=v0.31.0
6
FIREFOX_VER=91.0
7


8
apt update \
9
   && apt upgrade -y \
10
   && apt install -y
11


12
# Install firefox
13
apt install -y \
14
       firefox-esr \
15
       libx11-xcb1 \
16
       libdbus-glib-1-2 \
17
   && curl -sSLO https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VER}/linux-x86_64/en-US/firefox-${FIREFOX_VER}.tar.bz2 \
18
   && tar -jxf firefox-* \
19
   && mv firefox /opt/ \
20
   && chmod 755 /opt/firefox \
21
   && chmod 755 /opt/firefox/firefox
22


23
# Install geckodriver
24
curl -sSLO https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VER}/geckodriver-${GECKODRIVER_VER}-linux64.tar.gz \
25
   && tar zxf geckodriver-*.tar.gz \
26
   && mv geckodriver /usr/bin/
  1. Lastly, we add the Python script, selenium_demo.py.
python
Copied
1
import airplane
2


3
from selenium import webdriver
4
from selenium.webdriver import FirefoxOptions
5


6
@airplane.task()
7
def my_task():
8
    opts = FirefoxOptions()
9
    opts.add_argument("--headless")
10
    with webdriver.Firefox(options=opts) as driver:
11
        driver.get("https://airplane.dev")
12
        return {"page_source_len": len(driver.page_source)}
For the full source code, see the python_selenium_example template on GitHub. You can make a local copy of the template with the Airplane CLI:
Copied
1
airplane init --template=python_selenium_example
PrivacyTermsSupport