BigQuery

Connect to BigQuery databases from Airplane tasks and runbooks.
The BigQuery resource allows tasks and runbooks to connect to a BigQuery database. For a high-level introduction to resources, see the resources page.

Creating a BigQuery resource

To create a new BigQuery resource, visit resources for your team, click the "New" button, and select the BigQuery resource type. Enter your connection details in the form:
Name should be a human-readable name for your resource. You'll use this name to later reference this resource in tasks.
Project ID, Location, and Dataset can all be found in the BigQuery console under Dataset Info.
Credentials are the private keys for the service account with access to this dataset. The service account should have the permissions "BigQuery Job User" and "BigQuery User". Credentials can be found in the Google Cloud console under IAM & Admin > Service Accounts > Keys. The key should be of JSON type and can only be viewed at creation time. See "Create and manage service account keys" for more information about managing service account keys.

IP whitelisting

If your database has restrictions on incoming network connections, you'll need to ensure the Airplane IP addresses are whitelisted:
  • 34.121.89.145
  • 34.68.100.27
  • 34.72.182.225

Private networking

To connect to a database inside a private VPC, you can self-host the agent and run the agent within your own VPC. Since tasks and SQL queries run on agents, these tasks will run from within your network and be able to connect.