BigQuery

The BigQuery component is a data component that allows users to insert data to BigQuery tables. It can carry out the following tasks:

Insert
Read

Release Stage

Alpha

Configuration

The component definition and tasks are defined in the definition.yaml and tasks.yaml files respectively.

Setup

In order to communicate with Google, the following connection details need to be provided. You may specify them directly in a pipeline recipe as key-value pairs within the component's setup block, or you can create a Connection from the Integration Settings page and reference the whole setup as setup: ${connection.<my-connection-id>}.

Field	Field ID	Type	Note
JSON Key File Contents (required)	`json-key`	string	Contents of the JSON key file with access to the bucket.
BigQuery Project ID (required)	`project-id`	string	Fill in your BigQuery Project ID.
BigQuery Dataset ID (required)	`dataset-id`	string	Fill in your BigQuery Dataset ID.
BigQuery Table Name (required)	`table-name`	string	Fill in your BigQuery Table Name.

Supported Tasks

Insert

Insert data to BigQuery.

Input	Field ID	Type	Description
Task ID (required)	`task`	string	`TASK_INSERT`
Data	`data`	object	The data to be inserted to BigQuery.

Output	Field ID	Type	Description
Status	`status`	string	Status of the upload operation.

Read

Read data from BigQuery.

Input	Field ID	Type	Description
Task ID (required)	`task`	string	`TASK_READ`
Filtering	`filtering`	string	The filter to be applied to the data with SQL syntax, which starts with WHERE clause.

Output	Field ID	Type	Description
Data	`data`	array[object]	The data to be read from BigQuery.