The Google Cloud Storage component is a data component that allows users to upload data to Google's Cloud Storage.
It can carry out the following tasks:
The component definition and tasks are defined in the definition.yaml and tasks.yaml files respectively.
Setup
In order to communicate with Google, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup block, or you can create a Connection from
the Integration Settings
page and reference the whole setup as setup: ${connection.<my-connection-id>}.
Field
Field ID
Type
Note
JSON Key File Contents (required)
json-key
string
Contents of the JSON key file with access to the bucket.
Supported Tasks
Upload
Upload data to Google Cloud Storage.
Input
Field ID
Type
Description
Task ID (required)
task
string
TASK_UPLOAD
Bucket Name (required)
bucket-name
string
Name of the bucket to be used for object storage.
Object Name (required)
object-name
string
The name of the object to be created.
Data (required)
data
string
The data to be saved in the object.
Output
Field ID
Type
Description
Authenticated URL (optional)
authenticated-url
string
Only users granted permission can access the object with this link.
Gsutil URI (optional)
gsutil-uri
string
File path to this resource in Cloud Storage.
Public Access (optional)
public-access
boolean
Whether the object is publicly accessible.
Public URL (optional)
public-url
string
Anyone with this link can access the object on the public Internet.
Upload Status
status
string
Status of the upload operation.
Read Objects
Read objects from Google Cloud Storage.
Input
Field ID
Type
Description
Task ID (required)
task
string
TASK_READ_OBJECTS
Bucket Name (required)
bucket-name
string
Name of the bucket to be used for object storage.
Delimiter
delimiter
string
The delimiter to use when listing objects.
Prefix
prefix
string
The prefix to use when listing objects.
Versions
versions
boolean
Whether to list all versions of an object.
Start Offset
start-offset
string
The offset to start listing objects from.
End Offset
end-offset
string
The offset to stop listing objects at.
Include Trailing Delimiter
include-trailing-delimiter
boolean
Whether to include the delimiter in the object name.
