The Milvus component is a data component that allows users to build and search vector datasets.
It can carry out the following tasks:
Alpha
The component definition and tasks are defined in the definition.yaml and tasks.yaml files respectively.
In order to communicate with Milvus, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup
block, or you can create a Connection from
the Integration Settings
page and reference the whole setup
as setup: ${connection.<my-connection-id>}
.
Field | Field ID | Type | Note |
---|
Milvus URL Endpoint (required) | url | string | Fill in your Milvus public URL endpoint with port number, e.g http://3.25.202.142:19530. |
Milvus Username (required) | username | string | Fill in your Milvus username. |
Milvus Password (required) | password | string | Fill in your Milvus password. |
Perform a vector search on a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_VECTOR_SEARCH |
Collection Name (required) | collection-name | string | The name of the collection to perform vector search on. |
Partition Name | partition-name | string | The name of the partition to vector search the data from. |
Vector (required) | vector | array[number] | An array of dimensions for the vector search. |
Vector Field (required) | vector-field | string | The name of the field to perform vector search on. |
Fields | fields | array[string] | The fields to return in the data. If empty then all fields will be returned. |
Limit (required) | limit | integer | The limit of the data to return. |
Filter | filter | string | The properties filter to be applied to the data with milvus scalar filter, please refer to filter-search. |
Offset | offset | integer | The offset of the data to return. |
Grouping Field | grouping-field | string | The name of the field to group the data by, please refer to Grouping-search. |
Search Parameters | search-params | object | The search parameters to be applied to the data with milvus search parameters, please refer to Search-parameters. |
Output | Field ID | Type | Description |
---|
Result | result | object | Result of the vector search operation. |
Status | status | string | Vector search status. |
Output Objects in Vector Search
Result
Field | Field ID | Type | Note |
---|
Data | data | array | The points returned from the vector search operation. |
IDs | ids | array | The ids returned from the vector search operation. |
Metadata | metadata | array | The metadata returned from the vector search operation. |
Vectors | vectors | array | The vectors returned from the vector search operation. |
Insert a vector data into a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_UPSERT |
Collection Name (required) | collection-name | string | The name of the collection to upsert the data into. |
Partition Name | partition-name | string | The name of the partition to upsert the data from. If empty then default partition will be used. |
Data (required) | data | object | The data. |
Output | Field ID | Type | Description |
---|
Status | status | string | Upsert status. |
Insert a batch of vector data into a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_BATCH_UPSERT |
Collection Name (required) | collection-name | string | The name of the collection to upsert the data into. |
Partition Name | partition-name | string | The name of the partition to upsert the data from. If empty then default partition will be used. |
Array Data (required) | array-data | array[object] | The data. |
Output | Field ID | Type | Description |
---|
Status | status | string | Batch upsert status. |
Delete vector data from a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DELETE |
Collection Name (required) | collection-name | string | The name of the collection to delete the data from. |
Partition Name | partition-name | string | The name of the partition to delete the data from. If empty then default partition will be used. |
Filter | filter | string | The properties filter to be applied to the data with milvus scalar filter, please refer to Filtered-search. |
Output | Field ID | Type | Description |
---|
Status | status | string | Delete status. |
Create a collectio, please refer to Create-Collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_COLLECTION |
Collection Name (required) | collection-name | string | The name of the collection to create. |
Dimension (required) | dimension | integer | The dimension of the collection. |
ID Type | id-type | string | The type of the id. |
Schema | schema | object | The schema of the collection. |
Auto ID | auto-id | boolean | Whether to auto generate id. |
Metric Type | metric-type | string | The metric type of the collection.
Enum values |
Index Parameters | index-params | array[object] | The index parameters to be applied to the collection with milvus index parameters, please refer to Create. |
Parameters | params | object | The parameters to be applied to the collection with milvus parameters, please refer to Parameters. |
Output | Field ID | Type | Description |
---|
Status | status | string | Create collection status. |
Drop a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_COLLECTION |
Collection Name (required) | collection-name | string | The name of the collection to drop. |
Output | Field ID | Type | Description |
---|
Status | status | string | Drop collection status. |
Create a partition in a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_PARTITION |
Collection Name (required) | collection-name | string | The name of the collection to create the partition in. |
Partition Name (required) | partition-name | string | The name of the partition to create. |
Output | Field ID | Type | Description |
---|
Status | status | string | Create partition status. |
Drop a partition from a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_PARTITION |
Collection Name (required) | collection-name | string | The name of the collection to drop the partition from. |
Partition Name (required) | partition-name | string | The name of the partition to drop. |
Output | Field ID | Type | Description |
---|
Status | status | string | Drop partition status. |
Create an index in a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_INDEX |
Collection Name (required) | collection-name | string | The name of the collection to create the index in. |
Index Parameters (required) | index-params | object | The index parameters to be applied to the collection with milvus index parameters, please refer to Create. |
Output | Field ID | Type | Description |
---|
Status | status | string | Create index status. |
Drop an index from a collection
Input | Field ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_INDEX |
Collection Name (required) | collection-name | string | The name of the collection to drop the index from. |
Index Name (required) | index-name | string | The name of the index to drop. |
Output | Field ID | Type | Description |
---|
Status | status | string | Drop index status. |