Documentation
Start typing to search…

Artifact

Quickstart guide for Artifact: learn how to build your own knowledge bases

Welcome to the Artifact Quickstart Guide. Here you’ll learn how to create a Catalog—a RAG-ready knowledge base—and use the Ask Catalog API to query it.

Artifact transforms raw files into structured, AI-ready Catalogs. It handles parsing, chunking, and embedding, preparing your data for Retrieval-Augmented Generation (RAG) and downstream workflows.

In this guide, we'll use A Midsummer Night’s Dream as an example file.

Prerequisites

Create Catalog via Console

Follow these three steps in the Console UI:

  1. Create Catalog

    Navigate to Artifacts → + Create Catalog

    • Name: shakespeare
    • Description: Works of Shakespeare

  2. Upload File

    Open the shakespeare Catalog → Upload midsummer-nights-dream.pdf

  3. Process File

    Click Process Files → Wait for status to show Completed → Then view parsed files and chunks

Files tab view which shows the processing status

Chunks tab view which shows the text chunks after processing is complete

Create Catalog via the API

Step 1 - Create Catalog

  1. Generate an INSTILL_API_TOKEN by following the steps
  2. Copy and paste the following snippet into your Python script or terminal, replacing the ******** with your API token, and NAMESPACE_ID with your namespace ID.
from instill.clients import init_artifact_client

artifact = init_artifact_client(api_token="********", url="http://localhost:8080")
artifact.create_catalog(namespace_id="NAMESPACE_ID", name="shakespeare",
                        description="Works of Shakespeare")

export INSTILL_API_TOKEN=********

curl -X POST 'http://localhost:8080/v1alpha/namespaces/NAMESPACE_ID/catalogs' \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $INSTILL_API_TOKEN" \
--data '{
"name": "shakespeare",
"description": "Works of Shakespeare"
}'

A successful response will return details about the newly created Catalog:

{
  "catalog": {
    "catalogId": "shakespeare",
    "name": "shakespeare",
    "description": "Works of Shakespeare",
    "createTime": "2024-09-03 22:19:32.77246 +0000 UTC",
    "updateTime": "2024-09-03 22:19:32.774342789 +0000 UTC",
    "ownerName": "a7219ce0-4c6c-4dd5-8ac5-1fbf87aedd4a",
    "tags": [],
    "convertingPipelines": ["preset/indexing-convert-pdf"],
    "splittingPipelines": [
      "preset/indexing-split-text",
      "preset/indexing-split-markdown"
    ],
    "embeddingPipelines": ["preset/indexing-embed"],
    "downstreamApps": [],
    "totalFiles": 0,
    "totalTokens": 0,
    "usedStorage": "0"
  }
}

Step 2 - Upload Files

artifact.upload_catalog_file(
  namespace_id="NAMESPACE_ID",
  catalog_id="shakespeare",
  file_path="/path/to/midsummer-nights-dream.pdf")
base64 -i midsummer-nights-dream.pdf > midsummer-nights-dream-base64.txt
curl -X POST 'http://localhost:8080/v1alpha/namespaces/{namespaceID}/catalogs/shakespeare/files' \
--header "Authorization: Bearer $INSTILL_API_TOKEN" \
--header "Content-Type: application/json" \
--data @- <<EOF
  {
    "name": "midsummer-nights-dream.pdf",
    "type": "FILE_TYPE_PDF",
    "content": "$(cat midsummer-nights-dream-base64.txt)"
  }
EOF

A successful response will return details about the newly uploaded file:

{
  "file": {
    "fileUid": "cd750b8f-5769-407c-afc0-955424387863",
    "name": "midsummer-nights-dream.pdf",
    "type": "FILE_TYPE_PDF",
    "processStatus": "FILE_PROCESS_STATUS_NOTSTARTED",
    "processOutcome": "",
    "retrievable": false,
    "content": "",
    "ownerUid": "a7219ce0-4c6c-4dd5-8ac5-1fbf87aedd4a",
    "creatorUid": "a7219ce0-4c6c-4dd5-8ac5-1fbf87aedd4a",
    "catalogUid": "593e376c-87c5-49d1-9135-1e88eb5ee847",
    "createTime": "2024-09-03T22:20:29.025902Z",
    "updateTime": "2024-09-03T22:20:29.026284095Z",
    "deleteTime": null,
    "size": "1207112",
    "totalChunks": 0,
    "totalTokens": 0
  }
}

Step 3 - Process Files

Replace FILE_UID with the fileUid value from the previous Upload Files response

artifact.process_catalog_files(file_uids=["FILE_UID"])
curl -X POST 'http://localhost:8080/v1alpha/catalogs/files/processAsync' \
--header "Authorization: Bearer $INSTILL_API_TOKEN" \
--header "Content-Type: application/json" \
--data-raw '{
    "fileUids": ["FILE_UID"]
}'

A successful response will show that the files processStatus has changed to FILE_PROCESS_STATUS_WAITING. This means the file has been scheduled for processing.

Ask Catalog a Question

With the Ask Catalog API to query your processed Catalog:

artifact.question_answering(
  namespace_id="NAMESPACE_ID",
  catalog_id="shakespeare",
  question="Who are the main characters involved in the love triangle in Act I?",
  top_k=5,
)
curl -X POST 'http://localhost:8080/v1alpha/namespaces/NAMESPACE_ID/catalogs/shakespeare/ask' \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $INSTILL_API_TOKEN" \
--data '{
  "question": "Who are the main characters involved in the love triangle in Act I?",
  "topK": 5
}'

Explore your Catalog and test its knowledge by asking your own questions about the play!

A successful response will return the answer to the question, along with a list of similar chunks from the Catalog that the LLM used to generate the answer.

{
  "answer": "The main characters involved in the love triangle in Act I are Hermia, Lysander, and Demetrius.",
  "similarChunks": [
    {
      "chunkUid": "6fe30865-731f-4524-958b-5f12a6ab53a4",
      "similarityScore": 0.5944186,
      "textContent": "\n\n\n## ACT I \n\n\n### SCENE I. Athens. A room in the Palace of Theseus \npower I am made bold, Nor how it may concern my modesty In such a presence here to plead my thoughts: But I beseech your Grace that I may know The worst that may befall me in this case, If I refuse to wed Demetrius. THESEUS. Either to die the death, or to abjure For ever the society of men. Therefore, fair Hermia, question your desires, Know of your youth, examine well your blood, Whether, if you yield not to your father’s choice, You can endure the livery of a nun, For aye to be in shady cloister mew’d, To live a barren sister all your life, Chanting faint hymns to the cold fruitless moon. Thrice-blessèd they that master so their blood To undergo such maiden pilgrimage, But earthlier happy is",
      "sourceFile": "midsummer-nights-dream.pdf"
    },
    {
      "chunkUid": "838cf477-290c-4484-8f17-78be26bcd946",
      "similarityScore": 0.56566054,
      "textContent": "\n\n\n## ACT I \n\n\n### SCENE I. Athens. A room in the Palace of Theseus \n[ _Exeunt all but_ LYSANDER _and_ HERMIA_._ ] LYSANDER. How now, my love? Why is your cheek so pale? How chance the roses there do fade so fast? HERMIA. Belike for want of rain, which I could well Beteem them from the tempest of my eyes. LYSANDER. Ay me! For aught that I could ever read, Could ever hear by tale or history, The course of true love never did run smooth. But either it was different in blood— HERMIA. O cross! Too high to be enthrall’d to low. LYSANDER. Or else misgraffèd in respect of years— HERMIA. O spite! Too old to be engag’d to young. LYSANDER. Or else it stood upon the choice of friends— HERMIA. O hell! to choose love by another’s eyes! LYSANDER. Or, if there were a sympathy",
      "sourceFile": "midsummer-nights-dream.pdf"
    },
    {
      "chunkUid": "49c79ce7-e2a6-4f24-aa5f-12252911e382",
      "similarityScore": 0.5614925,
      "textContent": "\n## ACT III \n\n\n### SCENE I. The Wood. \nI pray thee, gentle mortal, sing again. Mine ear is much enamour’d of thy note. So is mine eye enthrallèd to thy shape; And thy fair virtue’s force perforce doth move me, On the first view, to say, to swear, I love thee. BOTTOM. Methinks, mistress, you should have little reason for that. And yet, to say the truth, reason and love keep little company together nowadays. The more the pity that some honest neighbours will not make them friends. Nay, I can gleek upon occasion. TITANIA. Thou art as wise as thou art beautiful. BOTTOM. Not so, neither; but if I had wit enough to get out of this wood, I have enough to serve mine own turn. TITANIA. Out of this wood do not desire to go. Thou shalt remain here whether thou wilt or no.",
      "sourceFile": "midsummer-nights-dream.pdf"
    },
    {
      "chunkUid": "25955be1-9f4a-4acd-a9a9-a337b0ed73f5",
      "similarityScore": 0.5575189,
      "textContent": "\n## ACT III \n\n### SCENE II. Another part of the wood \n\n###### HELENA. \nwhat, my love, shall I compare thine eyne? Crystal is muddy. O how ripe in show Thy lips, those kissing cherries, tempting grow! That pure congealèd white, high Taurus’ snow, Fann’d with the eastern wind, turns to a crow When thou hold’st up thy hand. O, let me kiss This princess of pure white, this seal of bliss! HELENA. O spite! O hell! I see you all are bent To set against me for your merriment. If you were civil, and knew courtesy, You would not do me thus much injury. Can you not hate me, as I know you do, But you must join in souls to mock me too? If you were men, as men you are in show, You would not use a gentle lady so; To vow, and swear, and superpraise my parts, When I am sure you",
      "sourceFile": "midsummer-nights-dream.pdf"
    },
    {
      "chunkUid": "d5f623aa-0985-4b48-aee5-a8f90e16a1f6",
      "similarityScore": 0.5506241,
      "textContent": "\n## ACT IV \n\n\n### SCENE I. The Wood \n\n###### TITANIA. \ntheir horns. _Horns, and shout within._ DEMETRIUS, LYSANDER, HERMIA _and_ HELENA _wake and start up._ Good morrow, friends. Saint Valentine is past. Begin these wood-birds but to couple now? LYSANDER. Pardon, my lord. _He and the rest kneel to_ THESEUS_._ THESEUS. I pray you all, stand up. I know you two are rival enemies. How comes this gentle concord in the world, That hatred is so far from jealousy To sleep by hate, and fear no enmity? LYSANDER. My lord, I shall reply amazedly, Half sleep, half waking; but as yet, I swear, I cannot truly say how I came here. But, as I think (for truly would I speak) And now I do bethink me, so it is: I came with Hermia hither. Our intent Was to be gone from Athens, where",
      "sourceFile": "midsummer-nights-dream.pdf"
    }
  ]
}

Next Steps

Updated about 6 hours ago