Documentation
Start typing to search…

Local Development

Instill Core CE is built with the microservice architecture. We use Docker Compose for local development. Each service is nothing but a running container. Developing new features simply means to develop, containerize and deploy the new codebase.

Setup

To conduct local development, firstly, spin up the latest Instill Core CE:

# Clone and move to the instill-core repository
git clone https://github.com/instill-ai/instill-core.git
cd instill-core

# Launch all services with their very latest versions
make compose-dev

Once every containers are up and running, the developer can access all services through:

  • api-gateway (localhost:8080)
    • A service to handle API requests and responses.
  • pipeline-backend(localhost:8081)
    • A service for building and managing unstructured data pipelines.
  • artifact-backend(localhost:8082)
    • A service for managing all RAG related resources.
  • model-backend(localhost:8083)
    • A service for importing and serving AI models.
  • mgmt-backend(localhost:8084)
    • A service for user account management, including authentication, authorization, admission control and usage metrics.
  • console (localhost:3000)
    • A service provides GUI user interface for accessing Instill Core CE.

Let's say, you are working on a new feature for pipeline-backend. You will need to remove the running containers first:

docker rm pipeline-backend
docker rm pipeline-backend-worker

And now move to the pipeline-backend codebase:

# Clone and move to the pipeline-backend repository
git clone https://github.com/instill-ai/pipeline-backend.git
cd pipeline-backend

Build and run the dev image

make build-dev
make dev

Now, you have the development environment set up in the container, where you can compile and run the binaries, as well as run the integration tests in the container.

Start the pipeline-backend server

docker exec -it pipeline-backend bash
go run ./cmd/migration
go run ./cmd/init
go run ./cmd/main

Start the Temporal worker

docker exec -it pipeline-backend bash
go run ./cmd/worker

Run the integration tests

During local development, you can run the integration test to make sure your latest pipeline-backend works as intended:

docker exec -it pipeline-backend bash
make integration-test API_GATEWAY_URL=api-gateway:8080 DB_HOST=pg-sql

Remove the dev container

make rm

Shut down all dependent Instill Core CE services

# In the `instill-core` repository folder
make down

Configurations

Backend Services

Instill Core CE loads a .env file that contains key/value pairs defining required environment variables. You can customize the file based on your configuration.

Besides, Instill Core CE uses Koanf library for configuration. It supports loading configuration from multiple sources and makes it available to the service. To override the default configuration, you can set the corresponding environment variables, which are passed to the Docker Compose file. Note that all configuration environment variables for each backend service are prefixed with CFG_.

Frontend Console

To access Instill Core Console, set the host by overriding the environment variables:

INSTILL_CORE_HOST={HOSTNAME}

By default HOSTNAME is set to localhost.

Updated 2 months ago