Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.chainlit.io/llms.txt

Use this file to discover all available pages before exploring further.

In this tutorial, we will guide you through the steps to create a Chainlit application integrated with LiteLLM Proxy The benefits of using LiteLLM Proxy with Chainlit is:
You shouldn’t configure this integration if you’re already using another integration like Langchain or LlamaIndex. Both integrations would record the same generation and create duplicate steps in the UI.

​
Prerequisites

Before getting started, make sure you have the following:

​
Step 1: Create a Python file

Create a new Python file named app.py in your project directory. This file will contain the main logic for your LLM application.

​
Step 2: Write the Application Logic

In app.py, import the necessary packages and define one function to handle messages incoming from the UI. 
from openai import AsyncOpenAI
import chainlit as cl
client = AsyncOpenAI(
    api_key="anything",            # litellm proxy virtual key
    base_url="http://0.0.0.0:4000" # litellm proxy base_url
)

# Instrument the OpenAI client
cl.instrument_openai()

settings = {
    "model": "gpt-3.5-turbo", # model you want to send litellm proxy
    "temperature": 0,
    # ... more settings
}

@cl.on_message
async def on_message(message: cl.Message):
    response = await client.chat.completions.create(
        messages=[
            {
                "content": "You are a helpful bot, you always reply in Spanish",
                "role": "system"
            },
            {
                "content": message.content,
                "role": "user"
            }
        ],
        **settings
    )
    await cl.Message(content=response.choices[0].message.content).send()

​
Step 3: Run the Application

To start your app, open a terminal and navigate to the directory containing app.py. Then run the following command: 
chainlit run app.py -w
The -w flag tells Chainlit to enable auto-reloading, so you don’t need to restart the server every time you make changes to your application. Your chatbot UI should now be accessible at http://localhost:8000.