LiteLLM

In this tutorial, we will guide you through the steps to create a Chainlit application integrated with LiteLLM Proxy The benefits of using LiteLLM Proxy with Chainlit is:

You can call 100+ LLMs in the OpenAI API format
Use Virtual Keys to set budget limits and track usage
see LLM API calls in a step in the UI, and you can explore them in the prompt playground.

You shouldn’t configure this integration if you’re already using another integration like Langchain or LlamaIndex. Both integrations would record the same generation and create duplicate steps in the UI.

Prerequisites

Before getting started, make sure you have the following:

A working installation of Chainlit
The OpenAI package installed
LiteLLM Proxy Running
A LiteLLM Proxy API Key
Basic understanding of Python programming

Step 1: Create a Python file

Create a new Python file named app.py in your project directory. This file will contain the main logic for your LLM application.

Step 2: Write the Application Logic

In app.py, import the necessary packages and define one function to handle messages incoming from the UI.

from openai import AsyncOpenAI
import chainlit as cl
client = AsyncOpenAI(
    api_key="anything",            # litellm proxy virtual key
    base_url="http://0.0.0.0:4000" # litellm proxy base_url
)

# Instrument the OpenAI client
cl.instrument_openai()

settings = {
    "model": "gpt-3.5-turbo", # model you want to send litellm proxy
    "temperature": 0,
    # ... more settings
}

@cl.on_message
async def on_message(message: cl.Message):
    response = await client.chat.completions.create(
        messages=[
            {
                "content": "You are a helpful bot, you always reply in Spanish",
                "role": "system"
            },
            {
                "content": message.content,
                "role": "user"
            }
        ],
        **settings
    )
    await cl.Message(content=response.choices[0].message.content).send()

Step 3: Run the Application

To start your app, open a terminal and navigate to the directory containing app.py. Then run the following command:

chainlit run app.py -w

The -w flag tells Chainlit to enable auto-reloading, so you don’t need to restart the server every time you make changes to your application. Your chatbot UI should now be accessible at http://localhost:8000.

Get Started

Deploy

Basic Concepts

Advanced Features

Data Persistence

Authentication

Customisation

Backend

Guides

Migration Guides

Prerequisites

Step 1: Create a Python file

Step 2: Write the Application Logic

Step 3: Run the Application

​Prerequisites

​Step 1: Create a Python file

​Step 2: Write the Application Logic

​Step 3: Run the Application

Prerequisites

Step 1: Create a Python file

Step 2: Write the Application Logic

Step 3: Run the Application