ChatGeneration
A ChatGeneration contains all of the data that has been sent to a message-based LLM (like GPT-4) as well as the response from the LLM.
It is designed to be passed to a Step to enable the Prompt Playground.
Attributes
The LLM model used.
The completion of the prompt. More info in GenerationMessage.
The variables to the prompt, represented as a dictionary.
The list of messages that form the prompt if in chat mode.
The provider of the LLM, such as “openai-chat”.
The list of tools that can be used by the LLM.
The settings the LLM provider was called with. Can include OpenAI tools and functions.
The error returned by the LLM.
The tags you want to assign to this generation.
Total tokens sent to the LLM.
Total tokens returned by the LLM
Total tokens used by the LLM.
Time from the request to receiving the first token.
LLM speed of token generation, in tokens per second.
Total duration of the LLM request.
Example
import os
from chainlit.playground.providers import ChatOpenAI
import chainlit as cl
# If no OPENAI_API_KEY is available, the ChatOpenAI provider won't be available in the prompt playground
os.environ["OPENAI_API_KEY"] = "sk-..."
template = """Hello, this is a template.
This is a variable1 {variable1}
And this is variable2 {variable2}
"""
variables = {
"variable1": "variable1 value",
"variable2": "variable2 value",
}
settings = {
"model": "gpt-3.5-turbo",
"temperature": 0,
# ... more settings
}
@cl.step(type="llm")
async def fake_llm():
# Pretend we're calling the LLM
await cl.sleep(2)
fake_completion = "The openai completion"
prompt = template.format(**variables)
# Add the generation to the current step
cl.context.current_step.generation = cl.ChatGeneration(
model=settings["model"],
provider=ChatOpenAI.id,
message_completion={
"content": fake_completion, "role": "assistant"
},
variables=variables,
settings=settings,
messages=[
{
"content": "You are a helpful assistant.", "role": "system"
},
{
"content": prompt, "role": "user"
},
],
)
return fake_completion
@cl.on_chat_start
async def start():
await fake_llm()
A chat completion displayed in the Prompt Playground
Was this page helpful?