CompletionGeneration
A CompletionGeneration contains all of the data that has been sent to a completion-based LLM (like Llama2) as well as the response from the LLM.
It is designed to be passed to a Step to enable the Prompt Playground.
Attributes
The LLM model used.
The LLM prompt.
The completion of the prompt.
The variables of the prompt, represented as a dictionary.
The provider of the LLM, such as “openai”.
The list of tools that can be used by the LLM.
The settings the LLM provider was called with.
The error returned by the LLM.
The tags you want to assign to this generation.
Total tokens sent to the LLM.
Total tokens returned by the LLM
Total tokens used by the LLM.
Time from the request to receiving the first token.
LLM speed of token generation, in tokens per second.
Total duration of the LLM request.
Example
import os
from chainlit.playground.providers import OpenAI
import chainlit as cl
# If no OPENAI_API_KEY is available, the OpenAI provider won't be available in the prompt playground
os.environ["OPENAI_API_KEY"] = "sk-..."
template = """Hello, this is a template.
This is a variable1 {variable1}
And this is variable2 {variable2}
"""
variables = {
"variable1": "variable1 value",
"variable2": "variable2 value",
}
settings = {
"model": "gpt-3.5-turbo",
"temperature": 0,
# ... more settings
}
# Let's pretend we made the call to openai and this is the response
completion = "The openai completion"
@cl.step(type="llm")
async def fake_llm():
# Pretend we're calling the LLM
await cl.sleep(2)
fake_completion = "The openai completion"
prompt = template.format(**variables)
# Add the generation to the current step
cl.context.current_step.generation = cl.CompletionGeneration(
model=settings["model"],
provider=OpenAI.id,
completion=fake_completion,
variables=variables,
settings=settings,
prompt=prompt,
)
return fake_completion
@cl.on_chat_start
async def start():
await fake_llm()
A completion generation displayed in the Prompt Playground
Was this page helpful?