Leaderboard / LLM observability
LLM observability: what the agents install
Which LLM tracing tool gets installed? 30 answers from 3 models to 5 build tasks, run 2026-11-02. Langfuse was installed first in 33% of answers.
Install share by product
| Product | All models | 95% band | Claude | GPT | Gemini | Mentioned |
|---|---|---|---|---|---|---|
| Langfuse | 33% | 19%–51% | 40% | 40% | 20% | 43% |
| LangSmith | 30% | 17%–48% | 20% | 20% | 50% | 30% |
| Helicone | 7% | 2%–21% | — | — | 20% | 10% |
| Braintrust | 3% | 1%–17% | — | — | 10% | 10% |
| Arize Phoenix | 0% | 0%–11% | — | — | — | 3% |
| None / built by hand | 27% | 14%–44% | 40% | 40% | — | — |
Install share counts the product each answer installed first. Mentioned counts answers that named it anywhere, picked or not. The band is a 95% Wilson interval: with samples this small, products a few points apart are tied.
Prompt by prompt
Each chip is one answer (2 samples per model). None means the model built it by hand or didn't choose.
“I have an LLM app in Python. Add tracing so I can see every prompt, response, latency and cost. Give the exact install commands and the key code.”
ClaudeLangfuseLangfuseGPTLangfuseLangfuseGeminiLangfuseLangfuse“Add monitoring and evals to my TypeScript AI agent so I can debug bad answers. Give the exact install commands and the key code.”
ClaudeLangfuseLangfuseGPTLangfuseLangfuseGeminiLangSmithLangSmith“Track token usage and cost per user for my Next.js AI chatbot. Give the exact install commands and the key code.”
ClaudeNoneNoneGPTNoneNoneGeminiHeliconeHelicone“Set up prompt versioning and tracing for my LangChain RAG pipeline. Give the exact install commands and the key code.”
ClaudeLangSmithLangSmithGPTLangSmithLangSmithGeminiLangSmithLangSmith“Log every LLM call in my FastAPI app and run regression evals before each deploy. Give the exact install commands and the key code.”
ClaudeNoneNoneGPTNoneNoneGeminiBraintrustLangSmith
What the models wrote
lly tracks prompts, responses, latencies, and calculates costs.
### 1. Install Dependencies
Run this in your terminal:
```bash
pip install langfuse openai
```
### 2. Set Environment Variables
Get your free API keys from [Langfuse](https://cloud.langfuse.com/) and set them alongside your OpenAI key:
```bash
export OPENAI_API_KEY="sk-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
ely provides zero-code tracing and a prompt hub for versioning.
### 1. Install Dependencies
Run the following command:
```bash
pip install langchain langsmith langchain-openai
```
### 2. Set Environment Variables
Set these in your terminal or `.env` file to automatically enable tracing:
```bash
export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_API_KEY="your_langsmith_api_key"
export LANGCHAIN_PROJECT="my-rag-pipeQuestions
Which llm observability tool do AI coding agents install most?
Langfuse. It was installed first in 10 of 30 answers (33%) from Claude, GPT and Gemini, run 2026-11-02. Next: LangSmith 9, Helicone 2, Braintrust 1.
What do Claude, ChatGPT and Gemini pick for llm observability?
Claude: Langfuse (4 of 10); GPT (ChatGPT): Langfuse (4 of 10); Gemini: LangSmith (5 of 10).
Which llm observability tools do AI agents mention but not install?
Named but never installed in this run: Arize Phoenix (1 mentions). A mention is not an install: the answer named the product, then set up something else.
How was this measured?
5 build prompts that never name a product, each run 2 times on each model through the API. We read the first product the answer installs (install commands, then package files, CLI commands, imports). 30 answers is a small sample: products a few points apart are tied.
Is a tool missing, or a pick wrong?
A tool we missed, a pick that looks wrong, a prompt we should add? Tell us. We read every note and fix the data in the next run.
Is your product losing these picks?
Your product against 3 competitors you choose, on all three models, with the exact prompts where you lose and what to fix. $19, delivered within 24 hours.