What is Seer?

What Seer Does

• Log retrieval events from your app using a lightweight SDK
• Compute metrics automatically from unlabeled inputs:
  • Recall: Our model enumerates the minimal requirements needed to answer a question and checks which are supported by the retrieved context. recall = covered_requirements / total_requirements
  • Precision: what proportion of retrieved documents in context are useful to answer the question/task. relevant_docs / total_docs
  • F1, nDCG: Derived from recall and precision
• Compare variants using feature_flag in metadata for A/B testing
• Monitor production with configurable sampling and real-time dashboards

How Seer Fits Your Stack

Where Seer Fits In Your App

1. Your app retrieves context from vector DBs, search APIs, or agent tools
2. After retrieval, call client.log() with the query and results
3. Seer evaluates quality asynchronously, with no impact on latency

Quick Example

from seer import SeerClient

client = SeerClient()  # reads SEER_API_KEY from env

# Your retrieval step (vector DB, search API, agent tool, etc.)
def retrieve(query: str) -> list[dict]:
    return [
        {"text": "Christopher Nolan directed Inception.", "score": 0.95},
        {"text": "Nolan is British-American.", "score": 0.89},
    ]

query = "Who directed Inception and what is their nationality?"
context = retrieve(query)

client.log(
    task=query,
    context=context,
    metadata={
        "env": "prod",
        "feature_flag": "retrieval-v1",  # for A/B testing
    },
)
# Events are sent automatically in the background (fire-and-forget mode)

Key Concepts

Task

Context

Metadata

• env: environment (prod, staging, dev)
• feature_flag: A/B test variant

Recall

Precision

Multi-Hop & Agentic Retrieval

• is_final_context: mark which retrieval step provides the final evidence for the answer
• subquery: track decomposed sub-questions for per-hop evaluation
• Trace linking: automatic OTEL trace context to group related retrievals

# Example: Multi-hop retrieval
client.log(
    task="What awards did the director of Inception win?",
    context=hop2_results,
    subquery="What awards did Christopher Nolan win?",  # this hop's goal
    is_final_context=True,  # final evidence for the answer
)

Use Cases

Getting Started

1. Quickstart: Make your first log in 5 minutes
2. Python SDK | TypeScript SDK
3. Metrics: Understand what Seer computes
4. Change Testing: A/B test retrieval changes
5. Production Monitoring: Set up ongoing monitoring