Reference Implementation: Data Copilot Answer Synthesis¶

What it is¶

Data Copilot Answer Synthesis (v3.2, July 2026) is a Pydantic-based schema and prompt contract for the final stage of an agentic data pipeline. It ensures that every response generated by models like Gemma 3 or Claude 5.1 includes underlying reasoning, specific source citations, confidence scores, and recommended next steps, now fully supporting MCP 3.0 structured outputs.

What problem it solves¶

It prevents "lazy" agent responses (e.g., just returning a raw JSON array) by forcing the model to provide context, cite its sources, and suggest practical actions. It addresses the "black box" nature of AI reasoning by making assumptions and confidence scores explicit and machine-readable.

Where it fits in the stack¶

It is the final Inference Stage of the Data Copilot pipeline, occurring after data retrieval and tool execution, and before the response is delivered to the user interface. It works in conjunction with Data Copilot Text-to-SQL Architecture to provide grounded answers.

Typical use cases¶

Executive Summaries: Providing high-level briefings of financial performance with direct links to SQL transaction logs.
Root-Cause Reports: Explaining why a metric changed, citing both SQL results and RAG context from Langfuse Tracing.
Automated Alerts: Sending structured notifications that include troubleshooting steps from a manual as a recommended action.
Audit Trails: Maintaining a structured record of AI-provided information and the specific queries used to generate it for compliance.

Strengths¶

Transparency: Every claim is linked to a specific source (SQL row or document snippet).
Actionability: Encourages the model to provide useful next steps rather than passive info.
Machine-Parseable: Standardized JSON allows for easy integration into dashboards.
Consistency: Ensures all Data Copilot instances across different domains return information in a predictable format.

Limitations¶

Token Usage: Structured JSON outputs require significantly more tokens than plain text.
Model Intelligence: Requires high-reasoning models (Claude 5.1, GPT-5) to maintain perfect schema adherence under complex constraints.
Schema Rigidity: May require frequent updates as new data modalities (video/audio) are integrated into the retrieval pipeline.

When to use it¶

When building user-facing data assistants where trust and clarity are paramount.
For multi-modal applications where the UI needs to parse specific fields like key_metrics for visualization.
When integrating with Data Copilot MCP Tooling for automated decision-making.

When not to use it¶

For internal debugging logs where raw, unformatted data is preferred.
In latency-critical systems where the overhead of an additional synthesis LLM call is prohibitive.
For simple, non-data-driven conversational tasks.

Getting started¶

Initialize the Pydantic model with your LLM's JSON output to ensure type safety and validation.

import json
from pydantic import BaseModel, Field, ValidationError
from typing import List, Dict, Any, Optional

class DataPoint(BaseModel):
    label: str
    value: Any
    unit: Optional[str] = None

class Source(BaseModel):
    type: str = Field(..., description="SQL, Doc, or API")
    id: str = Field(..., description="Unique identifier")
    description: str

class SynthesisResponse(BaseModel):
    answer_summary: str = Field(..., description="1-2 sentence direct answer.")
    key_metrics: List[DataPoint] = Field(..., description="Numerical findings.")
    explanation: str = Field(..., description="The 'Why' behind the data.")
    sources: List[Source] = Field(..., description="Traceability links.")
    confidence_score: float = Field(..., ge=0.0, le=1.0)
    assumptions: List[str] = Field(default_factory=list, description="Logical leaps the agent made.")
    recommended_actions: List[str] = Field(default_factory=list)
    needs_human_review: bool = Field(default=False)

# Example Usage
raw_json = '{"answer_summary": "Spend was £142.50", "key_metrics": [], "explanation": "Log analysis", "sources": [], "confidence_score": 1.0}'
response = SynthesisResponse(**json.loads(raw_json))

CLI examples¶

# Validate a synthesis JSON file against the schema
python -m data_copilot.validate_synthesis --file response.json

# Generate a mock synthesis response for testing UI components
python -m data_copilot.mock_synthesis --scenario "high_confidence_spend"

# Test the synthesis prompt against a specific LLM provider
python -m data_copilot.test_prompt --model "google/gemma3-27b-it" --prompt_type synthesis

API examples¶

from data_copilot.synthesis import AnswerSynthesizer
from data_copilot.models import SynthesisResponse

# Initialize synthesizer with MCP 3.0 support
synthesizer = AnswerSynthesizer(model="claude-5-1", use_mcp=True)

# Synthesize answer from raw SQL results and RAG context
results = {"sql_data": [...], "rag_context": "..."}
response: SynthesisResponse = synthesizer.generate(query="What is my spend?", data=results)

print(f"Confidence: {response.confidence_score}")
if response.needs_human_review:
    print("Warning: Low confidence synthesis.")

Sources / references¶

Contribution Metadata¶

Last reviewed: 2026-07-21
Confidence: high