Reports API Reference¶

The mcpbr.reports package provides report generation in three formats: interactive HTML, enhanced Markdown, and print-friendly PDF. All generators accept the standard mcpbr evaluation results dictionary.

from mcpbr.reports import (
    HTMLReportGenerator,
    EnhancedMarkdownGenerator,
    PDFReportGenerator,
)

HTMLReportGenerator¶

Generates standalone, interactive HTML reports with embedded Chart.js charts, responsive layout, dark mode support, and sortable per-task results tables.

Usage¶

from pathlib import Path
from mcpbr.reports import HTMLReportGenerator

generator = HTMLReportGenerator(
    results_data=results,
    title="My MCP Server Evaluation",
    dark_mode=False,
)

# Save to file
generator.save(Path("reports/evaluation.html"))

from mcpbr.reports import HTMLReportGenerator

generator = HTMLReportGenerator(results_data=results)
html_content = generator.generate()

# Use the HTML string (e.g., serve via web framework)
print(len(html_content), "characters")

from mcpbr.reports import HTMLReportGenerator

generator = HTMLReportGenerator(
    results_data=results,
    title="Dark Mode Report",
    dark_mode=True,
)
generator.save(Path("reports/dark-report.html"))

Constructor Parameters¶

Methods¶

Features¶

The generated HTML report includes:

• Summary cards: Resolution rate, baseline comparison, improvement, and cost metrics
• Charts (via Chart.js CDN):
  • Bar chart comparing MCP vs baseline resolution rates
  • Bar chart comparing costs
  • Stacked bar chart for token usage (input vs output, when data available)
  • Doughnut chart for MCP tool usage breakdown (when data available)
• Sortable per-task results table: Click column headers to sort
• Dark mode toggle: Fixed button in the top-right corner
• Responsive layout: Works on desktop, tablet, and mobile
• Self-contained: All CSS and JavaScript inline, only Chart.js loaded via CDN

EnhancedMarkdownGenerator¶

Generates GitHub-flavored Markdown reports with shields.io badges, mermaid charts, collapsible sections, and detailed analysis tables.

Usage¶

from pathlib import Path
from mcpbr.reports import EnhancedMarkdownGenerator

generator = EnhancedMarkdownGenerator(results_data=results)
generator.save(Path("reports/evaluation.md"))

from mcpbr.reports import EnhancedMarkdownGenerator

generator = EnhancedMarkdownGenerator(results_data=results)
markdown = generator.generate()
print(markdown)

Constructor Parameters¶

Methods¶

Features¶

The generated Markdown report includes:

• Shields.io badges: Resolution rate (color-coded), model, benchmark
• Summary table: Key metrics in a clean GFM table
• Mermaid charts:
  • Bar chart for cost comparison (MCP vs baseline)
  • Pie chart for resolution outcome distribution (MCP Only, Baseline Only, Both, Neither)
• Analysis section: MCP-only wins and baseline-only wins with instance IDs
• Collapsible sections (HTML <details>/<summary>):
  • Per-task results table (expandable)
  • Error details (expandable)

Utility Functions¶

The module also exports helper functions for building Markdown components:

from mcpbr.reports.enhanced_markdown import (
    generate_badge,
    generate_mermaid_pie,
    generate_mermaid_bar,
    generate_collapsible,
)

# Create a shields.io badge
badge = generate_badge("status", "passing", "brightgreen")
# ![status: passing](https://img.shields.io/badge/status-passing-brightgreen)

# Create a mermaid pie chart
chart = generate_mermaid_pie("Results", {"Passed": 45, "Failed": 5})

# Create a mermaid bar chart
bar = generate_mermaid_bar("Cost Comparison", {"MCP": 1.23, "Baseline": 0.98})

# Create a collapsible section
section = generate_collapsible("Click to expand", "Hidden content here")

PDFReportGenerator¶

Generates print-friendly HTML reports with CSS @media print styles, page breaks, page counters, and custom branding. Optionally converts to PDF using weasyprint.

Usage¶

from pathlib import Path
from mcpbr.reports import PDFReportGenerator

generator = PDFReportGenerator(
    results_data=results,
    title="Q4 Evaluation Report",
)
generator.save_html(Path("reports/evaluation-print.html"))

from pathlib import Path
from mcpbr.reports import PDFReportGenerator

generator = PDFReportGenerator(
    results_data=results,
    title="Q4 Evaluation Report",
)
generator.save_pdf(Path("reports/evaluation.pdf"))

from pathlib import Path
from mcpbr.reports import PDFReportGenerator

generator = PDFReportGenerator(
    results_data=results,
    title="Acme Corp MCP Evaluation",
    branding={
        "logo_text": "ACME",
        "primary_color": "#e63946",
        "company_name": "Acme Corporation",
    },
)
generator.save_pdf(Path("reports/acme-report.pdf"))

Constructor Parameters¶

Branding Configuration¶

Methods¶

Features¶

The generated report includes:

• Executive summary first: Summary cards with key metrics at the top
• Cost analysis table: MCP vs baseline cost comparison
• Detailed statistics: Token usage, tool usage, and error summary (when available)
• Per-task results table: Full task-by-task breakdown with page break before it
• Print-optimized CSS:
  • @media print rules for proper pagination
  • page-break-inside: avoid on sections
  • Page counters via CSS @page rules
  • Title in the page footer
• Custom branding: Colors, company name, and logo text

pip install weasyprint

Common Patterns¶

Generate All Report Formats¶

from pathlib import Path
from mcpbr.reports import (
    HTMLReportGenerator,
    EnhancedMarkdownGenerator,
    PDFReportGenerator,
)

output_dir = Path("reports")

# Interactive HTML
HTMLReportGenerator(results, title="Evaluation").save(output_dir / "report.html")

# GitHub Markdown
EnhancedMarkdownGenerator(results).save(output_dir / "report.md")

# Print-friendly HTML (always works)
PDFReportGenerator(results, title="Evaluation").save_html(output_dir / "report-print.html")

# PDF (if weasyprint is available)
try:
    PDFReportGenerator(results, title="Evaluation").save_pdf(output_dir / "report.pdf")
except ImportError:
    print("weasyprint not available, skipping PDF")

Results Data Structure¶

All report generators expect a results dictionary with this structure:

results_data = {
    "metadata": {
        "timestamp": "2025-01-15T10:30:00Z",
        "config": {
            "model": "sonnet",
            "provider": "anthropic",
            "benchmark": "swe-bench-verified",
            "agent_harness": "claude-code",
        },
    },
    "summary": {
        "mcp": {
            "resolved": 45,
            "total": 100,
            "rate": 0.45,
            "total_cost": 12.50,
            "cost_per_task": 0.125,
        },
        "baseline": {
            "resolved": 38,
            "total": 100,
            "rate": 0.38,
            "total_cost": 10.20,
        },
        "improvement": "+18.4%",
        "comprehensive_stats": { ... },  # Optional, adds more detail
    },
    "tasks": [
        {
            "instance_id": "django__django-11099",
            "mcp": {
                "resolved": True,
                "cost": 0.15,
                "tokens": {"input": 5000, "output": 1200},
                "iterations": 3,
                "tool_calls": 12,
                "runtime_seconds": 120.5,
            },
            "baseline": {
                "resolved": False,
                "cost": 0.12,
            },
        },
        # ... more tasks
    ],
}