Spaces:

MCP-1st-Birthday
/

TraceMind

Running

App Files Files Community

TraceMind / README.md

Mandark-droid

Fix README short_description to meet HF 60 char limit

98dc4d3 25 days ago

preview code

raw

history blame

6.31 kB

metadata

title: TraceMind AI
emoji: 🔍
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
short_description: AI agent evaluation with MCP-powered intelligence
pinned: false
tags:
  - mcp-in-action-track-enterprise
  - agent-evaluation
  - mcp-client
  - leaderboard
  - gradio

🔍 TraceMind-AI

Agent Evaluation Platform with MCP-Powered Intelligence

Overview

TraceMind-AI is a comprehensive platform for evaluating AI agent performance across different models, providers, and configurations. It provides real-time insights, cost analysis, and detailed trace visualization powered by the Model Context Protocol (MCP).

Features

📊 Real-time Leaderboard: Live evaluation data from HuggingFace datasets
🤖 MCP Integration: AI-powered analysis using remote MCP servers
💰 Cost Estimation: Calculate evaluation costs for different models and configurations
🔍 Trace Visualization: Detailed OpenTelemetry trace analysis
📈 Performance Metrics: GPU utilization, CO2 emissions, token usage tracking

MCP Integration

TraceMind-AI demonstrates enterprise MCP client usage by connecting to TraceMind-mcp-server via the Model Context Protocol.

MCP Tools Used:

analyze_leaderboard - AI-generated insights about evaluation trends
estimate_cost - Cost estimation with hardware recommendations
debug_trace - Interactive trace analysis and debugging
compare_runs - Side-by-side run comparison
analyze_results - Test case analysis with optimization recommendations

Quick Start

Prerequisites

Python 3.10+
HuggingFace account (for authentication)
HuggingFace token (optional, for private datasets)

Installation

Clone the repository:

git clone https://github.com/Mandark-droid/TraceMind-AI.git
cd TraceMind-AI

Install dependencies:

pip install -r requirements.txt

Configure environment:

cp .env.example .env
# Edit .env with your configuration

Run the application:

python app.py

Visit http://localhost:7860

Configuration

Create a .env file with the following variables:

# HuggingFace Configuration
HF_TOKEN=your_token_here

# MCP Server URL
MCP_SERVER_URL=https://kshitijthakkar-tracemind-mcp-server.hf.space/gradio_api/mcp/

# Dataset Configuration
LEADERBOARD_REPO=kshitijthakkar/smoltrace-leaderboard

# Development Mode (optional - disables OAuth for local testing)
DISABLE_OAUTH=true

Data Sources

TraceMind-AI loads evaluation data from HuggingFace datasets:

Leaderboard: Aggregate statistics for all evaluation runs
Results: Individual test case results
Traces: OpenTelemetry trace data
Metrics: GPU metrics and performance data

Architecture

Project Structure

TraceMind-AI/
├── app.py                 # Main Gradio application
├── data_loader.py         # HuggingFace dataset integration
├── mcp_client/            # MCP client implementation
│   ├── client.py          # Async MCP client
│   └── sync_wrapper.py    # Synchronous wrapper
├── utils/                 # Utilities
│   ├── auth.py            # HuggingFace OAuth
│   └── navigation.py      # Screen navigation
├── screens/               # UI screens
├── components/            # Reusable components
└── styles/                # Custom CSS

MCP Client Integration

TraceMind-AI uses the MCP Python SDK to connect to remote MCP servers:

from mcp_client.sync_wrapper import get_sync_mcp_client

# Initialize MCP client
mcp_client = get_sync_mcp_client()
mcp_client.initialize()

# Call MCP tools
insights = mcp_client.analyze_leaderboard(
    metric_focus="overall",
    time_range="last_week",
    top_n=5
)

Usage

Viewing the Leaderboard

Log in with your HuggingFace account
Navigate to the "Leaderboard" tab
Click "Load Leaderboard" to fetch the latest data
View AI-powered insights generated by the MCP server

Estimating Costs

Navigate to the "Cost Estimator" tab
Enter the model name (e.g., openai/gpt-4)
Select agent type and number of tests
Click "Estimate Cost" for AI-powered analysis

Viewing Trace Details

Select an evaluation run from the leaderboard
Click on a specific test case
View detailed OpenTelemetry trace visualization
Ask questions about the trace using MCP-powered analysis

Technology Stack

UI Framework: Gradio 5.49.1
MCP Protocol: MCP integration via Gradio
Data: HuggingFace Datasets API
Authentication: HuggingFace OAuth
AI: Google Gemini 2.5 Flash (via MCP server)

Development

Running Locally

# Install dependencies
pip install -r requirements.txt

# Set development mode (optional - disables OAuth)
export DISABLE_OAUTH=true

# Run the app
python app.py

Running on HuggingFace Spaces

This application is configured for deployment on HuggingFace Spaces using the Gradio SDK. The app.py file serves as the entry point.

Documentation

For detailed implementation documentation, see:

Data Loader API - Dataset loading and caching
MCP Client API - MCP protocol integration
Authentication - HuggingFace OAuth integration

Demo Video

[Link to demo video showing the application in action]

Social Media

[Link to social media post about this project]

License

MIT License - See LICENSE file for details

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

Acknowledgments

MCP Team - For the Model Context Protocol specification
Gradio Team - For Gradio 6 with MCP integration
HuggingFace - For Spaces hosting and dataset infrastructure
Google - For Gemini API access