README.md

metadata

title: LLM Council
emoji: 🏢
colorFrom: pink
colorTo: green
sdk: gradio
sdk_version: 6.0.0
app_file: app.py
pinned: false

🏢 LLM Council - Multi-Model AI Deliberation System

A sophisticated system where multiple LLMs collaboratively answer questions through a 3-stage deliberation process.

🎯 How It Works

1. Stage 1 - Individual Responses: 5 different AI models independently answer your question
2. Stage 2 - Peer Review: Each model ranks the anonymized responses from others
3. Stage 3 - Synthesis: A chairman model synthesizes the final answer based on all inputs

💰 Cost: Mostly FREE!

This version uses FREE HuggingFace Inference API models:

• ✅ Meta Llama 3.3 70B (FREE)
• ✅ Qwen 2.5 72B (FREE)
• ✅ Mixtral 8x7B (FREE)
• 💵 OpenAI GPT-4o-mini (low cost)
• 💵 OpenAI GPT-3.5-turbo (low cost)

Cost per query: ~$0.01-0.03 (mostly OpenAI, HF is free!)

⚡ Quick Start

🚀 Deploy to Hugging Face (Recommended)

1. Fork/Duplicate this Space
2. Add your API keys in Settings → Repository secrets:
   • OPENAI_API_KEY - Get from OpenAI
   • HUGGINGFACE_API_KEY - Get from HuggingFace
3. Done! Your space will auto-deploy

💻 Run Locally

# Clone repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
cd YOUR_SPACE_NAME

# Install dependencies
pip install -r requirements.txt

# Create .env file with your API keys
cp .env.example .env
# Edit .env and add:
#   OPENAI_API_KEY=your_openai_key
#   HUGGINGFACE_API_KEY=your_hf_token

# Run the app
python app.py

Visit http://localhost:7860

🔑 Getting API Keys

OpenAI API Key (Required)

1. Go to https://platform.openai.com/api-keys
2. Create new secret key
3. Copy and save it (costs ~$0.01-0.03 per query)

HuggingFace Token (Required for FREE models)

1. Go to https://huggingface.co/settings/tokens
2. Create new token (read access is enough)
3. Copy and save it (100% FREE to use!)

🤖 Council Models

Current Configuration

• Meta Llama 3.3 70B - Excellent reasoning, FREE
• Qwen 2.5 72B - Strong performance, FREE
• Mixtral 8x7B - Mixture of experts, FREE
• OpenAI GPT-4o-mini - Fast & capable, low cost
• OpenAI GPT-3.5-turbo - Reliable, low cost

Chairman

• OpenAI GPT-4o-mini - Excellent synthesis capabilities

Want to customize? Edit backend/config_free.py!

📊 Performance

• Response Time: 60-120 seconds (3 stages, parallel processing)
• Quality: Better than single-model responses
• Cost: ~$0.01-0.03 per query (mostly FREE!)
• Reliability: Automatic retries & error handling

🛠️ Tech Stack

• Frontend: Gradio 6.0+ (with MCP server support)
• Backend: Python async/await
• APIs:
  • HuggingFace Inference API (FREE models)
  • OpenAI API (paid models)
• Storage: JSON-based conversation persistence

📁 Project Structure

llm_council/
├── app.py                     # Main Gradio interface
├── requirements.txt           # Python dependencies
├── .env.example              # Environment template
├── backend/
│   ├── config_free.py        # FREE model configuration
│   ├── api_client.py         # HF + OpenAI API client
│   ├── council_free.py       # 3-stage orchestration
│   ├── storage.py            # Conversation storage
│   └── main.py               # FastAPI backend (optional)
└── docs/
    ├── QUICKSTART.md
    ├── DEPLOYMENT_GUIDE.md
    └── CODE_ANALYSIS.md

🔧 Configuration

Want different models? Edit backend/config_free.py:

# Use ALL FREE models (no OpenAI cost):
COUNCIL_MODELS = [
    {"id": "meta-llama/Llama-3.3-70B-Instruct", "provider": "huggingface"},
    {"id": "Qwen/Qwen2.5-72B-Instruct", "provider": "huggingface"},
    {"id": "mistralai/Mixtral-8x7B-Instruct-v0.1", "provider": "huggingface"},
    {"id": "google/gemma-2-27b-it", "provider": "huggingface"},
]

🤝 Contributing

Improvements welcome! See CODE_ANALYSIS.md for refactoring suggestions.

📝 Credits

• Implementation: Community contributions
• FREE models: Meta, Qwen, Mistral via HuggingFace

📄 License

See original repository for license information.

---

Need Help? Check the docs folder for detailed guides!