Krishna Chaitanya Cheedella
commited on
Commit
ยท
537891a
1
Parent(s):
4b93eb4
Remove all original source attribution and URLs
Browse files- DEPLOYMENT_GUIDE.md +4 -5
- DEPLOYMENT_SUCCESS.md +0 -183
- FIXES_REQUIRED.md +0 -99
- IMPROVEMENTS_SUMMARY.md +0 -216
- QUICKSTART.md +1 -2
- README.md +2 -3
- app.py +1 -2
- backend/openrouter_improved.py +0 -2
DEPLOYMENT_GUIDE.md
CHANGED
|
@@ -121,7 +121,7 @@ CHAIRMAN_MODEL = "deepseek/deepseek-reasoner"
|
|
| 121 |
#### Method 1: Using Existing Space (Fork)
|
| 122 |
|
| 123 |
1. **Fork the Space**
|
| 124 |
-
- Visit
|
| 125 |
- Click "โฎ" โ "Duplicate this Space"
|
| 126 |
- Choose a name for your space
|
| 127 |
|
|
@@ -220,9 +220,9 @@ pydantic>=2.0.0 # Optional - for REST API
|
|
| 220 |
## ๐ป Running Locally
|
| 221 |
|
| 222 |
```bash
|
| 223 |
-
# 1. Clone repository
|
| 224 |
-
git clone https://huggingface.co/spaces/
|
| 225 |
-
cd
|
| 226 |
|
| 227 |
# 2. Create virtual environment
|
| 228 |
python -m venv venv
|
|
@@ -324,7 +324,6 @@ The app will be available at `http://localhost:7860`
|
|
| 324 |
|
| 325 |
## ๐ Additional Resources
|
| 326 |
|
| 327 |
-
- [Original LLM Council by Machine Theory](https://github.com/machine-theory/lm-council)
|
| 328 |
- [OpenRouter Documentation](https://openrouter.ai/docs)
|
| 329 |
- [Gradio Documentation](https://gradio.app/docs)
|
| 330 |
- [Hugging Face Spaces Guide](https://huggingface.co/docs/hub/spaces)
|
|
|
|
| 121 |
#### Method 1: Using Existing Space (Fork)
|
| 122 |
|
| 123 |
1. **Fork the Space**
|
| 124 |
+
- Visit your existing HuggingFace Space
|
| 125 |
- Click "โฎ" โ "Duplicate this Space"
|
| 126 |
- Choose a name for your space
|
| 127 |
|
|
|
|
| 220 |
## ๐ป Running Locally
|
| 221 |
|
| 222 |
```bash
|
| 223 |
+
# 1. Clone repository (use your own space URL)
|
| 224 |
+
git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
|
| 225 |
+
cd YOUR_SPACE_NAME
|
| 226 |
|
| 227 |
# 2. Create virtual environment
|
| 228 |
python -m venv venv
|
|
|
|
| 324 |
|
| 325 |
## ๐ Additional Resources
|
| 326 |
|
|
|
|
| 327 |
- [OpenRouter Documentation](https://openrouter.ai/docs)
|
| 328 |
- [Gradio Documentation](https://gradio.app/docs)
|
| 329 |
- [Hugging Face Spaces Guide](https://huggingface.co/docs/hub/spaces)
|
DEPLOYMENT_SUCCESS.md
DELETED
|
@@ -1,183 +0,0 @@
|
|
| 1 |
-
# ๐ SUCCESS! Your LLM Council is Deployed!
|
| 2 |
-
|
| 3 |
-
## โ
What Was Done
|
| 4 |
-
|
| 5 |
-
### 1. **Completely Refactored** to Use FREE Models
|
| 6 |
-
- โ Removed dependency on OpenRouter (which you didn't have)
|
| 7 |
-
- โ
Added **FREE HuggingFace Inference API** support
|
| 8 |
-
- โ
Added **OpenAI API** support (using your key)
|
| 9 |
-
|
| 10 |
-
### 2. **Council Members** (Mix of FREE + Low Cost)
|
| 11 |
-
- **Meta Llama 3.3 70B** - FREE via HuggingFace
|
| 12 |
-
- **Qwen 2.5 72B** - FREE via HuggingFace
|
| 13 |
-
- **Mixtral 8x7B** - FREE via HuggingFace
|
| 14 |
-
- **OpenAI GPT-4o-mini** - Low cost (~$0.01/query)
|
| 15 |
-
- **OpenAI GPT-3.5-turbo** - Low cost (~$0.01/query)
|
| 16 |
-
|
| 17 |
-
### 3. **Cost**: ~$0.01-0.03 per query
|
| 18 |
-
- HuggingFace models: **100% FREE!**
|
| 19 |
-
- OpenAI models: Very cheap for synthesis
|
| 20 |
-
|
| 21 |
-
### 4. **Pushed to Your HuggingFace Space**
|
| 22 |
-
โ
Code deployed to: https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council
|
| 23 |
-
|
| 24 |
-
## ๐ FINAL STEP: Add Secrets to HuggingFace
|
| 25 |
-
|
| 26 |
-
Your space is deployed but **needs API keys** to work. Here's how:
|
| 27 |
-
|
| 28 |
-
### Step 1: Go to Your Space Settings
|
| 29 |
-
Visit: https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council/settings
|
| 30 |
-
|
| 31 |
-
### Step 2: Add Repository Secrets
|
| 32 |
-
Click on "Repository secrets" section and add these two secrets:
|
| 33 |
-
|
| 34 |
-
#### Secret 1: OPENAI_API_KEY
|
| 35 |
-
```
|
| 36 |
-
Name: OPENAI_API_KEY
|
| 37 |
-
Value: <your OpenAI API key from https://platform.openai.com/api-keys>
|
| 38 |
-
```
|
| 39 |
-
|
| 40 |
-
#### Secret 2: HUGGINGFACE_API_KEY
|
| 41 |
-
```
|
| 42 |
-
Name: HUGGINGFACE_API_KEY
|
| 43 |
-
Value: <your HuggingFace token from https://huggingface.co/settings/tokens>
|
| 44 |
-
```
|
| 45 |
-
|
| 46 |
-
### Step 3: Restart the Space
|
| 47 |
-
After adding secrets:
|
| 48 |
-
1. Click "Factory reboot" or just wait a moment
|
| 49 |
-
2. The space will rebuild automatically
|
| 50 |
-
3. Your app will be live!
|
| 51 |
-
|
| 52 |
-
## ๐ Using Your LLM Council
|
| 53 |
-
|
| 54 |
-
### Web Interface
|
| 55 |
-
Once secrets are added, visit:
|
| 56 |
-
https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council
|
| 57 |
-
|
| 58 |
-
### How to Use
|
| 59 |
-
1. Type your question
|
| 60 |
-
2. Click "Submit"
|
| 61 |
-
3. Wait ~1-2 minutes for the 3-stage process:
|
| 62 |
-
- Stage 1: 5 models answer independently
|
| 63 |
-
- Stage 2: Models rank each other
|
| 64 |
-
- Stage 3: Chairman synthesizes final answer
|
| 65 |
-
|
| 66 |
-
### Example Questions
|
| 67 |
-
- "What is the best programming language to learn in 2025?"
|
| 68 |
-
- "Explain quantum computing in simple terms"
|
| 69 |
-
- "Compare React vs Vue.js for web development"
|
| 70 |
-
|
| 71 |
-
## ๐ New Files Created
|
| 72 |
-
|
| 73 |
-
### Core Functionality
|
| 74 |
-
- `backend/config_free.py` - FREE model configuration
|
| 75 |
-
- `backend/api_client.py` - HuggingFace + OpenAI API client
|
| 76 |
-
- `backend/council_free.py` - 3-stage council logic
|
| 77 |
-
|
| 78 |
-
### Documentation
|
| 79 |
-
- `README.md` - Updated with FREE model info
|
| 80 |
-
- `DEPLOYMENT_SUCCESS.md` - This file!
|
| 81 |
-
- `.gitignore` - Protects secrets
|
| 82 |
-
|
| 83 |
-
### Configuration
|
| 84 |
-
- `.env.example` - Template for local development
|
| 85 |
-
- `requirements.txt` - Updated with openai package
|
| 86 |
-
|
| 87 |
-
## ๐ก Cost Breakdown
|
| 88 |
-
|
| 89 |
-
### Per Query
|
| 90 |
-
- HuggingFace models (3): **$0.00** (FREE!)
|
| 91 |
-
- OpenAI GPT-4o-mini: ~$0.01
|
| 92 |
-
- OpenAI GPT-3.5-turbo: ~$0.01
|
| 93 |
-
- **Total**: ~$0.01-0.03 per query
|
| 94 |
-
|
| 95 |
-
### Monthly Estimates
|
| 96 |
-
- Light use (10 queries/day): ~$3-10/month
|
| 97 |
-
- Medium use (50 queries/day): ~$15-50/month
|
| 98 |
-
- Heavy use (200 queries/day): ~$60-200/month
|
| 99 |
-
|
| 100 |
-
## ๐ง Customization Options
|
| 101 |
-
|
| 102 |
-
### Use ALL FREE Models
|
| 103 |
-
Edit `backend/config_free.py` and uncomment the FREE config:
|
| 104 |
-
|
| 105 |
-
```python
|
| 106 |
-
COUNCIL_MODELS = [
|
| 107 |
-
{"id": "meta-llama/Llama-3.3-70B-Instruct", "provider": "huggingface"},
|
| 108 |
-
{"id": "Qwen/Qwen2.5-72B-Instruct", "provider": "huggingface"},
|
| 109 |
-
{"id": "mistralai/Mixtral-8x7B-Instruct-v0.1", "provider": "huggingface"},
|
| 110 |
-
{"id": "google/gemma-2-27b-it", "provider": "huggingface"},
|
| 111 |
-
{"id": "microsoft/Phi-3.5-mini-instruct", "provider": "huggingface"},
|
| 112 |
-
]
|
| 113 |
-
CHAIRMAN_MODEL = {"id": "meta-llama/Llama-3.3-70B-Instruct", "provider": "huggingface"}
|
| 114 |
-
```
|
| 115 |
-
|
| 116 |
-
This would be **100% FREE** (no OpenAI costs)!
|
| 117 |
-
|
| 118 |
-
### Add More OpenAI Models
|
| 119 |
-
If you want higher quality:
|
| 120 |
-
|
| 121 |
-
```python
|
| 122 |
-
COUNCIL_MODELS = [
|
| 123 |
-
{"id": "openai/gpt-4o", "provider": "openai", "model": "gpt-4o"}, # Premium
|
| 124 |
-
{"id": "openai/gpt-4o-mini", "provider": "openai", "model": "gpt-4o-mini"},
|
| 125 |
-
# ... keep HF models too
|
| 126 |
-
]
|
| 127 |
-
```
|
| 128 |
-
|
| 129 |
-
## ๐ Troubleshooting
|
| 130 |
-
|
| 131 |
-
### "Model is loading" Error
|
| 132 |
-
- HuggingFace models may need to warm up (20-30 seconds)
|
| 133 |
-
- Code automatically waits and retries
|
| 134 |
-
- Normal on first use
|
| 135 |
-
|
| 136 |
-
### OpenAI Errors
|
| 137 |
-
- Check API key is correct in secrets
|
| 138 |
-
- Verify you have OpenAI credits
|
| 139 |
-
- Check usage at https://platform.openai.com/usage
|
| 140 |
-
|
| 141 |
-
### HuggingFace Errors
|
| 142 |
-
- Make sure token has "read" permission
|
| 143 |
-
- Some models may be rate-limited
|
| 144 |
-
- Try using different models
|
| 145 |
-
|
| 146 |
-
## ๐ What's Different from Original
|
| 147 |
-
|
| 148 |
-
| Aspect | Original | Your Version |
|
| 149 |
-
|--------|----------|--------------|
|
| 150 |
-
| **API** | OpenRouter | HuggingFace + OpenAI |
|
| 151 |
-
| **Cost** | $0.05-0.15/query | $0.01-0.03/query |
|
| 152 |
-
| **Free Models** | None | 3 out of 5 |
|
| 153 |
-
| **Setup** | Need OpenRouter account | Use existing keys |
|
| 154 |
-
| **Flexibility** | Fixed models | Can use 100% free |
|
| 155 |
-
|
| 156 |
-
## ๐ฏ Next Steps
|
| 157 |
-
|
| 158 |
-
### Immediate
|
| 159 |
-
1. โ
Add secrets to HuggingFace Space (see above)
|
| 160 |
-
2. โ
Test with a simple question
|
| 161 |
-
3. โ
Monitor costs in OpenAI dashboard
|
| 162 |
-
|
| 163 |
-
### Optional
|
| 164 |
-
1. Customize model selection in `backend/config_free.py`
|
| 165 |
-
2. Add more FREE HuggingFace models
|
| 166 |
-
3. Share your space with others!
|
| 167 |
-
|
| 168 |
-
## ๐ Resources
|
| 169 |
-
|
| 170 |
-
- **Your Space**: https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council
|
| 171 |
-
- **HuggingFace Inference API**: https://huggingface.co/docs/api-inference/
|
| 172 |
-
- **OpenAI Pricing**: https://openai.com/api/pricing/
|
| 173 |
-
- **Original Project**: https://github.com/machine-theory/lm-council
|
| 174 |
-
|
| 175 |
-
## ๐ You're All Set!
|
| 176 |
-
|
| 177 |
-
Your LLM Council is deployed and ready to use FREE HuggingFace models + your OpenAI key!
|
| 178 |
-
|
| 179 |
-
**Just add those two secrets and you're live!** ๐
|
| 180 |
-
|
| 181 |
-
---
|
| 182 |
-
|
| 183 |
-
Questions? Check the other documentation files or the HuggingFace Space logs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
FIXES_REQUIRED.md
DELETED
|
@@ -1,99 +0,0 @@
|
|
| 1 |
-
# ๐ง Issues Fixed + Action Required
|
| 2 |
-
|
| 3 |
-
## โ
Fixed Issue #1: HuggingFace API Endpoint
|
| 4 |
-
|
| 5 |
-
**Problem**: HuggingFace deprecated `api-inference.huggingface.co`
|
| 6 |
-
**Error**: `HTTP 410 - endpoint is no longer supported`
|
| 7 |
-
|
| 8 |
-
**Solution Applied**: Updated to use new endpoint `router.huggingface.co/v1/chat/completions`
|
| 9 |
-
|
| 10 |
-
โ
**This is now fixed and deployed!**
|
| 11 |
-
|
| 12 |
-
## โ ๏ธ Issue #2: Missing API Secrets (YOU MUST FIX THIS)
|
| 13 |
-
|
| 14 |
-
**Problem**: OpenAI returns 401 errors - secrets not configured
|
| 15 |
-
**Error**: `HTTP error querying OpenAI gpt-4o-mini: 401`
|
| 16 |
-
|
| 17 |
-
**Why**: The API keys I used in .env are NOT in your HuggingFace Space secrets yet!
|
| 18 |
-
|
| 19 |
-
### ๐ YOU MUST ADD THESE SECRETS NOW:
|
| 20 |
-
|
| 21 |
-
1. **Go to your space settings**:
|
| 22 |
-
https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council/settings
|
| 23 |
-
|
| 24 |
-
2. **Click "Repository secrets"**
|
| 25 |
-
|
| 26 |
-
3. **Add these TWO secrets** (click "+ Add a secret"):
|
| 27 |
-
|
| 28 |
-
**Secret #1:**
|
| 29 |
-
- Name: `OPENAI_API_KEY`
|
| 30 |
-
- Value: `<your OpenAI API key>`
|
| 31 |
-
|
| 32 |
-
**Secret #2:**
|
| 33 |
-
- Name: `HUGGINGFACE_API_KEY`
|
| 34 |
-
- Value: `<your HuggingFace token>`
|
| 35 |
-
|
| 36 |
-
4. **Save and restart the space**
|
| 37 |
-
|
| 38 |
-
After adding secrets, the 401 errors will disappear!
|
| 39 |
-
|
| 40 |
-
## ๐ Expected Behavior After Fix
|
| 41 |
-
|
| 42 |
-
Once you add the secrets, you should see:
|
| 43 |
-
|
| 44 |
-
```
|
| 45 |
-
STAGE 1: Collecting individual responses from council members...
|
| 46 |
-
๐ Querying 5 models in parallel...
|
| 47 |
-
โ
Model meta-llama/Llama-3.3-70B-Instruct completed
|
| 48 |
-
โ
Model Qwen/Qwen2.5-72B-Instruct completed
|
| 49 |
-
โ
Model mistralai/Mixtral-8x7B-Instruct-v0.1 completed
|
| 50 |
-
โ
Model openai/gpt-4o-mini completed
|
| 51 |
-
โ
Model openai/gpt-3.5-turbo completed
|
| 52 |
-
๐ 5/5 models responded successfully
|
| 53 |
-
STAGE 1 COMPLETE: Received 5 responses.
|
| 54 |
-
```
|
| 55 |
-
|
| 56 |
-
## ๐งช I Cannot Test on HuggingFace Because...
|
| 57 |
-
|
| 58 |
-
I don't have access to:
|
| 59 |
-
1. Your HuggingFace Space admin panel to add secrets
|
| 60 |
-
2. The running Space to see logs
|
| 61 |
-
3. The ability to modify Space settings
|
| 62 |
-
|
| 63 |
-
**Only YOU can add the secrets** through the Space settings UI.
|
| 64 |
-
|
| 65 |
-
## ๐ Summary of Changes Pushed
|
| 66 |
-
|
| 67 |
-
### What I Fixed:
|
| 68 |
-
1. โ
Updated HuggingFace API endpoint from deprecated to new router
|
| 69 |
-
2. โ
Changed API format to OpenAI-compatible chat completions
|
| 70 |
-
3. โ
Removed hardcoded secrets from documentation
|
| 71 |
-
4. โ
Pushed all fixes to your Space
|
| 72 |
-
|
| 73 |
-
### What YOU Need to Do:
|
| 74 |
-
1. โ ๏ธ Add `OPENAI_API_KEY` secret to Space
|
| 75 |
-
2. โ ๏ธ Add `HUGGINGFACE_API_KEY` secret to Space
|
| 76 |
-
3. โ
Test the Space after secrets are added
|
| 77 |
-
|
| 78 |
-
## ๐ฏ Quick Test After Adding Secrets
|
| 79 |
-
|
| 80 |
-
1. Go to your space: https://huggingface.co/spaces/zade-frontier/andrej-karpathy-llm-council
|
| 81 |
-
2. Wait for it to restart (after adding secrets)
|
| 82 |
-
3. Ask a simple question like: "What is 2+2?"
|
| 83 |
-
4. You should see all 5 models respond successfully
|
| 84 |
-
5. Check the logs - should show โ
instead of โ
|
| 85 |
-
|
| 86 |
-
## ๐ก Why This Happened
|
| 87 |
-
|
| 88 |
-
1. **HuggingFace changed their API** - not our fault, they deprecated the old endpoint
|
| 89 |
-
2. **Secrets aren't committed** - by design! They must be added through Space settings for security
|
| 90 |
-
3. **Local .env doesn't sync** - Environment variables are local only, not pushed to git
|
| 91 |
-
|
| 92 |
-
## ๐ After You Add Secrets...
|
| 93 |
-
|
| 94 |
-
The Space will work perfectly with:
|
| 95 |
-
- 3 FREE HuggingFace models (Llama, Qwen, Mixtral)
|
| 96 |
-
- 2 Low-cost OpenAI models (GPT-4o-mini, GPT-3.5-turbo)
|
| 97 |
-
- Total cost: ~$0.01-0.03 per query
|
| 98 |
-
|
| 99 |
-
**Please add those secrets now and let me know the results!** ๐
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
IMPROVEMENTS_SUMMARY.md
DELETED
|
@@ -1,216 +0,0 @@
|
|
| 1 |
-
# ๐ SUMMARY - LLM Council Code Review & Improvements
|
| 2 |
-
|
| 3 |
-
## โ
What Was Done
|
| 4 |
-
|
| 5 |
-
### 1. **Complete Code Analysis** โ
|
| 6 |
-
- Analyzed the 3-stage council architecture
|
| 7 |
-
- Identified strengths and weaknesses
|
| 8 |
-
- Reviewed all backend modules
|
| 9 |
-
|
| 10 |
-
### 2. **Created Missing Files** โ
|
| 11 |
-
- `requirements.txt` - All Python dependencies
|
| 12 |
-
- `.env.example` - Environment variable template
|
| 13 |
-
- `DEPLOYMENT_GUIDE.md` - Comprehensive deployment instructions
|
| 14 |
-
- `CODE_ANALYSIS.md` - Detailed code review
|
| 15 |
-
- `QUICKSTART.md` - Fast setup guide
|
| 16 |
-
|
| 17 |
-
### 3. **Improved Code Files** โ
|
| 18 |
-
- `backend/config_improved.py` - Better model selection
|
| 19 |
-
- `backend/openrouter_improved.py` - Enhanced error handling & retries
|
| 20 |
-
|
| 21 |
-
## ๐ฏ Key Improvements
|
| 22 |
-
|
| 23 |
-
### Model Recommendations
|
| 24 |
-
|
| 25 |
-
#### Current (Original) โ
|
| 26 |
-
```python
|
| 27 |
-
# Using experimental/unstable endpoints
|
| 28 |
-
"openai/gpt-oss-120b:hyperbolic"
|
| 29 |
-
"deepseek-ai/DeepSeek-V3.2-Exp:novita"
|
| 30 |
-
"Qwen/Qwen3-235B-A22B-Instruct-2507:hyperbolic"
|
| 31 |
-
```
|
| 32 |
-
|
| 33 |
-
#### Recommended (Improved) โ
|
| 34 |
-
```python
|
| 35 |
-
# Stable, latest models from trusted providers
|
| 36 |
-
COUNCIL_MODELS = [
|
| 37 |
-
"deepseek/deepseek-chat", # DeepSeek V3 - excellent reasoning
|
| 38 |
-
"anthropic/claude-3.7-sonnet", # Claude 3.7 - strong analysis
|
| 39 |
-
"openai/gpt-4o", # GPT-4o - reliable & versatile
|
| 40 |
-
"google/gemini-2.0-flash-thinking-exp:free", # Fast thinking
|
| 41 |
-
"qwen/qwq-32b-preview", # Strong reasoning
|
| 42 |
-
]
|
| 43 |
-
|
| 44 |
-
CHAIRMAN_MODEL = "deepseek/deepseek-reasoner" # DeepSeek R1
|
| 45 |
-
```
|
| 46 |
-
|
| 47 |
-
**Why These Models?**
|
| 48 |
-
- โ
Latest stable versions
|
| 49 |
-
- โ
Diverse providers (OpenAI, Anthropic, Google, DeepSeek, Qwen)
|
| 50 |
-
- โ
Proven performance
|
| 51 |
-
- โ
Good cost/quality balance
|
| 52 |
-
- โ
Readily available on OpenRouter
|
| 53 |
-
|
| 54 |
-
### Code Enhancements
|
| 55 |
-
|
| 56 |
-
#### Error Handling & Reliability
|
| 57 |
-
```python
|
| 58 |
-
# โ
Retry logic with exponential backoff
|
| 59 |
-
# โ
Timeout configuration
|
| 60 |
-
# โ
Proper error categorization (4xx vs 5xx)
|
| 61 |
-
# โ
Graceful degradation
|
| 62 |
-
# โ
Detailed logging
|
| 63 |
-
```
|
| 64 |
-
|
| 65 |
-
#### Configuration Options
|
| 66 |
-
```python
|
| 67 |
-
# โ
Budget Council (fast & cheap)
|
| 68 |
-
# โ
Balanced Council (recommended)
|
| 69 |
-
# โ
Premium Council (maximum quality)
|
| 70 |
-
# โ
Reasoning Council (complex problems)
|
| 71 |
-
```
|
| 72 |
-
|
| 73 |
-
## ๐ Files Created
|
| 74 |
-
|
| 75 |
-
```
|
| 76 |
-
llm_council/
|
| 77 |
-
โโโ requirements.txt โจ NEW - Dependencies
|
| 78 |
-
โโโ .env.example โจ NEW - Environment template
|
| 79 |
-
โโโ QUICKSTART.md โจ NEW - Fast setup guide
|
| 80 |
-
โโโ DEPLOYMENT_GUIDE.md โจ NEW - Full documentation
|
| 81 |
-
โโโ CODE_ANALYSIS.md โจ NEW - Code review
|
| 82 |
-
โโโ backend/
|
| 83 |
-
โโโ config_improved.py โจ NEW - Better model config
|
| 84 |
-
โโโ openrouter_improved.py โจ NEW - Enhanced API client
|
| 85 |
-
```
|
| 86 |
-
|
| 87 |
-
## ๐ How to Use
|
| 88 |
-
|
| 89 |
-
### Option 1: Keep Original + Test Improvements
|
| 90 |
-
|
| 91 |
-
The improved files are separate (`*_improved.py`) so you can:
|
| 92 |
-
1. Test new versions alongside originals
|
| 93 |
-
2. Compare performance
|
| 94 |
-
3. Roll back if needed
|
| 95 |
-
|
| 96 |
-
```bash
|
| 97 |
-
# When ready to use improved versions:
|
| 98 |
-
mv backend/config_improved.py backend/config.py
|
| 99 |
-
mv backend/openrouter_improved.py backend/openrouter.py
|
| 100 |
-
```
|
| 101 |
-
|
| 102 |
-
### Option 2: Deploy to Hugging Face Now
|
| 103 |
-
|
| 104 |
-
1. **Fork existing space** at https://huggingface.co/spaces/burtenshaw/karpathy-llm-council
|
| 105 |
-
2. **Add your API key** in Settings โ Repository secrets โ `OPENROUTER_API_KEY`
|
| 106 |
-
3. **Optional**: Update to improved models by editing `backend/config.py`
|
| 107 |
-
|
| 108 |
-
See `DEPLOYMENT_GUIDE.md` for step-by-step instructions.
|
| 109 |
-
|
| 110 |
-
## ๐ฐ Cost Comparison
|
| 111 |
-
|
| 112 |
-
| Configuration | Cost/Query | Speed | Quality |
|
| 113 |
-
|--------------|------------|-------|---------|
|
| 114 |
-
| **Budget Council** | $0.01-0.03 | Fast (30-60s) | Good |
|
| 115 |
-
| **Balanced Council** | $0.05-0.15 | Medium (45-90s) | Very Good |
|
| 116 |
-
| **Premium Council** | $0.20-0.50 | Slow (60-135s) | Excellent |
|
| 117 |
-
|
| 118 |
-
## ๐ Architecture Understanding
|
| 119 |
-
|
| 120 |
-
### 3-Stage Process
|
| 121 |
-
|
| 122 |
-
```
|
| 123 |
-
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 124 |
-
โ USER QUESTION โ
|
| 125 |
-
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 126 |
-
โ
|
| 127 |
-
โผ
|
| 128 |
-
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 129 |
-
โ STAGE 1: Individual Responses (Parallel) โ
|
| 130 |
-
โ โข DeepSeek answers โ
|
| 131 |
-
โ โข Claude answers โ
|
| 132 |
-
โ โข GPT-4o answers โ
|
| 133 |
-
โ โข Gemini answers โ
|
| 134 |
-
โ โข QwQ answers โ
|
| 135 |
-
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 136 |
-
โ
|
| 137 |
-
โผ
|
| 138 |
-
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 139 |
-
โ STAGE 2: Peer Rankings (Anonymous) โ
|
| 140 |
-
โ โข Each model ranks "Response A, B, C..." โ
|
| 141 |
-
โ โข Aggregate rankings calculated โ
|
| 142 |
-
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 143 |
-
โ
|
| 144 |
-
โผ
|
| 145 |
-
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 146 |
-
โ STAGE 3: Chairman Synthesis โ
|
| 147 |
-
โ โข DeepSeek Reasoner reviews all โ
|
| 148 |
-
โ โข Considers responses + rankings โ
|
| 149 |
-
โ โข Generates final comprehensive answer โ
|
| 150 |
-
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 151 |
-
```
|
| 152 |
-
|
| 153 |
-
### Why This Works
|
| 154 |
-
|
| 155 |
-
1. **Stage 1 Diversity**: Different models have different strengths
|
| 156 |
-
2. **Stage 2 Validation**: Anonymous ranking reduces bias
|
| 157 |
-
3. **Stage 3 Synthesis**: Chairman combines best insights
|
| 158 |
-
|
| 159 |
-
## ๐ฏ Next Steps
|
| 160 |
-
|
| 161 |
-
### Immediate
|
| 162 |
-
1. โ
Review `QUICKSTART.md` for setup
|
| 163 |
-
2. โ
Test locally with your API key
|
| 164 |
-
3. โ
Deploy to HuggingFace Spaces
|
| 165 |
-
|
| 166 |
-
### Short-term
|
| 167 |
-
1. Compare original vs improved models
|
| 168 |
-
2. Monitor costs and performance
|
| 169 |
-
3. Adjust configuration to your needs
|
| 170 |
-
|
| 171 |
-
### Long-term
|
| 172 |
-
1. Add caching for repeated questions
|
| 173 |
-
2. Implement conversation history
|
| 174 |
-
3. Add custom model selection UI
|
| 175 |
-
4. Track quality metrics
|
| 176 |
-
|
| 177 |
-
## ๐ Documentation Map
|
| 178 |
-
|
| 179 |
-
- **`QUICKSTART.md`** โ Fast 5-minute setup
|
| 180 |
-
- **`DEPLOYMENT_GUIDE.md`** โ Complete deployment guide
|
| 181 |
-
- **`CODE_ANALYSIS.md`** โ Detailed code review
|
| 182 |
-
- **`README.md`** โ Original project info
|
| 183 |
-
|
| 184 |
-
## โจ Key Takeaways
|
| 185 |
-
|
| 186 |
-
### What's Good (Original)
|
| 187 |
-
- โ
Clean architecture
|
| 188 |
-
- โ
Smart 3-stage design
|
| 189 |
-
- โ
Async parallel processing
|
| 190 |
-
- โ
Good Gradio integration
|
| 191 |
-
|
| 192 |
-
### What Was Missing
|
| 193 |
-
- โ Error handling & retries
|
| 194 |
-
- โ Stable model selection
|
| 195 |
-
- โ Configuration flexibility
|
| 196 |
-
- โ Deployment documentation
|
| 197 |
-
|
| 198 |
-
### What's Fixed (Improved)
|
| 199 |
-
- โ
Robust error handling
|
| 200 |
-
- โ
Latest stable models
|
| 201 |
-
- โ
Multiple config presets
|
| 202 |
-
- โ
Comprehensive docs
|
| 203 |
-
|
| 204 |
-
## ๐ You're Ready!
|
| 205 |
-
|
| 206 |
-
Everything you need is now in your workspace:
|
| 207 |
-
|
| 208 |
-
```bash
|
| 209 |
-
z:\projects\llm_council\
|
| 210 |
-
```
|
| 211 |
-
|
| 212 |
-
**Start here**: Open `QUICKSTART.md` for immediate setup instructions.
|
| 213 |
-
|
| 214 |
-
**Questions?** Check `DEPLOYMENT_GUIDE.md` for comprehensive information.
|
| 215 |
-
|
| 216 |
-
Good luck with your LLM Council! ๐
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
QUICKSTART.md
CHANGED
|
@@ -39,7 +39,7 @@ Visit `http://localhost:7860` ๐
|
|
| 39 |
## ๐ Deploy to Hugging Face Spaces (FREE)
|
| 40 |
|
| 41 |
### Option A: Fork Existing Space
|
| 42 |
-
1. Visit
|
| 43 |
2. Click "โฎ" โ "Duplicate this Space"
|
| 44 |
3. Settings โ Repository secrets โ Add `OPENROUTER_API_KEY`
|
| 45 |
4. Done! Your space will auto-deploy
|
|
@@ -125,7 +125,6 @@ Typical costs:
|
|
| 125 |
|
| 126 |
- **Complete Guide**: See `DEPLOYMENT_GUIDE.md`
|
| 127 |
- **Code Analysis**: See `CODE_ANALYSIS.md`
|
| 128 |
-
- **Original Project**: https://github.com/machine-theory/lm-council
|
| 129 |
|
| 130 |
## ๐ก Tips
|
| 131 |
|
|
|
|
| 39 |
## ๐ Deploy to Hugging Face Spaces (FREE)
|
| 40 |
|
| 41 |
### Option A: Fork Existing Space
|
| 42 |
+
1. Visit your HuggingFace Space
|
| 43 |
2. Click "โฎ" โ "Duplicate this Space"
|
| 44 |
3. Settings โ Repository secrets โ Add `OPENROUTER_API_KEY`
|
| 45 |
4. Done! Your space will auto-deploy
|
|
|
|
| 125 |
|
| 126 |
- **Complete Guide**: See `DEPLOYMENT_GUIDE.md`
|
| 127 |
- **Code Analysis**: See `CODE_ANALYSIS.md`
|
|
|
|
| 128 |
|
| 129 |
## ๐ก Tips
|
| 130 |
|
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: ๐ข
|
| 4 |
colorFrom: pink
|
| 5 |
colorTo: green
|
|
@@ -11,7 +11,7 @@ pinned: false
|
|
| 11 |
|
| 12 |
# ๐ข LLM Council - Multi-Model AI Deliberation System
|
| 13 |
|
| 14 |
-
A sophisticated system where multiple LLMs collaboratively answer questions through a 3-stage deliberation process
|
| 15 |
|
| 16 |
## ๐ฏ How It Works
|
| 17 |
|
|
@@ -143,7 +143,6 @@ Improvements welcome! See `CODE_ANALYSIS.md` for refactoring suggestions.
|
|
| 143 |
|
| 144 |
## ๐ Credits
|
| 145 |
|
| 146 |
-
- Original concept: [Machine Theory](https://github.com/machine-theory/lm-council) & [Andrej Karpathy](https://github.com/karpathy)
|
| 147 |
- Implementation: Community contributions
|
| 148 |
- FREE models: Meta, Qwen, Mistral via HuggingFace
|
| 149 |
|
|
|
|
| 1 |
---
|
| 2 |
+
title: LLM Council
|
| 3 |
emoji: ๐ข
|
| 4 |
colorFrom: pink
|
| 5 |
colorTo: green
|
|
|
|
| 11 |
|
| 12 |
# ๐ข LLM Council - Multi-Model AI Deliberation System
|
| 13 |
|
| 14 |
+
A sophisticated system where multiple LLMs collaboratively answer questions through a 3-stage deliberation process.
|
| 15 |
|
| 16 |
## ๐ฏ How It Works
|
| 17 |
|
|
|
|
| 143 |
|
| 144 |
## ๐ Credits
|
| 145 |
|
|
|
|
| 146 |
- Implementation: Community contributions
|
| 147 |
- FREE models: Meta, Qwen, Mistral via HuggingFace
|
| 148 |
|
app.py
CHANGED
|
@@ -91,8 +91,7 @@ async def ask_council(question: str, progress=gr.Progress()):
|
|
| 91 |
|
| 92 |
|
| 93 |
description = """
|
| 94 |
-
An LLM Council that consults multiple AI models to answer questions
|
| 95 |
-
and Andrej Karpathy.
|
| 96 |
|
| 97 |
๐ฏ **Council Members**: Mix of FREE HuggingFace models + OpenAI models
|
| 98 |
- Meta Llama 3.3 70B
|
|
|
|
| 91 |
|
| 92 |
|
| 93 |
description = """
|
| 94 |
+
An LLM Council that consults multiple AI models to answer questions through a 3-stage deliberation process.
|
|
|
|
| 95 |
|
| 96 |
๐ฏ **Council Members**: Mix of FREE HuggingFace models + OpenAI models
|
| 97 |
- Meta Llama 3.3 70B
|
backend/openrouter_improved.py
CHANGED
|
@@ -33,7 +33,6 @@ async def query_model(
|
|
| 33 |
headers = {
|
| 34 |
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
|
| 35 |
"Content-Type": "application/json",
|
| 36 |
-
"HTTP-Referer": "https://huggingface.co/spaces/burtenshaw/karpathy-llm-council",
|
| 37 |
"X-Title": "LLM Council",
|
| 38 |
}
|
| 39 |
|
|
@@ -104,7 +103,6 @@ async def query_model_stream(
|
|
| 104 |
headers = {
|
| 105 |
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
|
| 106 |
"Content-Type": "application/json",
|
| 107 |
-
"HTTP-Referer": "https://huggingface.co/spaces/burtenshaw/karpathy-llm-council",
|
| 108 |
"X-Title": "LLM Council",
|
| 109 |
}
|
| 110 |
|
|
|
|
| 33 |
headers = {
|
| 34 |
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
|
| 35 |
"Content-Type": "application/json",
|
|
|
|
| 36 |
"X-Title": "LLM Council",
|
| 37 |
}
|
| 38 |
|
|
|
|
| 103 |
headers = {
|
| 104 |
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
|
| 105 |
"Content-Type": "application/json",
|
|
|
|
| 106 |
"X-Title": "LLM Council",
|
| 107 |
}
|
| 108 |
|