Spaces:
Paused
Paused
File size: 2,095 Bytes
a52f96d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# Performance Notes: Test Slowness
## Why Tests Are Slow
The `test_student.py` tests can be slow for several reasons:
### 1. **DistilBERT Model Loading** (Main Cause)
- Loading DistilBERT from HuggingFace is **expensive** (downloads models, loads weights)
- Each test creates a new `StudentAgent()` which loads the model
- This can take **10-30+ seconds** per test on slower systems
- **This is normal** - not your laptop's fault!
### 2. **Model Inference**
- Each `student.answer()` call runs neural network inference
- Each `student.learn()` call does forward + backward pass
- On CPU, this is slower than GPU
### 3. **Multiple Evaluations**
- Tests evaluate on multiple tasks multiple times
- Each evaluation runs model inference
## Solutions Implemented
✅ **Added tqdm progress bars** - Shows progress during slow operations
✅ **Reduced iteration counts** - Fewer training loops for faster tests
✅ **Smaller eval sets** - Fewer tasks to evaluate on
✅ **Graceful fallback** - Works even if model loading fails
## Speedup Options
### Option 1: Skip Model Loading (Fastest)
```bash
# Tests will use dummy mode (much faster)
python test_student.py
```
### Option 2: Use GPU (if available)
```python
student = StudentAgent(device='cuda') # Much faster if you have GPU
```
### Option 3: Cache Model Loading
- Model is downloaded/cached automatically by transformers
- First run is slowest (downloads model)
- Subsequent runs are faster (uses cache)
### Option 4: Use Smaller Model
- DistilBERT is already small (67M parameters)
- Could use even smaller model for testing, but DistilBERT is a good balance
## Expected Times
- **Model loading**: 10-30 seconds (first time), 5-10 seconds (cached)
- **Per test**: 5-15 seconds (with model)
- **Total test suite**: 30-90 seconds (with model)
- **Without model (dummy)**: < 5 seconds total
## It's Not Your Laptop!
This is normal for:
- Neural network model loading
- Transformer models (they're large)
- CPU inference (GPU would be faster but requires CUDA)
The progress bars help you see what's happening even if it's slow!
|