Update deployment documentation and implement cloud data loading for customization pipeline

- Revise deployment steps for HuggingFace Spaces
- Add cloud data loading functionality in customization pipeline
- Create generate_mapping_json.py for mapping CSV to JSON
- Implement tests for customization and full integration pipelines

Next_after_pushlargefiles_to_dataset.md

@@ -1,50 +1,333 @@
-## Phase 2: Planning & Analysis
+# Next Steps: Deploy to HuggingFace Spaces - Phase 5
 
-### Step 2.1 - 執行策略規劃
+## 📋 當前狀態確認
 
-**我建議分階段漸進式執行：**
+### ✅ 已完成的階段：
+- **階段 1**: 雲端載入器基礎功能 ✅ 
+- **階段 2**: General Pipeline 雲端載入 ✅
+- **階段 3**: Customization Pipeline 雲端載入 ✅
+- **階段 4**: 完整整合測試 ✅
 
-## **🏗️ 漸進式執行計畫**
+### 📊 雲端載入驗證：
+- **Dataset Repository**: `ybchen928/oncall-guide-ai-models` 
+- **總資料大小**: ~1.6GB (models/ + customization_data/)
+- **下載性能**: 首次 ~2分鐘，後續使用快取
+- **功能完整性**: 兩條 Pipeline 都正常運作
 
-### **階段 1: 建立雲端載入基礎架構**
+---
 
-1. **新增** `src/cloud_loader.py`
-2. **測試**: 雲端載入器獨立功能
-3. **確認**: Dataset 連線正常
+## Phase 5: 部署到 HuggingFace Spaces
 
-### **階段 2: 修改核心系統**
+### 🎯 部署目標
+**將不含大檔案的程式碼推送到 Spaces，實現雲端資料載入**
 
-1. **修改** `src/retrieval.py`
-2. **測試**: 核心檢索系統雲端載入
-3. **確認**: General pipeline 正常運作
+---
 
-### **階段 3: 修改 Customization 系統**
+## Step 5.1: 準備部署檔案
 
-1. **新增** `customization/src/cloud_config.py`
-2. **修改** `customization/customization_pipeline.py`
-3. **測試**: Customization pipeline 雲端載入
-4. **確認**: Hospital-specific 功能正常
+### 檢查當前 Git 狀態
+```bash
+# 確認在正確的 branch
+git branch
+# 應該顯示: * HuggingFace_space_dataset_deployment
 
-### **階段 4: 整合測試**
+# 檢查檔案狀態
+git status
+```
 
-1. **整合測試**: 所有功能一起運作
-2. **環境變數測試**: 本地/雲端模式切換
-3. **完整功能驗證**: General + Hospital + Combined 模式
+### 確認要部署的檔案清單
+**✅ 必須包含的檔案：**
+```
+├── README.md ✅ (Spaces 配置)
+├── app.py ✅ (主應用程式)
+├── requirements.txt ✅ (依賴清單)
+├── .gitattributes ✅ (Git 配置)
+└── src/ ✅ (核心程式碼)
+    ├── user_prompt.py
+    ├── retrieval.py (已修改支援雲端載入)
+    ├── generation.py
+    ├── llm_clients.py
+    ├── medical_conditions.py
+    ├── data_processing.py
+    └── cloud_loader.py (新增)
+```
 
-### **階段 5: 部署到 Spaces**
+**✅ Customization 程式碼：**
+```
+└── customization/ ✅ (只保留程式碼)
+    ├── customization_pipeline.py (已修改支援雲端載入)
+    ├── generate_embeddings.py
+    ├── test/
+    └── src/ (20個 .py 檔案)
+        ├── cloud_config.py (新增)
+        ├── indexing/storage.py
+        ├── indexing/annoy_manager.py
+        └── 其他程式碼檔案
+```
 
-1. **推送程式碼** (不含大檔案)
-2. **監控部署**
-3. **線上功能驗證**
+**❌ 不要包含的檔案：**
+```
+❌ models/ (已移至 Dataset)
+❌ customization/processing/ (已移至 Dataset)  
+❌ evaluation/, tests/, docs/ (開發用)
+❌ dataset/, onCallGuideAIvenv/ (本地環境)
+❌ .env (敏感資訊)
+❌ test_stage*.py (測試腳本)
+```
 
-## **💡 每階段的測試方法：**
+---
 
-- **獨立測試**: 只測試該階段的功能
-- **漸進整合**: 確保不破壞現有功能
-- **快速回退**: 如果有問題可以立即復原
+## Step 5.2: 確認配置檔案
 
-## **🎯 現在開始階段 1？**
+### 檢查 requirements.txt
+```bash
+grep "huggingface-hub" requirements.txt
+# 確保包含: huggingface-hub>=0.33,<0.35
+```
 
-**先執行 `src/cloud_loader.py` 的創建和獨立測試，確認雲端連線正常後再繼續下一階段？**
+### 檢查 README.md YAML frontmatter
+```yaml
+---
+title: OnCall.ai - Medical Emergency Assistant
+emoji: 🏥
+colorFrom: red
+colorTo: blue
+sdk: gradio
+sdk_version: "5.38.0"
+app_file: app.py
+python_version: "3.11"
+pinned: false
+license: mit
+tags:
+  - medical
+  - healthcare
+  - RAG
+  - emergency
+  - clinical-guidance
+  - gradio
+---
+```
 
-**你同意這個漸進式方法嗎？**
+### 檢查環境變數設置
+**在 Spaces Settings 中確認：**
+- `HF_TOKEN`: 你的 HuggingFace API token ✅ (已設置)
+- `USE_CLOUD_DATA`: `true` (自動使用雲端模式)
+
+---
+
+## Step 5.3: Git 提交準備
+
+### 暫時移除本地大檔案資料夾
+```bash
+# 備份本地 processing 資料夾 (如果存在)
+if [ -d "customization/processing" ]; then
+    mv customization/processing customization_processing_backup
+    echo "✅ 已備份 customization/processing"
+fi
+
+# 確認大檔案不在 Git 追蹤中
+git status | grep "models\|processing"
+# 不應該看到任何 models/ 或 processing/ 相關檔案
+```
+
+### 添加修改的程式碼檔案
+```bash
+# 添加所有程式碼修改
+git add src/cloud_loader.py
+git add src/retrieval.py  
+git add customization/src/cloud_config.py
+git add customization/customization_pipeline.py
+git add requirements.txt
+git add README.md
+git add app.py
+git add .gitattributes
+
+# 檢查 staging area
+git status
+```
+
+---
+
+## Step 5.4: 提交並推送到 Spaces
+
+### Git 提交
+```bash
+git commit -m "Implement cloud data loading for HuggingFace Spaces deployment
+
+- Add cloud_loader.py for core system data loading
+- Add customization cloud_config.py for hospital-specific data
+- Modify retrieval.py to use cloud data loading
+- Modify customization_pipeline.py to use preloading
+- Support both local and cloud deployment modes
+- Tested with full integration verification"
+```
+
+### 檢查推送目標
+```bash
+# 確認 remote 設置
+git remote -v
+# 應該看到: hf git@hf.co:spaces/ybchen928/oncall-guide-ai
+
+# 檢查要推送的檔案大小
+du -sh src/ customization/src/ *.py *.txt *.md
+# 確保總大小 < 1GB
+```
+
+### 推送到 Spaces
+```bash
+git push hf HuggingFace_space_dataset_deployment:main --force
+```
+
+---
+
+## Step 5.5: 監控部署過程
+
+### 檢查建置狀態
+1. **前往 Spaces**: https://huggingface.co/spaces/ybchen928/oncall-guide-ai
+2. **點擊 "App" 標籤**查看建置進度
+3. **觀察 Logs**:
+   - 依賴安裝進度
+   - 雲端檔案下載進度
+   - 系統初始化狀態
+
+### 預期建置過程
+```
+Phase 1: Installing dependencies (2-3 minutes)
+├── Installing Python packages from requirements.txt
+├── Setting up Gradio environment
+└── Configuring Python 3.11 environment
+
+Phase 2: Application startup (3-5 minutes)  
+├── Downloading models/ from Dataset (1.5GB)
+├── Downloading customization_data/ from Dataset (150MB)
+├── Initializing retrieval systems
+└── Starting Gradio interface
+
+Phase 3: Ready for use
+├── App status: "Running" 🟢
+├── Interface accessible
+└── All features available
+```
+
+---
+
+## 🚨 故障排除指南
+
+### 常見問題與解決方案
+
+#### 1. 依賴安裝失敗
+**症狀**: 建置卡在 "Installing dependencies"
+**解決**:
+```bash
+# 檢查 requirements.txt 語法
+pip check
+
+# 確認版本相容性
+pip install --dry-run -r requirements.txt
+```
+
+#### 2. 雲端檔案下載失敗
+**症狀**: "404 Not Found" 或 "Access denied"
+**解決**:
+- 確認 Dataset Repository 是 Public
+- 檢查 `HF_TOKEN` 環境變數設置
+- 驗證 Dataset 中的檔案路徑
+
+#### 3. 記憶體不足
+**症狀**: "Out of memory" 或建置超時
+**解決**:
+- 考慮升級 Spaces 硬體 (CPU basic → CPU upgrade)
+- 或實施懶載入 (需要時才載入大檔案)
+
+#### 4. Gradio 介面無法啟動
+**症狀**: 建置成功但無法存取
+**解決**:
+- 檢查 app.py 中的 launch() 配置
+- 確認沒有硬編碼的 port/host 設定
+
+#### 5. Customization 功能失效
+**症狀**: Hospital Only 模式無法運作
+**解決**:
+- 檢查 customization/src/cloud_config.py 的 dataset_repo 名稱
+- 確認所有 processing 檔案都在 Dataset 中
+
+---
+
+## Step 5.6: 功能驗證
+
+### 線上測試清單
+**部署成功後進行的驗證：**
+
+#### 基本功能測試
+- [ ] **General Mode**: 測試基本醫療查詢
+- [ ] **Hospital Mode**: 測試 customization 功能  
+- [ ] **Combined Mode**: 測試混合功能
+
+#### 性能測試
+- [ ] **首次載入時間**: 記錄冷啟動時間
+- [ ] **查詢響應時間**: 測試熱啟動性能
+- [ ] **記憶體使用**: 監控系統資源
+
+#### 錯誤處理測試
+- [ ] **無效查詢**: 測試非醫療查詢的拒絕機制
+- [ ] **網路錯誤**: 模擬檔案下載失敗的處理
+- [ ] **資料不一致**: 測試部分檔案缺失的容錯性
+
+---
+
+## 📊 部署成功指標
+
+### 技術指標
+- **建置時間**: < 10 分鐘
+- **首次載入**: < 5 分鐘  
+- **查詢響應**: < 3 秒
+- **記憶體使用**: < 4GB (免費額度內)
+
+### 功能指標  
+- **General Pipeline**: 正常返回醫療建議
+- **Hospital Pipeline**: 正常返回院所特定建議
+- **Combined Mode**: 整合結果正確
+- **錯誤處理**: 適當的錯誤訊息和回退機制
+
+---
+
+## 🎯 部署後的後續步驟
+
+### 文檔更新
+- [ ] 更新 README.md 說明雲端部署架構
+- [ ] 創建使用指南
+- [ ] 記錄已知限制和注意事項
+
+### 監控和維護
+- [ ] 設置 Spaces 的使用監控
+- [ ] 定期檢查 Dataset Repository 狀態
+- [ ] 監控建置和執行 logs
+
+### 擴展計畫
+- [ ] 考慮實施檔案版本控制
+- [ ] 評估硬體升級需求
+- [ ] 規劃額外功能的雲端化
+
+---
+
+## ⚠️ 重要注意事項
+
+1. **首次使用者體驗**: 第一個使用者需要等待檔案下載 (2-5分鐘)
+2. **快取共享**: 所有使用者共享 HuggingFace 的檔案快取
+3. **成本考量**: 免費 Spaces 有使用時間限制，考慮升級計畫
+4. **資料同步**: Dataset 更新時，Spaces 需要清除快取重新下載
+5. **備份策略**: 保持本地開發環境的完整性，以備緊急回退
+
+---
+
+## 🚀 執行檢查清單
+
+部署前確認：
+- [ ] 階段 4 整合測試通過
+- [ ] Git 狀態乾淨 (無大檔案)
+- [ ] Requirements.txt 包含所有依賴
+- [ ] README.md YAML ��置正確
+- [ ] 環境變數已設置
+- [ ] Dataset Repository 可正常存取
+
+準備就緒後，執行 Phase 5 部署步驟！

ToDo_huggingface_deployment.md

@@ -572,3 +572,22 @@ mv customization_processing_backup customization/processing
 - [ ] 本地測試雲端整合功能
 - [ ] 部署到 Spaces (不含大檔案)
 - [ ] 驗證完整功能正常運作
+## ✅ 階段測試結果更新
+
+### 階段 1 測試 ✅ 成功
+- 雲端載入器連線正常
+- Dataset Repository 存取成功
+
+### 階段 2 測試 ✅ 成功  
+- 核心檢索系統雲端載入正常
+- General Pipeline 完整功能驗證
+- 效能: Emergency (84.5M), Treatment (331M) 檔案下載成功
+
+### 階段 3 測試 ✅ 成功
+- Customization Pipeline 雲端載入正常  
+- 預載入 10 個檔案 (~150MB) 成功
+- Hospital-specific 功能完整驗證
+- 測試結果: "chest pain" (36 results), "emergency treatment" (59 results)
+- 系統載入: 110 tags, 3,784 chunks
+
+**當前狀態: 準備階段 4 整合測試**

customization/customization_pipeline.py

@@ -87,14 +87,17 @@ def retrieve_document_chunks(query: str, top_k: int = 5, llm_client=None) -> Lis
     # Load model and existing embeddings
     embedding_model = load_biomedbert_model()
     
-    # Load from processing folder
-    processing_path = Path(__file__).parent / "processing"
+    # Load processing data from cloud or local with preloading
+    from cloud_config import customization_loader
+    
+    # Preload all processing files and get directory paths
+    embeddings_dir, indices_dir = customization_loader.preload_all_processing_files()
     
     # Load the saved system with ANNOY indices
     document_index, tag_embeddings, doc_tag_mapping, chunk_embeddings, annoy_manager = \
         load_document_system_with_annoy(
-            input_dir=str(processing_path / "embeddings"),
-            annoy_dir=str(processing_path / "indices")
+            input_dir=embeddings_dir,
+            annoy_dir=indices_dir
         )
     
     if annoy_manager is None:

customization/processing/generate_mapping_json.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""
+Generate mapping.json from combined_er_symptoms_diagnoses.csv
+This script creates the mapping file needed for the customization pipeline.
+"""
+
+import csv
+import json
+import os
+from pathlib import Path
+
+def csv_to_mapping_json():
+    """Convert CSV to mapping.json format"""
+    
+    # Define paths
+    processing_dir = Path(__file__).parent
+    customization_dir = processing_dir.parent
+    csv_path = customization_dir / "docs" / "combined_er_symptoms_diagnoses.csv"
+    output_path = processing_dir / "mapping.json"
+    
+    # Read CSV and convert to mapping format
+    mappings = []
+    
+    with open(csv_path, 'r', encoding='utf-8-sig') as csvfile:  # Handle BOM
+        reader = csv.DictReader(csvfile)
+        
+        for row in reader:
+            # Skip empty rows
+            if not row.get('PDF Abbreviation'):
+                continue
+                
+            # Extract symptoms and diagnoses
+            symptoms_raw = row['ER Symptom (Surface)'].strip()
+            diagnoses_raw = row['Underlying Diagnosis (Core)'].strip()
+            
+            # Split symptoms by comma and clean
+            symptoms = [s.strip() for s in symptoms_raw.split(',') if s.strip()]
+            
+            # Split diagnoses by comma and clean
+            diagnoses = [d.strip() for d in diagnoses_raw.split(',') if d.strip()]
+            
+            # Create PDF filename based on abbreviation
+            pdf_name = get_pdf_filename(row['PDF Abbreviation'])
+            
+            # Create mapping entry
+            mapping = {
+                "pdf": pdf_name,
+                "symptoms": symptoms,
+                "diagnoses": diagnoses
+            }
+            
+            mappings.append(mapping)
+    
+    # Write to JSON file
+    with open(output_path, 'w', encoding='utf-8') as jsonfile:
+        json.dump(mappings, jsonfile, indent=2, ensure_ascii=False)
+    
+    print(f"✅ Generated mapping.json with {len(mappings)} entries")
+    print(f"📄 Output saved to: {output_path}")
+    
+    # Verify all PDFs exist
+    docs_dir = customization_dir / "docs"
+    missing_pdfs = []
+    
+    for mapping in mappings:
+        pdf_path = docs_dir / mapping['pdf']
+        if not pdf_path.exists():
+            missing_pdfs.append(mapping['pdf'])
+    
+    if missing_pdfs:
+        print(f"\n⚠️ Warning: {len(missing_pdfs)} PDF files not found:")
+        for pdf in missing_pdfs[:5]:  # Show first 5
+            print(f"   - {pdf}")
+        if len(missing_pdfs) > 5:
+            print(f"   ... and {len(missing_pdfs) - 5} more")
+    else:
+        print("\n✅ All PDF files found in docs directory")
+    
+    return mappings
+
+def get_pdf_filename(abbreviation):
+    """Convert abbreviation to actual PDF filename based on files in docs directory"""
+    
+    # Mapping of abbreviations to actual PDF filenames
+    pdf_mapping = {
+        "SpinalCordEmergencies": "Recognizing Spinal Cord Emergencies.pdf",
+        "DizzinessApproach": "*Dizziness - A Diagnostic Approach.pdf",
+        "CodeHeadache": "*Code Headache - Development of a protocol for optimizing headache management in the emergency room.pdf",
+        "EarlyAFTherapy": "Early Rhythm-Control Therapy in Patients with Atrial Fibrillation.pdf",
+        "2024ESC_AF_Guidelines": "2024 ESC Guidelines for the management of atrial fibrillation developed in collaboration with the European Association for Cardio-Thoracic Surgery.pdf",
+        "PregnancyBleeding_ED": "What assessment, intervention and diagnostics should women with early pregnancy bleeding receive in the emergency department and when A scoping review and synthesis of evidence.pdf",
+        "UGIB_Guideline": "acg_clinical_guideline__upper_gastrointestinal_and.14.pdf",
+        "PulmonaryEmbolism": "Acute Pulmonary Embolism A Review.pdf",
+        "CAP_Review": "Community-Acquired Pneumonia.pdf",
+        "AcuteIschemicStroke_Guideline": "Guidelines for the Early Management of Patients With Acute Ischemic Stroke.pdf",
+        "ChestPain_Guideline_2021": "2021 Guideline for the Evaluation and Diagnosis of Chest Pain.pdf",
+        "FUO_Neutropenia_2024": "2024 update of the AGIHO guideline on diagnosis and empirical treatment of fever of unknown origin (FUO) in adult neutropenic patients with solid tumours and hematological malignancies.pdf",
+        "Eclampsia_ER_Management": "*Management of eclampsia in the accident and emergency department.pdf",
+        "UTI_Mazzulli": "Diagnosis and Management of simple and complicated urinary tract infections (UTIs).pdf",
+        "Pediatric_Seizures_2016": "J Paediatrics Child Health - 2016 - Lawton - Seizures in the paediatric emergency department.pdf",
+        "PregnancyLoss_Review": "A REVIEW OF THE MANAGEMENT OF LOSS OF PREGNANCY IN THE EMERGENCY DEPARTMENT.pdf",
+        "FUO_Children": "Update on Fever of Unknown Origin in Children Focus on Etiologies and Clinical Apporach.pdf",
+        # New entries based on actual files in docs directory
+        "MyastheniaGravis": "[Transition of Japanese clinical guidelines for myasthenia gravis].pdf",
+        "AcutePorphyrias": "AGA Clinical Practice Update on Diagnosis and Management of Acute Hepatic Porphyrias- Expert Review.pdf",
+        "Botulism": "Clinical Guidelines for Diagnosis and Treatment of Botulism, 2021.pdf",
+        "WilsonsDisease": "EASL-ERN Clinical Practice Guidelines on Wilsons disease.pdf",
+        "HereditaryAngioedema": "The international WAO:EAACI guideline for the management of hereditary angioedema-The 2021 revision and update.pdf",
+    }
+    
+    # Return mapped filename or create a generic one based on abbreviation
+    return pdf_mapping.get(abbreviation, f"{abbreviation}.pdf")
+
+if __name__ == "__main__":
+    csv_to_mapping_json()

test_stage3_customization.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+"""階段 3 測試：Customization Pipeline 雲端載入測試"""
+
+import os
+import sys
+from pathlib import Path
+
+# 設置環境變數測試雲端模式
+os.environ['USE_CLOUD_DATA'] = 'true'
+
+# 添加路徑
+current_dir = Path(__file__).parent
+src_dir = current_dir / "src"
+sys.path.insert(0, str(src_dir))
+sys.path.insert(0, str(current_dir))
+
+def test_customization_pipeline():
+    """測試 Customization Pipeline 雲端載入"""
+    print("🧪 階段 3 測試：Customization Pipeline 雲端載入...")
+    
+    try:
+        from customization.customization_pipeline import retrieve_document_chunks
+        print("✅ customization_pipeline 模組載入成功")
+        
+        # 測試 customization pipeline (會觸發雲端下載)
+        print("🏥 測試 customization 查詢...")
+        results = retrieve_document_chunks("chest pain", top_k=3)
+        print(f"✅ Customization search 成功，返回 {len(results)} 個結果")
+        
+        # 測試另一個查詢
+        print("🏥 測試另一個 customization 查詢...")
+        results2 = retrieve_document_chunks("emergency treatment", top_k=5)
+        print(f"✅ 第二個查詢成功，返回 {len(results2)} 個結果")
+        
+        print("🎉 階段 3 測試通過：Customization Pipeline 雲端載入正常！")
+        return True
+        
+    except Exception as e:
+        print(f"❌ 階段 3 測試失敗: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+if __name__ == "__main__":
+    success = test_customization_pipeline()
+    print(f"\n📋 測試結果: {'成功' if success else '失敗'}")
+    exit(0 if success else 1)

test_stage4_full_integration.py

@@ -0,0 +1,186 @@
+#!/usr/bin/env python3
+"""階段 4 測試：完整整合測試 - 模擬 app.py 流程"""
+
+import os
+import sys
+import time
+from pathlib import Path
+from typing import Dict, Any
+
+# 設置環境變數測試雲端模式
+os.environ['USE_CLOUD_DATA'] = 'true'
+
+# 添加路徑
+current_dir = Path(__file__).parent
+src_dir = current_dir / "src"
+sys.path.insert(0, str(src_dir))
+sys.path.insert(0, str(current_dir))
+
+def test_full_integration():
+    """完整整合測試"""
+    print("🧪 階段 4 測試：完整整合測試...")
+    print("=" * 60)
+    
+    results = {}
+    
+    try:
+        # Test 1: Core system initialization
+        print("\n🔧 測試 1: 核心系統初始化")
+        start_time = time.time()
+        
+        from user_prompt import UserPromptProcessor
+        from retrieval import BasicRetrievalSystem
+        
+        processor = UserPromptProcessor()
+        retrieval_system = BasicRetrievalSystem()
+        
+        init_time = time.time() - start_time
+        print(f"✅ 核心系統初始化成功 ({init_time:.2f}秒)")
+        results['core_init_time'] = init_time
+        
+        # Test 2: General Pipeline test
+        print("\n🔍 測試 2: General Pipeline 功能")
+        test_queries = [
+            "chest pain emergency",
+            "heart attack symptoms", 
+            "respiratory distress"
+        ]
+        
+        for query in test_queries:
+            start_time = time.time()
+            search_results = retrieval_system.search(query, top_k=5)
+            search_time = time.time() - start_time
+            
+            result_count = len(search_results.get('processed_results', []))
+            print(f"  📊 '{query}': {result_count} 個結果 ({search_time:.3f}秒)")
+            
+        print("✅ General Pipeline 測試完成")
+        
+        # Test 3: Customization Pipeline test
+        print("\n🏥 測試 3: Customization Pipeline 功能")
+        try:
+            from customization.customization_pipeline import retrieve_document_chunks
+            
+            for query in test_queries:
+                start_time = time.time()
+                custom_results = retrieve_document_chunks(query, top_k=3)
+                custom_time = time.time() - start_time
+                
+                print(f"  🏥 '{query}': {len(custom_results)} 個結果 ({custom_time:.3f}秒)")
+                
+            print("✅ Customization Pipeline 測試完成")
+            results['customization_available'] = True
+            
+        except Exception as e:
+            print(f"❌ Customization Pipeline 錯誤: {e}")
+            results['customization_available'] = False
+        
+        # Test 4: Combined mode simulation (like app.py)
+        print("\n🔄 測試 4: Combined Mode 模擬")
+        test_query = "chest pain emergency treatment"
+        
+        # Step 1: UserPromptProcessor (correct method name)
+        start_time = time.time()
+        extraction_result = processor.extract_condition_keywords(test_query)
+        extraction_time = time.time() - start_time
+        print(f"  📝 Condition extraction: {extraction_time:.3f}秒")
+        print(f"      Condition: {extraction_result.get('condition', 'None')}")
+        print(f"      Emergency keywords: {extraction_result.get('emergency_keywords', 'None')}")
+        print(f"      Treatment keywords: {extraction_result.get('treatment_keywords', 'None')}")
+        
+        # Step 2: General retrieval
+        start_time = time.time()
+        general_results = retrieval_system.search(test_query, top_k=5)
+        general_time = time.time() - start_time
+        general_count = len(general_results.get('processed_results', []))
+        print(f"  🔍 General retrieval: {general_count} 個結果 ({general_time:.3f}秒)")
+        
+        # Step 3: Customization retrieval (if available)
+        if results['customization_available']:
+            start_time = time.time()
+            custom_results = retrieve_document_chunks(test_query, top_k=3)
+            custom_time = time.time() - start_time
+            print(f"  🏥 Hospital retrieval: {len(custom_results)} 個結果 ({custom_time:.3f}秒)")
+        
+        print("✅ Combined Mode 模擬完成")
+        
+        # Test 5: Performance comparison
+        print("\n⚡ 測試 5: 性能測試 (熱啟動)")
+        queries_for_speed = ["emergency", "treatment", "chest pain"]
+        
+        for query in queries_for_speed:
+            # General pipeline speed
+            start_time = time.time()
+            retrieval_system.search(query, top_k=3)
+            general_speed = time.time() - start_time
+            
+            # Customization pipeline speed (if available)
+            if results['customization_available']:
+                start_time = time.time() 
+                retrieve_document_chunks(query, top_k=3)
+                custom_speed = time.time() - start_time
+                print(f"  ⚡ '{query}': General {general_speed:.3f}s, Hospital {custom_speed:.3f}s")
+            else:
+                print(f"  ⚡ '{query}': General {general_speed:.3f}s")
+        
+        print("✅ 性能測試完成")
+        
+        print("\n" + "=" * 60)
+        print("🎉 階段 4 整合測試完全成功！")
+        print("📊 摘要:")
+        print(f"  - 核心系統初始化時間: {results['core_init_time']:.2f}秒") 
+        print(f"  - Customization 功能: {'可用' if results['customization_available'] else '不可用'}")
+        print(f"  - 兩條 Pipeline 都能從雲端載入資料")
+        print(f"  - 系統整合功能完整")
+        
+        return True
+        
+    except Exception as e:
+        print(f"❌ 階段 4 整合測試失敗: {e}")
+        import traceback
+        traceback.print_exc()
+        return False
+
+def test_environment_switching():
+    """測試環境變數切換功能"""
+    print("\n🔄 額外測試: 環境變數切換")
+    
+    try:
+        # Test cloud mode
+        os.environ['USE_CLOUD_DATA'] = 'true'
+        from cloud_loader import CloudDataLoader
+        loader_cloud = CloudDataLoader()
+        print(f"  ☁️ 雲端模式: {loader_cloud.use_cloud}")
+        
+        # Test local mode  
+        os.environ['USE_CLOUD_DATA'] = 'false'
+        loader_local = CloudDataLoader()
+        print(f"  💻 本地模式: {loader_local.use_cloud}")
+        
+        # Reset to cloud mode
+        os.environ['USE_CLOUD_DATA'] = 'true'
+        
+        print("✅ 環境變數切換測試成功")
+        return True
+        
+    except Exception as e:
+        print(f"❌ 環境變數切換測試失敗: {e}")
+        return False
+
+if __name__ == "__main__":
+    print("🚀 開始階段 4: 完整整合測試")
+    
+    # Main integration test
+    integration_success = test_full_integration()
+    
+    # Environment switching test
+    env_success = test_environment_switching()
+    
+    overall_success = integration_success and env_success
+    
+    print(f"\n📋 階段 4 總結果: {'完全成功' if overall_success else '部分失敗'}")
+    
+    if overall_success:
+        print("🎯 準備進入階段 5: 部署到 Spaces")
+    
+    exit(0 if overall_success else 1)