Re-check multilevel fallback validation test suite for OnCall.ai

feat(user_prompt): Enhance Medical Query Processing Pipeline

feat(user_prompt): update UserPromptProcessor to integrate Llama3-Med42-70B and enhance query validation; add unit tests for condition extraction and matching mechanisms

feat(retrieval): add sliding window search method for enhanced semantic search

feat(user_prompt): add keyword index checking and enhance medical query validation (add validate_medical_query() and _check_keyword_in_index(), based on implementation_todo_20250730_user_prompt.md

feat(llm_clients): enhance MeditronClient to support local model loading and improve error handling

feat(.gitignore): add cache directory to ignored files

WIP(llm_clients): add MeditronClient for medical query processing with Hugging Face integration

🚀 Implement Advanced Condition Extraction for Medical Query Processing

refactor(deduplication): change deduplication logic from distance-based to exact text matching

feat(retrieval) 1st MVP: enhance search logging and deduplication logic with distance threshold

WIP: feat(retrieval): implement basic vector retrieval system for medical documents

refactor(data_processing): enhance chunking and embedding generation

WIP: Enhance dual keyword chunking to include pre-calculated metadata for treatment chunks

Add comprehensive tests for chunk quality analysis and embedding validation

Add requirements.txt for test dependencies

chore(.gitignore): update ignored files for environment and editor settings

Add new optimized subsets comparison reports in CSV and JSONL formats

chore(.gitignore): add json and png file extensions to ignored files

chore: remove obsolete analysis files and plots for emergency and treatment datasets

refactor(.gitignore): reorganize and expand ignored files for clarity and completeness

Remove obsolete embedding and index files; add comprehensive embedding test analysis and validation suite

feat(data_processing): Implement token length control with semantic preservation

refactor(data_processing): add token-based chunking strategy for improved keyword context

refactor(data_processing): optimize chunking strategy with token-based approach

fix: add .vscode/ to .gitignore

fix: update .gitignore to include docs/ and *.pyc

feat(data-processing): implement data processing pipeline with embeddings

Merge pull request #1 from YanBoChen0928/dataprocessing

Update dataset/scripts/data_explorer_treatment.py

Update dataset/scripts/data_explorer_treatment_opt.py

feat: add emergency subset analysis plots and statistics report

refactor: migrate special terms to JSON configuration

feat: implement special_term (added) emergency keyword matching and metadata extraction

feat: update treatment analysis with keyword density calculations and enhanced visualization(test previous 2 dataset, especially treatment_subset)

feat: add integrity check report for dataset analysis (previous commit test result)

WIP: during pre-process dataset, when doing dataset_treatment exploration, some abnormality happen, thus we now create certain test script to identify the problem

feat: add comprehensive treatment analysis statistics and update treatment keywords

WIP: add dual keyword and text length distribution plots for treatment subset analysis

WIP: Try to analysis treatment_subset enhance emergency and treatment filtering scripts with metadata and analysis functionality

feat: update emergency subset analysis scripts and visualizations, enhance statistics, and fix keyword files

WIP: add emergency subset analysis scripts and visualizations

refactor: remove outdated emergency subset preprocessing documentation

feat(dataset): Implement emergency subset extraction with enhanced matching

fix: correct dataset path in .gitignore

feat: update requirements.txt to include additional dependencies

feat: add requirements.txt with initial package dependencies

docs: update .gitignore to include virtual environment and docs directory

docs: add .gitignore and init