Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
update documentation
Browse files- app.py +3 -3
- content.py +56 -25
app.py
CHANGED
|
@@ -404,7 +404,7 @@ def add_new_eval(
|
|
| 404 |
except Exception as e:
|
| 405 |
print(f"Error during submission: {e}")
|
| 406 |
yield (
|
| 407 |
-
f"An error occurred, please open a discussion
|
| 408 |
)
|
| 409 |
|
| 410 |
|
|
@@ -534,7 +534,7 @@ def create_overview_dataframe(results_dfs: Dict[str, pd.DataFrame]) -> pd.DataFr
|
|
| 534 |
value = model_row.iloc[0][source_col]
|
| 535 |
rank = model_row.index[0]
|
| 536 |
else:
|
| 537 |
-
value =
|
| 538 |
rank = df.shape[0]
|
| 539 |
overview_data[display_col].append(value)
|
| 540 |
model_rankings[model_entry].append(rank)
|
|
@@ -559,7 +559,7 @@ def create_submission_interface() -> Tuple[gr.components.Component, ...]:
|
|
| 559 |
"""
|
| 560 |
Create the submission interface components.
|
| 561 |
"""
|
| 562 |
-
with gr.Accordion("Submit predictions"):
|
| 563 |
with gr.Row():
|
| 564 |
gr.Markdown(SUBMISSION_TEXT, elem_classes="markdown-text")
|
| 565 |
with gr.Row():
|
|
|
|
| 404 |
except Exception as e:
|
| 405 |
print(f"Error during submission: {e}")
|
| 406 |
yield (
|
| 407 |
+
f"An error occurred, please open a discussion/issue if you continue to have submission issues.\n{e}"
|
| 408 |
)
|
| 409 |
|
| 410 |
|
|
|
|
| 534 |
value = model_row.iloc[0][source_col]
|
| 535 |
rank = model_row.index[0]
|
| 536 |
else:
|
| 537 |
+
value = np.nan
|
| 538 |
rank = df.shape[0]
|
| 539 |
overview_data[display_col].append(value)
|
| 540 |
model_rankings[model_entry].append(rank)
|
|
|
|
| 559 |
"""
|
| 560 |
Create the submission interface components.
|
| 561 |
"""
|
| 562 |
+
with gr.Accordion("Submit predictions for evaluation"):
|
| 563 |
with gr.Row():
|
| 564 |
gr.Markdown(SUBMISSION_TEXT, elem_classes="markdown-text")
|
| 565 |
with gr.Row():
|
content.py
CHANGED
|
@@ -3,47 +3,78 @@ TITLE = """<h1 align="center" id="space-title">FAIR Chemistry Leaderboard</h1>""
|
|
| 3 |
|
| 4 |
# Main introduction text
|
| 5 |
INTRODUCTION_TEXT = """
|
| 6 |
-
|
| 7 |
|
| 8 |
-
This space
|
| 9 |
-
Leaderboards previously hosted on EvalAI ([OC20](https://eval.ai/web/challenges/challenge-page/712/overview)) will also be migrated here in the future.
|
| 10 |
|
|
|
|
| 11 |
|
| 12 |
-
|
| 13 |
-
This leaderboard showcases the performance of various machine learning interatomic potentials (MLIP) on the Open Molecules 2025 (OMol25) dataset.
|
| 14 |
-
OMol25 represents a diverse, high-quality dataset uniquely blending elemental, chemical, and structural diversity.
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
-
- **S2EF (Structure to Energy and Forces)**: Test and Validation splits across different molecular categories
|
| 20 |
-
- **Specialized Evaluations**: Practically relevant chemistry tasks to evaluate models beyond just S2EF metrics (i.e. ligand-strain, spin gap, etc.)
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
"""
|
| 24 |
|
| 25 |
# Submission instructions
|
| 26 |
SUBMISSION_TEXT = """
|
| 27 |
## How to Submit
|
| 28 |
|
| 29 |
-
To submit your model predictions:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
-
|
| 32 |
-
2. **Fill in the model information** including name, organization, and contact details
|
| 33 |
-
3. **Select the evaluation type** that matches your prediction file
|
| 34 |
-
4. **Upload your file** and click Submit
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
- Ensure your prediction file format matches the expected format for the selected evaluation
|
| 40 |
-
- Your email will be stored privately and only used for communication regarding your submission
|
| 41 |
-
- Results will appear on the leaderboard after successful validation
|
| 42 |
-
- Remain on the page until you see the "Success" message. Evaluations can take several minutes, please be patient
|
| 43 |
-
- If you wish to have your model removed from the leaderboard please reach out to mshuaibi@meta.com with the model name and submission date
|
| 44 |
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
| 47 |
"""
|
| 48 |
|
| 49 |
# Citation information
|
|
|
|
| 3 |
|
| 4 |
# Main introduction text
|
| 5 |
INTRODUCTION_TEXT = """
|
| 6 |
+
# Welcome to the FAIR Chemistry Leaderboard! π§ͺ
|
| 7 |
|
| 8 |
+
This space hosts comprehensive leaderboards across different chemical domains including molecules, catalysts, and materials.
|
|
|
|
| 9 |
|
| 10 |
+
*Note: Leaderboards previously hosted on EvalAI (such as [OC20](https://eval.ai/web/challenges/challenge-page/712/overview)) will be migrated here in the future.*
|
| 11 |
|
| 12 |
+
## 𧬠OMol25
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
This leaderboard evaluates performance on the **Open Molecules 2025 (OMol25)** datasetβa diverse, high-quality collection that uniquely combines elemental, chemical, and structural diversity.
|
| 15 |
|
| 16 |
+
π **Learn more:** [OMol25 Paper](https://arxiv.org/pdf/2505.08762)
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
### Benchmarks
|
| 19 |
+
|
| 20 |
+
**S2EF (Structure to Energy and Forces)**
|
| 21 |
+
- Test and validation sets across different molecular categories
|
| 22 |
+
|
| 23 |
+
**Evaluations.** Downstream chemistry tasks that evaluate practical applications:
|
| 24 |
+
- **Ligand Pocket:** Protein-ligand interaction energy as a proxy for binding energy
|
| 25 |
+
- **Ligand Strain:** Ligand-strain energy crucial for understanding protein-ligand binding
|
| 26 |
+
- **Conformers:** Identifying the lowest energy conformer
|
| 27 |
+
- **Protonation:** Energy differences between protonated structures (proxy for pKa prediction)
|
| 28 |
+
- **IE/EA:** Ionization energy and electron affinity
|
| 29 |
+
- **Distance Scaling:** Short and long range intermolecular interactions
|
| 30 |
+
- **Spin Gap:** Energy differences between varying spin states
|
| 31 |
+
|
| 32 |
+
## π Getting Started
|
| 33 |
+
|
| 34 |
+
Ready to submit your model? Check out our steps for running benchmarks and generating prediction files:
|
| 35 |
+
|
| 36 |
+
π **[Submission Documentation](https://fair-chem.github.io/molecules/leaderboard.html)**
|
| 37 |
"""
|
| 38 |
|
| 39 |
# Submission instructions
|
| 40 |
SUBMISSION_TEXT = """
|
| 41 |
## How to Submit
|
| 42 |
|
| 43 |
+
### To submit your model predictions:
|
| 44 |
+
<div style="background-color: #f8f9fa; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
| 45 |
+
|
| 46 |
+
**π Step 1:** Generate prediction files for the appropriate task (see [here](https://fair-chem.github.io/molecules/leaderboard.html) for details)
|
| 47 |
+
|
| 48 |
+
**π Step 2:** Sign in with Hugging Face
|
| 49 |
+
|
| 50 |
+
**π Step 3:** Fill in the submission metadata (name, organization, contact info, etc.)
|
| 51 |
+
|
| 52 |
+
**π― Step 4:** Select the evaluation type that matches your prediction file
|
| 53 |
+
|
| 54 |
+
**π€ Step 5:** Upload your file and click Submit Eval
|
| 55 |
+
|
| 56 |
+
**β±οΈ Step 6:** Wait for the evaluation to complete and see the "β
" message in the Status bar
|
| 57 |
+
|
| 58 |
+
</div>
|
| 59 |
+
|
| 60 |
+
<div style="background-color: #e8f4fd; padding: 15px; border-left: 4px solid #2196F3; margin: 20px 0;">
|
| 61 |
|
| 62 |
+
**π Submission Limits:** Users are limited to **5 successful submissions per month** for each evaluation type.
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
+
</div>
|
| 65 |
|
| 66 |
+
### β οΈ Important Notes:
|
| 67 |
+
- β
**File Format:** Ensure your prediction file format matches the expected format for the selected evaluation (.npz for S2EF and .json for Evaluations)
|
| 68 |
+
- π **Privacy:** Your email will be stored privately and only used for communication regarding your submission
|
| 69 |
+
- π **Results:** Results will appear on the leaderboard after successful validation
|
| 70 |
+
- β±οΈ **Wait Time:** Remain on the page until you see the "Success" message. Evaluations can take several minutes, please be patient
|
| 71 |
+
- ποΈ **Removal:** If you wish to have your model removed from the leaderboard, please reach out to mshuaibi@meta.com with the model name and submission date
|
| 72 |
|
| 73 |
+
### π¬ Need Help?
|
| 74 |
+
This leaderboard is actively being developed and we welcome any feedback and contributions!
|
| 75 |
+
**π Contact us:**
|
| 76 |
+
- π [GitHub Issues](https://github.com/facebookresearch/fairchem)
|
| 77 |
+
- π¬ [Discussion Forum](https://huggingface.co/spaces/facebook/fairchem_leaderboard/discussions)
|
| 78 |
"""
|
| 79 |
|
| 80 |
# Citation information
|