Skip to content

Commit aaf90f7

Browse files
authored
Update index.md
1 parent 24b7400 commit aaf90f7

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

docs/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,8 @@ In designing test cases for evaluation, we incorporate domain-specific test case
8989
![Image Title](figures/SciCode_chart.png)
9090
<p style="text-align: center;">Left: Distribution of Main Problems Right: Distribution of Subproblems</p>
9191

92+
We include several research problems that are built upon or reproduce methods used in Nobel Prize-winning studies to highlight current trends in scientific research: the self-consistent field (SCF) method for density functional theory (DFT) calculations (**The Nobel Prize in Chemistry 1998**), the PMNS matrix for neutrino oscillation in matter (**The Nobel Prize in Physics 2015**), the Haldane model for the anomalous quantum Hall effect (**The Nobel Prize in Physics 2016**), optical tweezer simulations for microscopic thermodynamics (**The Nobel Prize in Physics 2018**), and the replica method for spin glasses (**The Nobel Prize in Physics 2021**).
93+
9294
## Experiment Results
9395
We evaluate our model using zero-shot prompts. We keep the prompts general and design different ones for different evaluation setups only to inform the model about the tasks. We keep prompts the same across models and fields, and they contain the model’s main and sub-problem instructions and code for previous subproblems. The standard setup means the model is tested without background knowledge and carrying over generated solutions to previous subproblems. The scientists' annotated background provides the necessary knowledge and reasoning steps to solve the problems, shifting the evaluation’s focus more towards the models’ coding and instruction-following capabilities.
9496
![Image Title](figures/Standard_Setup.png)

0 commit comments

Comments
 (0)