You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #180 we believe that the MMLU evaluation benchmark dataset only needs question, choices and answer features because that is all the lm-eval task will use:
We should update the generator to remove any other features but these, to make the requirements of the evaluation process more clear and to reduce the size of the dataset.
The text was updated successfully, but these errors were encountered:
markmc
changed the title
Reduce the MMLU evaluation benchmark dataset to the minimum set of columns
Reduce the MMLU evaluation benchmark dataset to the minimum set of features
Jul 23, 2024
This issue has been automatically marked as stale because it has not had activity within 90 days. It will be automatically closed if no further activity occurs within 30 days.
In #180 we believe that the MMLU evaluation benchmark dataset only needs
question
,choices
andanswer
features because that is all the lm-eval task will use:We should update the generator to remove any other features but these, to make the requirements of the evaluation process more clear and to reduce the size of the dataset.
The text was updated successfully, but these errors were encountered: