Skip to content
This repository has been archived by the owner on Oct 27, 2023. It is now read-only.

Implement ELO rating function #4

Open
paulbricman opened this issue May 30, 2023 · 1 comment
Open

Implement ELO rating function #4

paulbricman opened this issue May 30, 2023 · 1 comment
Assignees

Comments

@paulbricman
Copy link
Owner

We should have a function which receives as arguments:

  • a list of model names

After every game, the winning player takes points from the losing one. (https://en.wikipedia.org/wiki/Elo_rating_system)

  • a "number of games" parameter (needs looking into: are we randomly pitting "players" against each other? Are we rather going through all possible games?
    And returns a dictionary whose keys are model names and values are ELO ratings.

This part on the wiki page also seems relevant for implementation:

An example may help to clarify: Suppose player A has a rating of 1613...

@paulbricman
Copy link
Owner Author

Suggestion: Test using multiple small models: distilgpt2, gpt2, gpt2-medium, for example. Actually, it should be possibly to simply send in a list of e.g. three identical model names, too, right?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants