Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: MBT for votekeeper #63

Merged
merged 210 commits into from
Nov 28, 2023
Merged

tests: MBT for votekeeper #63

merged 210 commits into from
Nov 28, 2023

Conversation

hvanz
Copy link
Member

@hvanz hvanz commented Nov 14, 2023

Closes #44
Depends on #18 and #40.
Also depends on #73.

This PR adds a CI workflow .github/workflows/mbt.yml that generates trace files from voteBookkeeperTest.qnt and then executes each trace in the code as a unit test. In particular:

  • In the spec:
    • Update the spec with a new variable for the weighted vote that is passed as input to the applyVote function in each step.
    • Add a new Quint module voteBookkeeperSM for the parameterized state machine for the vote keeper.
  • In the code:
    • Implement VotekeeperRunner that builds the initial state and the steps from an ITF trace.

@romac
Copy link
Member

romac commented Nov 28, 2023

@rnbguy Any chance we can make the MBT tests count towards code coverage? Apparently we can just upload multiple coverage reports and Codecov will merge them, which is pretty neat :)

@romac
Copy link
Member

romac commented Nov 28, 2023

Or is that the case already? I see that the coverage job now sets the QUINT_SEED env variable, so presumably it does generate the traces and runs the MBT tests?

If so, then I am not sure I understand why we need the MBT workflow?

@rnbguy
Copy link
Member

rnbguy commented Nov 28, 2023

You're right. we don't need the MBT workflow anymore - as MBT tests are already executed with Rust workflow and again in coverage workflow.

@rnbguy
Copy link
Member

rnbguy commented Nov 28, 2023

Since Rust and coverage workflows are picking up the default seed for Quint, I set MBT workflow to use a UNIX epoch as random seed.

Copy link
Member

@rnbguy rnbguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, guys ! 🎉

Just one last nit. Since the rust MBT tests now generate traces, fixing and executing them on the fly, maybe some of the Scripts are redundant. Feel free to remove them accordingly before merging.

@romac romac removed the work in progress Work in progress label Nov 28, 2023
@hvanz hvanz marked this pull request as ready for review November 28, 2023 09:54
Copy link
Member

@romac romac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work guys, thanks so much! 🚀

@romac romac merged commit 1dafd49 into main Nov 28, 2023
10 checks passed
@romac romac deleted the hvanz/mbt-votekeeper branch November 28, 2023 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mbt Model-Based Testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test: MBT tests for vote keeper
5 participants