Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add update option to integration script #544

Merged
merged 4 commits into from
Nov 14, 2023

Conversation

benmss
Copy link
Member

@benmss benmss commented Nov 3, 2023

This PR aims to provide a way of automatically updating the expected results used by the integration tests, closing #542.

The initial implementation achieves this by replacing all python commands with a variable that can be overridden when the integration test script is passed a specific third argument, thereby performing a copy operation on the two file arguments that would otherwise be compared. At the current time, this avoids performing copy operations on other possible python commands by relying on the fact that only comparison commands use three arguments, with arguments 2 and 3 being files.

Usage:

  1. Run the integration tests as usual.
  2. Verify any differences between Macaron output and expected results are legitimate.
  3. Run updater to replace outdated expected files.

Command:
make integration-tests-update

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Nov 3, 2023
scripts/dev_scripts/integration_tests.sh Outdated Show resolved Hide resolved
scripts/dev_scripts/integration_tests.sh Outdated Show resolved Hide resolved
scripts/dev_scripts/integration_tests.sh Outdated Show resolved Hide resolved
scripts/dev_scripts/integration_tests.sh Outdated Show resolved Hide resolved
@benmss benmss force-pushed the 542-add-integration-test-updater branch from 43b870e to b4d895c Compare November 13, 2023 05:32
@behnazh-w behnazh-w changed the title chore: add update option to integration script test: add update option to integration script Nov 13, 2023
@benmss benmss merged commit 96e5e87 into staging Nov 14, 2023
10 checks passed
@nathanwn nathanwn deleted the 542-add-integration-test-updater branch December 14, 2023 02:16
art1f1c3R pushed a commit that referenced this pull request Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants