Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test bitte-ci #1106

Open
wants to merge 1 commit into
base: mig-bitte-ci
Choose a base branch
from
Open

test bitte-ci #1106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .buildkite/default.nix
View file
Open in desktop

This file was deleted.

181 changes: 0 additions & 181 deletions .buildkite/pipeline.nix
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions .buildkite/pipeline.yml
View file
Open in desktop

This file was deleted.

25 changes: 0 additions & 25 deletions .buildkite/shell.nix
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions ci.cue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ ci: {
test_ets: {
label: "ETS tests"
after: ["compile"]
command: ["./test-ets-bitte.sh"]
command: ["./test-ets.sh"]
outputs: [
"/local/repo/mantis-log.txt",
"/local/repo/retesteth-GeneralStateTests-log.txt",
Expand All @@ -100,7 +100,7 @@ ci: {
publish: {
label: "Publishing libraries to Maven"
after: ["test_crypto", "test_rlp", "test_unit"]
command: [ ".buildkite/publish.sh" ] // TODO
command: [ "./publish.sh" ] // TODO
outputs: [
"/local/repo/target/universal/mantis-*.zip",
]
Expand Down
7 changes: 3 additions & 4 deletions ets/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,8 @@ running them separately.

## Continous integration

The tests are run on CI. For more details look at `.buildkite/pipeline.nix` and
`test-ets.sh`. Output is stored as artifacts and a summary is added as
annotation.
The tests are run on CI. For more details look at `./ci.cue` and
`test-ets.sh`. Output is stored as artifacts.

Two test suites are run; GeneralStateTests and BlockchainTests. These seem to
be the only ones maintained and recommended at the moment.
Expand Down Expand Up @@ -72,4 +71,4 @@ made by retesteth and also print out the state by using our `debug_*` endpoints.

- `debug_accountRange` will only list accounts known at the genesis state.
- `debug_storageRangeAt` is not able to show the state after an arbitrary transaction inside a block.
It will just return the state after all transaction in the block have run.
It will just return the state after all transaction in the block have run.
0 .buildkite/publish.sh → publish.sh
View file
Open in desktop
File renamed without changes.
14 changes: 5 additions & 9 deletions shell.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
{ sources ? import nix/sources.nix, pkgs ? import ./nix { } }:
with pkgs;

if __getEnv "BUILDKITE" == "true" then
import .buildkite/shell.nix { inherit sources pkgs; }
else
with pkgs;

mkShell {
nativeBuildInputs = [ protobuf sbt ];
inputsFrom = [ mantis ];
}
mkShell {
nativeBuildInputs = [ protobuf sbt ];
inputsFrom = [ mantis ];
}
28 changes: 0 additions & 28 deletions test-ets-bitte.sh
View file
Open in desktop

This file was deleted.

28 changes: 3 additions & 25 deletions test-ets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,15 @@ done

final_exit_code=0

function run_and_annotate {
function run {
echo "running retesteth $1"
timeout 15m ets/retesteth -t "$1" -- --verbosity 3 &> "retesteth-$1-log.txt"
exit_code=$?
echo "retesteth $1 exit code: $exit_code"

style="info"
if [[ "$exit_code" -gt "0" ]]; then
final_exit_code="$exit_code"
style="error"
fi

summary=$(sed -n '/Total Tests Run/,$p' "retesteth-$1-log.txt")
if [[ -z "$summary" ]]; then
summary="retesteth crashed; check the artifacts"
fi
passed=$(grep -oP 'Total Tests Run: \d+' "retesteth-$1-log.txt")
failed=$(grep -oP 'TOTAL ERRORS DETECTED: \d+' "retesteth-$1-log.txt")

cat <<EOF | buildkite-agent annotate --context "retesteth-$1" --style "$style"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any Bitte CI equivalent of this? Since the ETS step currently fails (was marked "softFail" in Buildkite) it would be convenient to be able to see the actual success / failure status.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not (yet). We are trying to get a PoC running for now. You'll be able to use buildkite in parallel for a while until we can make sure Bitte CI is in a usable shape.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that sounds good 👍

Can you make a bitte / ci-specific version of this file (again)? I'm also making one soon for doing the same thing but locally and it would be good to distinguish these use cases.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean this https://github.com/input-output-hk/mantis/blob/mig-bitte-ci/test-ets-bitte.sh ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly.

<details>
<summary>retesteth: $1 -- $passed -- $failed</summary>
<pre class="term"><code>
$summary
</code></pre>
</details>
EOF
}

run_and_annotate "GeneralStateTests"
run_and_annotate "BlockchainTests"
run "GeneralStateTests"
run "BlockchainTests"

echo "shutting down mantis"
kill %1
Expand Down