Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple chain support for benchmarking #1060

Merged
merged 2 commits into from
Oct 17, 2023
Merged

Conversation

mc2eqe
Copy link
Contributor

@mc2eqe mc2eqe commented Oct 17, 2023

Running benchmarks only allows user to specify one runtime at a time.
This is problematic if we want to benchmark all of them at once, since each trigger
will rebuild the entire binary which takes time.

To improve this, suggestion is to change

[chain name] - astar-dev, shiden-dev, shibuya-dev, dev
[pallet names] - Use "," for multiple pallets, "all" for all pallets
/bench [chain name] [pallet names]

into

[chain names] - Use "," for multiple runtimes. Available values are: astar-dev, shiden-dev, shibuya-dev, dev
[pallet names] - Use "," for multiple pallets, "all" for all pallets
/bench [chain names] [pallet names]

Pull Request Summary

Check list

  • added or updated unit tests
  • updated Astar official documentation
  • added OnRuntimeUpgrade hook for precompile revert code registration
  • updated spec version
  • updated semver

@mc2eqe mc2eqe added the ci This PR/Issue is related to the topic "CI" label Oct 17, 2023
@mc2eqe mc2eqe self-assigned this Oct 17, 2023
@github-actions
Copy link

Code Coverage

Package Line Rate Branch Rate Health
pallets/contracts-migration/src 0% 0%
pallets/xc-asset-config/src 53% 0%
precompiles/utils/src 55% 0%
precompiles/xcm/src 75% 0%
primitives/src/xcm 66% 0%
precompiles/assets-erc20/src 76% 0%
pallets/ethereum-checked/src 48% 0%
chain-extensions/pallet-assets/src 0% 0%
pallets/dynamic-evm-base-fee/src 81% 0%
pallets/xvm/src 40% 0%
precompiles/dapps-staking/src 93% 0%
chain-extensions/types/dapps-staking/src 0% 0%
precompiles/sr25519/src 79% 0%
precompiles/utils/macro/src 0% 0%
chain-extensions/types/xvm/src 0% 0%
pallets/block-reward/src 85% 0%
precompiles/substrate-ecdsa/src 78% 0%
precompiles/xvm/src 75% 0%
primitives/src 62% 0%
chain-extensions/types/assets/src 0% 0%
chain-extensions/xvm/src 0% 0%
precompiles/utils/src/testing 38% 0%
pallets/dapps-staking/src/pallet 85% 0%
pallets/dapps-staking/src 81% 0%
pallets/unified-accounts/src 80% 0%
pallets/collator-selection/src 69% 0%
chain-extensions/dapps-staking/src 0% 0%
Summary 56% (2192 / 3892) 0% (0 / 0)

Minimum allowed line rate is 50%

Copy link
Member

@Dinonard Dinonard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR description is missing 😄

# Benchmark each pallet.
for PALLET in "${PALLETS[@]}"; do
NAME_PRFX=${PALLET#*_}
NAME=${NAME_PRFX//::/_}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding this 👍🏻

@mc2eqe mc2eqe merged commit c223157 into master Oct 17, 2023
10 checks passed
@mc2eqe mc2eqe deleted the fix/multiple-chain-bench branch October 17, 2023 15:11
@mc2eqe mc2eqe linked an issue Oct 17, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci This PR/Issue is related to the topic "CI"
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] Benchmark multiple runtimes at once
3 participants