Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Continuous Integration on ScaLAPACK #56

Merged
merged 2 commits into from
Mar 11, 2022

Conversation

weslleyspereira
Copy link
Collaborator

@weslleyspereira weslleyspereira commented Mar 7, 2022

Adds simple example that builds ScaLAPACK using GNU make.

Note that:

  1. The workflow finishes as expected. See: https://github.com/weslleyspereira/scalapack/actions/runs/1948529370
  2. I tried to add a workflow for MacOS. In the process I found an implicit function declaration: (note that option -Werror=implicit-function-declaration is active by default on MacOS based on Implicit declaration of function is invalid in C99 RTimothyEdwards/magic#69 (comment))
cgsum2d_.c:230:10: error: implicit declaration of function 'BI_svmcopy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
         BI_cvmcopy(Mpval(m), Mpval(n), A, tlda, bp->Buff); 
  1. I tried to add a workflow for CMake. I still couldn't compile ScaLAPACK using CMake in my personal computer.

@weslleyspereira weslleyspereira changed the title Add Continuous Integration on ScaLAPACK Adds Continuous Integration on ScaLAPACK Mar 7, 2022
@weslleyspereira weslleyspereira force-pushed the try-CI branch 5 times, most recently from 547d5a2 to a25740d Compare March 7, 2022 23:25
@weslleyspereira weslleyspereira marked this pull request as ready for review March 8, 2022 00:03
@weslleyspereira
Copy link
Collaborator Author

I tried to add a workflow for CMake. I still couldn't compile ScaLAPACK using CMake in my personal computer. The reason:

  • I can compile the library scalapack.
  • I couldn't build the tests because there are several undefined reference errors related to the MPI interface. I believe I should include the dependency on the MPI library somewhere.

Does anyone know how to compile ScaLAPACK using CMake on a Linux machine? Thanks!

This was referenced Mar 8, 2022
@ajaypanyala ajaypanyala mentioned this pull request Mar 8, 2022
@joseeroman
Copy link
Contributor

On macOS I am using -Wno-implicit-function-declaration in CFLAGS

@weslleyspereira weslleyspereira force-pushed the try-CI branch 8 times, most recently from 7fa4564 to 93dd51c Compare March 8, 2022 18:03
@weslleyspereira
Copy link
Collaborator Author

The current CI has:

@Baljak
Copy link
Contributor

Baljak commented Mar 8, 2022

Looks great, do you think that a Windows build could be added using Mingw?

@weslleyspereira
Copy link
Collaborator Author

Looks great, do you think that a Windows build could be added using Mingw?

That is the idea. As soon as the CI is working for Linux, it would be great to have it working for Windows and MacOS.

Currently, I am facing the following problem: The GitHub-hosted runners only have 2 cores (https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners), and some of the tests in ScaLAPACK need at least 4 MPI processes. Those tests, therefore, take too long to complete. Does anyone have an idea to improve this workflow? Thanks!

@weslleyspereira
Copy link
Collaborator Author

One example of how using 2 MPI processes instead of 4 changed the times on the Github workflow:

Using 4 MPI processes for every test: (https://github.com/weslleyspereira/scalapack/runs/5471640656) (still running)

      Start 44: xzhrd
32/70 Test #44: xzhrd ............................   Passed  120.30 sec

Using 2 MPI processes whenever possible: (https://github.com/weslleyspereira/scalapack/runs/5472056161)

      Start 44: xzhrd
54/70 Test #44: xzhrd ............................   Passed    0.04 sec

The only difference between 2baaae6 and 28ff3c8 is the number of MPI processes I am using.

This a recent workflow using 4 MPI processes per test: https://github.com/weslleyspereira/scalapack/runs/5472056161. Total time 3h30min. The workflow https://github.com/weslleyspereira/scalapack/runs/5472056161 finished with 1h10min.

Amongst the tests using 4 MPI processes, the following take more than 10min to complete:

 The following tests FAILED:
	 65 - xssyevr (Timeout)
	 66 - xdsyevr (Timeout)
	 67 - xcheevr (Timeout)
	 68 - xzheevr (Timeout)

@joseeroman
Copy link
Contributor

A workaround would be to use a self-hosted runner. Or change the tests (if possible) so that they use 2 processes at most.

@weslleyspereira
Copy link
Collaborator Author

A workaround would be to use a self-hosted runner. Or change the tests (if possible) so that they use 2 processes at most.

Thanks! Yes, I found those two approaches. I was hoping there was some other configuration on the GIthub side that could help us. So far, I reduced the number of MPI processes for all the tests that allowed me to do so.

@weslleyspereira weslleyspereira force-pushed the try-CI branch 2 times, most recently from 6d2ff94 to 1d4a278 Compare March 11, 2022 00:30
- The first one builds with GNU Make and tests the examples.
- Second one builds with CMake and runs all tests.
- Fix bugs on the CMake build system (same fix from Reference-ScaLAPACK#57).
- ${MPIEXEC_PREFLAGS} was added to the test executables and allow for passing flags to mpiexec.
@weslleyspereira
Copy link
Collaborator Author

weslleyspereira commented Mar 11, 2022

Hi. I think I got everything working good. See:
https://github.com/weslleyspereira/scalapack/actions/runs/1966370083
https://github.com/weslleyspereira/scalapack/actions/runs/1966370085

To summarize:

This PR adds 2 Github Workflows to ScaLAPACK.

  • The first one builds with GNU Make and tests the examples.
  • Second one builds with CMake and runs all tests.
  • This PR also fix bugs on the CMake build system (same fix from Fixes #47 #57).
  • ${MPIEXEC_PREFLAGS} was added to the test executables. This allow for passing flags to mpiexec.

I rebased all commits to simplify the log. I think this PR is ready to go.

Again: It would be great to have recipes working for Windows and MacOS. But I think we should merge the current modifications just to have some tests in place.

@langou langou merged commit 681a6cf into Reference-ScaLAPACK:master Mar 11, 2022
@weslleyspereira
Copy link
Collaborator Author

Looks great, do you think that a Windows build could be added using Mingw?

Hi! I started a discussion on #60. Maybe you want to comment on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants