Old additional test output on buildserver available?

[escay]

I probably made the build fail with a refactoring. The build fails in the Old Additonal / 'ant tests'. For example:
https://ci.eclipse.org/glassfish/job/glassfish_build-and-test-using-jenkinsfile/job/PR-24870/2/#showFailuresLink

Test Result (15 failures / +15)
    ant-tests / ejb_group_2 / ejb_group_2.ejb-mdb-cmt. ejb-mdb-cmtID

The question: is there a way to see more detail on the build server?
I looked at the Console Ouput log, but I cannot seem to find test details, only a list of PASSED, FAILED and TOTAL.

[2024-03-21T13:02:35.435Z]      [exec] input file=/home/jenkins/agent/workspace/_test-using-jenkinsfile_PR-24870/appserver/tests/appserv-tests/test_resultsValid.xml
[2024-03-21T13:02:35.435Z]      [exec] 
[2024-03-21T13:02:35.435Z]      [exec] ************************
[2024-03-21T13:02:35.435Z]      [exec] PASSED=   31
[2024-03-21T13:02:35.435Z]      [exec] ------------  =========
[2024-03-21T13:02:35.435Z]      [exec] FAILED=   1

I found the TOTAL definition in the "resultCount.sh" files.

I probably have to run the tests locally using the

./runtests.sh ejb_group_2
./runtests.sh ejb_group_embedded

scripts.

I succeeded running the 'ant tests' on a Windows machine using WLS. Some notes for running these tests:

• Use WLS (version 1 for higher disk speed)
• Install Sdkman
• Install java via sdkman
• Install maven via sdkman
• Install ant via sdkman
• Build glassfish again due to maven repo not being reused within WLS environment.
• Run ./runtests.sh ejb_group_2 -> succeeds.
  (TODO: glassfish/target/glassfish7 version created by the ant test seems to be an older version than my own build, perhaps -Pfastest is not enough)

[escay]

You can find the logs based on the group name of the tests. Example:

You can search for "ejb_group_embedded:" in the console output. It gives you the start point of the tests like:

[2024-03-21T12:57:55.059Z] ejb_group_embedded:
[2024-03-21T12:57:55.059Z]      [echo] 
[2024-03-21T12:57:55.059Z]      [echo]  
[2024-03-21T12:57:55.059Z]      [echo]  **************************************************
[2024-03-21T12:57:55.059Z]      [echo]  * Running ejb32/embedded 

To find the exact test in the set is more difficult. Output to recognize 'ant test' output depends on each test implementation
Example some tests start with .......... Testing module, but there are over 400+ Client.java classes and not all follow this pattern.

     [java] .......... Testing module: ejb-ejb32-embedded-autoclose-ejb

So it is a bit easier to find the logs near a failing 'ant test' and search for "[java] Total FAIL:" where the number is not 0.
Above the failure you find the output of running the test.

     [java] Total PASS: 0
     [java] Total FAIL: 1
     [java] Total DID NOT RUN: 0

Finding the failing test in the code based on the test name on the build server is a bit more difficult.
Example finding:

Requires searching in *.java for part of the string. I think these two match the code in:

stat.addStatus("bmp twolevel", stat.FAIL); -> /appserv-tests/devtests/ejb/bmp/twolevel/client/TwoLevelClient.java

stat.addStatus("EJB embedded with remote " + run, stat.FAIL); ->/appserv-tests/devtests/ejb/ejb31/embedded/remote/client/Client.java

NOTE: the . and dashes in the generated report name, so use short search strings to avoid these.

[dmatej]

Yes, that is why I call these tests an army of zombies, it is hard to debug them, and we are slowly moving them to the "application-tests" module. It means always to read them through, do some maintenance, understand what exactly is tested and then using Shrinkwrap instead of Ant to build-deploy-test-undeploy them.

With Ant you can comment out some parts in build.xml files, so iterations are faster, set some custom logging and then compare logs before/after to find out where it does something different (in extreme, usually you will find rather some exception or so and you guess what is broken).

[arjantijms]

In addition, many of those Ant files also allow you to set a global environment property to execute only a single test.

See stuff like this:

 <macrodef name="run-security-test" xmlns:if="ant:if" xmlns:unless="ant:unless">
        <attribute name="path" />
        <sequential>
            <local name="do-run-test" />
            <condition property="do-run-test" else="false">
                <or>
                    <equals arg1="${env.security}" arg2="@{path}" />
                    <not>
                        <isset property="env.security" />
                    </not>
                </or>
            </condition>

            <sequential if:true="${do-run-test}">
                <local name="absolute.path" />
                <property name="absolute.path" location="@{path}" />
                <echo message="${line.separator}${line.separator}${line.separator}" />
                <echo message=" ***************************************************************************" />
                <echo message=" * Running group @{path} " />
                <echo message=" * Path: ${absolute.path} " />
                <echo message=" ***************************************************************************" />
                <echo message="${line.separator}${line.separator}${line.separator}" />
                <ant dir="@{path}" target="all" />
            </sequential>
        </sequential>
    </macrodef>