Ampelofilosofies

homeaboutrss
The mind is like a parachute. If it doesn't open, you're meat.

ForgetMeNot: Organized xUnit reports in Jenkins

12 Mar 2020

One of the hackiest formats under wide adoption is the JUnit report format.

It is not really from JUnit, it’s from Ant. It is not really documented anywhere - some people try to figure it out by reading the source (after looking for it first in the JUnit sources - lots of fun following the different StackOverflow threads on this).

Given that there is no official schema or spec I am not surprised in the totally inconsistent behaviour of any tool claiming to read/write or understand the JUnit report format.

There is one particular hero @WindyRoadTech, who created an XSD schema. What I am about to do will fail the schema validation (at the time of writing).

The problem with JUnit reports created by tools other than Ant is that the original implementation makes some assumptions.

These are that you are basically running Java and that your code is organized in Java packages, i.e. it has a format of Package.Something.Whatever.Class.

What Ant did was split the full class name of a testcase and add pieces of it as attributes to the testsuite and testcase elements of the report.

Time goes by, and all kinds of tools start using this JUnit format - I blame Jenkins and the JUnit report plugin for introducing people to pretty colors they could show other people.

More time went by, generations in Internet years. The JUnit plugin was too limited, so the xUnit plugin was born. This one could understand lots of report formats.

But the way the report is organized did not change. xUnit shows tests grouped in packages. It figures out what to use as a package name by using the classname attribute of the testsuite elements

<testsuite name="InterestingTests" tests="5" failures="0" disabled="0" errors="0" time="50.582" timestamp="2020-03-12T17:40:33" classname="Integration">

You would expect that all test cases within that element would be listed under that package. But no. Apparently the reporter parses the classname for each testcase and builds “packages” out of that.

So if you have a bunch of let’s say GTest tests and emit an XML report, all of your tests are going in the Root package, since, you know, no Java namespaces and one has to choose a default name (which GTest hard-codes).

What is a build enginneer to do, but post-process that XML report so that his team gets nicely categorized tests. No need for fancy code, brute force it in Ruby:

# Rewrites (in place) an XML report produced by gtest to add a classname to testsuites and
# adapt the classnames to test cases.
# so that they sort nicely when presented by the xUnit plugin in Jenkins
def classify_test_report(test_report, test_class)
  content = File.read(test_report)
  xml = Nokogiri::XML(content)

  suites = xml.root.xpath("//testsuite")
  testcases = xml.root.xpath("//testcase")
  testcases.each do |entry|
    entry["classname"] = "#{test_class}.#{entry["classname"]}"
  end
  suites.each do |entry|
    entry["classname"] = test_class
  end
  write_file(test_report, xml.serialize)
  return test_report
end

That write_file comes from gaudi, but, you know, it writes the file to disk.

blog comments powered by Disqus