Two Behat 3.0 + Jenkins tricks

These are a couple of really useful tips if you're use Behat 3.0 and Jenkins

Adding a screenshot to junit xml on failure

It's fairly common and easy to save a screenshot on a test failure, and whilst you can print out the name of the saved file to the console, it's much more useful to have it written into the xml file. The difficulty with this is that although the exception is written out to Behat's junit formatter, it's not easy to modify the exception message after it's been thrown. Taking inspiration from a post on loft digital's blog (http://blog.loftdigital.com/take-screenshot-with-behat-after-failed-step) we adopted their approach, but modified it for use with Behat 3. Add this function to your feature context, implement your own takeScreenshot method which returns the filename of the screenshot. This method will then run after each step, and if the step failed it takes a screenshot, then use PHP's reflection classes to access the private message property of the exception to append the name of the saved screenshot onto the error message, which is then written out to the junit xml.

  /**
   * Take screenshot on failing step, appending filename to exception.
   *
   * @AfterStep
   */public function takeScreenShotAfterFailedStep(AfterStepScope $scope){if (!$scope->getTestResult()->isPassed()) {$screenshot_filename = $this->takeScreenshot();$result = $scope->getTestResult();if ($result->hasException()) {$exception = $result->getException();$reflectionObject = new ReflectionObject($exception);$reflectionObjectProp = $reflectionObject->getProperty('message');$reflectionObjectProp->setAccessible(true);$reflectionObjectProp->setValue($exception, $exception->getMessage() . " - Screenshot saved to {$filename}");}}}

Grouping test results

Behat's junit formatter generates xml that considers each feature file as its own test suite. If you're using the Multi Module Tests Publisher for Jenkins (https://wiki.jenkins-ci.org/display/JENKINS/Multi+Module+Tests+Publisher) you end up with each modules corresponding to feature files, rather than test runs. Processing your junit xml through this xslt rectifies that. The xml can be transformed using xsltproc like this:

xsltproc -o out.xml --stringparam suite_name my-test-suite-name reform_junit.xslt in.xml

<!--?xml version="1.0"?--><xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"><xsl:param name="suite_name"><xsl:template match="/testsuites"><testsuites><xsl:attribute name="name"><xsl:value-of select="$suite_name"></xsl:value-of></xsl:attribute><testsuite><xsl:attribute name="name"><xsl:value-of select="$suite_name"></xsl:value-of></xsl:attribute><xsl:attribute name="tests"><xsl:value-of select="sum(testsuite/@tests)"></xsl:value-of></xsl:attribute><xsl:attribute name="skipped"><xsl:value-of select="sum(testsuite/@skipped)"></xsl:value-of></xsl:attribute><xsl:attribute name="failures"><xsl:value-of select="sum(testsuite/@failures)"></xsl:value-of></xsl:attribute><xsl:attribute name="errors"><xsl:value-of select="sum(testsuite/@errors)"></xsl:value-of></xsl:attribute><xsl:copy-of select="testsuite/testcase"></xsl:copy-of></testsuite></testsuites></xsl:template></xsl:param></xsl:stylesheet> 

I'm Andrew - a Tech Architect and Agile Delivery Consultant. Find out more about the services I offer here