Upload Selenium/JUnit test results to Quality Center

Since I’ve had so many requests (the latest being today), and since I’m working on something very similar for a current client, I decided to take some time today to respond in detail about integrating QC with JUnit & Selenium.

The steps I’ve used for Selenium / QC integration are:

1. Write tests with Selenium using an open source test framework (JUnit/TestNG/PHPUnit/NUnit/RSpec/py.test)

2. Map your test cases to QC test cases. I described how I did this in detail by extending JUnit in these posts:

Note that this isn’t necessarily the way I’d do it now. Annotations are useful, but I think a mapping file is perhaps easier. Simply create a spreadsheet with the QC TestId in one column and the xUnit test name in another column. This is a little trickier with parameterized tests.

3. Parse the tests results from your test runner and update Quality Center using the OTA API (OTAClient.dll is installed with QC Explorer & can be downloaded from QC. Go to Help->Addins Page->HP Quality Center Client Side Setup Add-in)

I have a quick example of how to connect to Quality Center using OTAClient here:

Connecting to HP/Mercury Quality Center from a client side script

You can check out my sample QCIntegration project on GitHub:
http://github.com/fijiaaron/QCIntegration

I’m adding more details about integrating with Quality Center on One Shore.

Advertisements

12 thoughts on “Upload Selenium/JUnit test results to Quality Center

  1. Hi Aaron,

    Thanks a lot for this post. I have been working on the integration of selenium and quality center from the past few days but could not find anything that would help.Sure yours is the first post which I found very useful. Could you please explain this part of your post.

    You can now have your junit test cases extend TestBase and get a csv report of test coverage.

    public class MyTest extends TestBase {

    @Test
    @QCTestCases(covered = { “QC-TEST-1”, “QC-TEST-2” }, related = { “QC-TEST-3”, “QC-TEST-4”, “QC-TEST-5” })
    public void testSomething() {
    //implementation…
    }

    @Test
    @QCTestCases(covered = { “QC-TEST-6”} })
    public void testSomethingElse() {
    //implementation…
    }
    }

    • That is just an example of using an annotation to identify which Quality Center test cases are covered in your JUnit test cases.


      public class MyTest extends TestBase {
      @Test
      @QCTestCases(covered={"QC-TEST-1", "QC-TEST-2"}, related={"QC-TEST-3"})
      public void testSomething() {
      assertTrue(true);
      }
      }

      For a clear example of what’s happening with a test case that looks like this:


      public class MyTest extends TestBase {
      @Test
      @QCTestCases(covered={"QC-TEST-1", "QC-TEST-2"}, related={"QC-TEST-3"})
      public void testSomething() {
      assertTrue(true);
      }
      }

      You can use reflection in the base class to get the annotations like this:


      public class TestBase {
      @Rule public TestName testName = new TestName();
      QCTestCases qcTestCases;
      @Before
      public void getQCTestCoverage() {
      try {
      Method m = this.getClass().getMethod(testName.getMethodName());
      if (m.isAnnotationPresent(QCTestCases.class)) {
      qcTestCases = m.getAnnotation(QCTestCases.class);
      }
      } catch (SecurityException e) {
      e.printStackTrace();
      } catch (NoSuchMethodException e) {
      e.printStackTrace();
      }
      }
      @After
      public void writeCoverageReport() {
      StringBuilder result = new StringBuilder();
      result.append(testName.getMethodName());
      result.append(",");
      for (String qcTestId : qcTestCases.covered()) {
      result.append(qcTestId);
      result.append(",");
      }
      System.out.println("qc tests covered: " + result);
      }
      }

      It’s probably a bit unnecessarily complex, since “related” tests are not needed in most cases — I was using it purely for analysis, and if you have a 1:1 mapping you wouldn’t need to use arrays either, and your annotation interface could be as simple as this:


      public @interface QCTestCase {
      String name();
      }

      I’ve added the code you see here to the QCIntegration repository on github as well.

      Of course, the next step is to get the test results in the csv file. That’s not detailed here, but I’ll work on demonstrating that another day.

  2. Thank you Aaron. My TestBase class is generating qcCoverageReport.csv but it is not storing any logs in the file.
    This is my log4j.properties file:

    log4j.appender.stdout=org.apache.log4j.ConsoleAppender
    log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
    log4j.appender.stdout.layout.ConversionPattern =%d{ABSOLUTE} %5p %c{1}:%L – %m%n
    log4j.appender.stdout=org.apache.log4j.DailyRollingFileAppender
    log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
    log4j.appender.stdout.layout.ConversionPattern =%5p (%F:%L) [%d] – %m%n
    log4j.appender.stdout.File=C:\\junit\\qcCoverageReport.csv
    log4j.rootLogger=INFO, stdout

    Can you please tell me where the mistake is..?

    Thanks,
    Jyothi

  3. Pingback: Selenium QC integration summary « Fiji Ecuador Seattle Greece Montana

  4. Pingback: Selenium QC integration update « Fiji Ecuador Seattle Greece Montana

  5. I have a requirement. I do the verification in unix box. Once the verification is done the test results(if it fails) are send as attachment through email. I need to send this attachment to QC and open a defect.

    • Well, the DLL can’t be used on Unix, but you could put the results on a database and then poll it from an app that uses QCIntegration to update reports in Quality Center. Opening a defect in Quality Center isn’t supported by the QC Integration script, but it can be accomplished with the OTA API. You can send me an email at one-shore.com and I could put together a quote for you.

  6. Hey Aaron,

    I am wondering if you can help me with these two issues:
    1) I am trying to update the results from Selenium into HP ALM and I am facing a problem when I try to launch the tests.
    I am using JUnit and even if the test should fail (ex: wrong username or wrong password tapped) , JUnit sees it as a passed test. (it fails only if it can’t find the url).
    How can I retrieve the good results that Selenium gives? Is there a way that I can access Selenium classes/ methods in order to get the good results?
    (for not using the Asserts of JUnit).
    2) I want to update the TestSteps into HP ALM also. I know that Selenium commands could be considered as TestSteps.
    Is there a possibility to get their results too ?

    Thank you very much,
    Alla

    • Alla-

      I’m not quite clear on your first question. Selenium allows you to access the DOM at any point (unless the browser crashes), but selenium itself isn’t a test framework, it’s just a driver for the browser. Whether using Junit or not, you can mark a failure however you want. Junit uses assertions as it’s default mechanism, but you could do something like:

      if (selenium.isTextPresent(“404”)) { fail(“page not found”); }

      But that accomplishes the same thing as this:

      assertFalse(selenium.isTextPresent(“404”));

      Or even bypass the Junit mechanism altogether and implement this:

      if (! selenium.isElementPresent(logoutButton)) { writeResultToQualityCenter(testcase, “Failed”); }

      If you just do this:

      selenium.click(logoutButton);

      If logoutButton is not found, it will timeout and throw an exception — in which case you will have a Junit error instead of a failure.

      For your second question, yes, you can use the OTA API to create test cases, but I don’t believe HPQC allows you to create test case steps on the fly during execution. But you could do it in two steps:

      1. Create a test case in QC based on your selenium steps.
      2. Start a test run and update the results in QC.

      I don’t have code for doing that, because it was never a requirement for me, but I am available for consulting. However, I don’t have a Quality Center/ALM license so you’d have to supply me with access.

      If you have any other questions or would like to clarify, I’m always willing to help.

      -Aaron

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s