Can I integrate jscoverage with automation framework.

[Amit]

We have automation framework written using selenium. I want to find out code coverage for our automation framework. I see jsCoverage does give code coverage report but you always have to start by accessing jscoverage.html. Can we avoid this?

Ideally I want to do this...
1. Start code coverage measurement.
2. Run the automation framework.
3. See the code coverage reports.

If possible, there should not be any manual interaction involved in performing above three steps.

Regards,
Amit

[Ed]

You may want to look at http://mercenary-code.blogspot.com/2010 ... enium.html, which describes a method of getting coverage data from Selenium tests without using jscoverage.html.

[Guest]

I'm very new to jsCoverage. I have written my selenium tests in java. Can you please tell me what libraries I need to import in my java project so that I can use jsCoverage functions like jscoverage_report().

Do I need to use window mode? If yes, How do we use window mode? Detailed steps will be appreciated.

Regards,
Amit

[Ed]

Basically, if you want to automatically generate a code coverage report when your tests are run, without any manual interaction, you are going to need to add some code to your tests to generate the report. Keep in mind that you will need a good understanding of how both Selenium and JSCoverage work to do this. Broadly speaking, the steps you would follow would be something like this:

1. Modify your Java test code by adding some reporting code to be executed after your tests are run, e.g., to your tearDown method:

Code: Select all

    String script =
      "(function () {\n" +
      "  var result = '';\n" +
      "  for (var file in window._$jscoverage) {\n" +
      "    result += file + '\\n';\n" +
      "    var coverage = window._$jscoverage[file];\n" +
      "    var length = coverage.length;\n" +
      "    for (var line = 0; line < length; line++) {\n" +
      "      var value = coverage[line];\n" +
      "      if (value === undefined || value === null) {\n" +
      "        continue;\n" +
      "      }\n" +
      "      result += ' ' + line + ': ' + value + '\\n';\n" +
      "    }\n" +
      "  }\n" +
      "  return result;\n" +
      "})()\n";
    String result = selenium.getEval(script);
    System.out.println(result);

2. Instrument your JavaScript code using the jscoverage program (see the manual for instructions on running jscoverage).

3. Run your test suite, using the instrumented JavaScript code. You should see a simple text report showing which lines are covered in each JavaScript file.

[Amit]

I have followed all the three steps as mentioned above. Instrumented code on my server and run tests on this code. Also called same function as mentioned below on test completion but did not see any code coverage report.

Some debugging showed that jscoverage_report is returning null everytime.
If I run same test operation manually and try to access code coverage report through jscoverage.html; everything is working fine.

Am I doing anything wrong?
Do I need to run jscoverage-server program even though I am running tests on instrumented code?

[Ed]

Don't call the jscoverage_report function - that won't be available. What you want to do is generate a custom report, using the code above. Just the code above is all you need to generate the report.

[Amit]

Ed,
Thanks for your help. I'm able to generate reports in text format.

What do I need to do to generate exactly same reports that I get by manually using jsCoverage program.

Basically I want my reports to show following fields at least...
1. js File Name.(Already displayed.)
2. Code coverage -- Total lines, lines executed.
3. Link to the source code files with non-covered source code lines highlighted.


Regards,
Amit

[Ed]

Once you have the coverage data, you can do anything you like with that; if you just want a report which looks the same as a regular JSCoverage report, you can do it like this:

1. Modify the above code to generate JSON:

Code: Select all

    String script =
      "(function () {\n" +
      "  var pad = function (s) {\n" +
      "    return '0000'.substr(s.length) + s;\n" +
      "  };\n" +
      "  var quote = function (s) {\n" +
      "    return '\"' + s.replace(/[^ -~]|\"|\\\\/g, function (c) {\n" +
      "      return '\\\\u' + pad(c.charCodeAt(0).toString(16));\n" +
      "    }) + '\"';\n" +
      "  };\n" +
      "  var json = [];\n" +
      "  for (var file in window._$jscoverage) {\n" +
      "    var coverage = window._$jscoverage[file];\n" +
      "    var array = [];\n" +
      "    var length = coverage.length;\n" +
      "    for (var line = 0; line < length; line++) {\n" +
      "      var value = coverage[line];\n" +
      "      if (value === undefined || value === null) {\n" +
      "        value = 'null';\n" +
      "      }\n" +
      "      array.push(value);\n" +
      "    }\n" +
      "    var source = coverage.source;\n" +
      "    var lines = [];\n" +
      "    length = source.length;\n" +
      "    for (var line = 0; line < length; line++) {\n" +
      "      lines.push(quote(source[line]));\n" +
      "    }\n" +
      "    json.push(quote(file) + ':{\"coverage\":[' + array.join(',') + '],\"source\":[' + lines.join(',') + ']}');\n" +
      "  }\n" +
      "  json = '{' + json.join(',') + '}'\n" +
      "  return json;\n" +
      "})()\n";
    String result = selenium.getEval(script);

2. Instead of printing out the result, store it to a file named "jscoverage.json" in a directory somewhere; e.g., "jscoverage-report/jscoverage.json".

3. Copy the following files (these are from the JSCoverage source distribution) into the same directory containing "jscoverage.json":

• jscoverage.html
• jscoverage.js
• jscoverage.css
• jscoverage-highlight.css
• jscoverage-ie.css
• jscoverage-throbber.gif

4. Append the following line to the end of the file "jscoverage.js":

Code: Select all

jscoverage_isReport = true;

Once you have done all that, you can view the report by opening "jscoverage.html" in your web browser.

[Amit]

Thanks a lot Ed. I'm able to generate reports with your code.

The only problem I'm facing is I'm not able to merge results coming from different tests. Is there any way I can acheive this?

[Ed]

Depending on your test suite, you may be able to change the Selenium run mode to *firefoxproxy to get the coverage from your entire test suite. Note that some tests may not work with *firefoxproxy. See http://seleniumhq.org/docs/05_selenium_rc.html for details on different Selenium run modes.

[Amit]

Thanks Ed. Actually lot of my test cases do not run with *firefoxproxy option. Anyways I have added merge logic to my code which gets executed in @aftertest.

All the results are getting merged properly and I'm also getting reports. But if I click on js fileName I could not see the source code. It is just displaying the name of file.

I have copied 6-files mentioned by you from JSCoverage source distribution. Am I missing something here?

[Ed]

Thanks Ed. Actually lot of my test cases do not run with *firefoxproxy option. Anyways I have added merge logic to my code which gets executed in @aftertest.

Yes, that's the other way to do it, to merge results from different tests.

All the results are getting merged properly and I'm also getting reports. But if I click on js fileName I could not see the source code. It is just displaying the name of file.

I have copied 6-files mentioned by you from JSCoverage source distribution. Am I missing something here?

Are you getting any error message from your browser when you click on a .js file name? (e.g., in Firefox, under Tools -> Error Console)

[Amit]

Yes. I'm getting error in console as for json file as "not well formed".

But I get this error when I open jscoverage.html file itself. Then I deleted this error and clicked on file name link. But this error is not reported again in Error console.

I have re-checked my json contents for me they are well formed. Here is example.
{"img/images.css.js":{"coverage":[null,2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1]}}

[Ed]

The jscoverage.json file should have 2 components for each .js file: coverage data and source code. Basically the .json file should look like this:

Code: Select all

{
  "file1.js": {
    "coverage": [null,1,1],
    "source": ["var a = 1;", "var b = 2;"]
  },
  "file2.js": {
    "coverage": [null,0,0],
    "source": ["var x = 1;", "var y = 2;"]
  }
}

Your .json file has the coverage data, but no source code.

[Amit]

Actually when I tried to generate coverage report by firing jscoverage_report() function from firebug, I got report which has exactly same json file format as I have entered in my previous comment.

The only difference is that filename has following format...

http://serve_name/js/Startup2_all.js?v=100625130026

I do not see any source code line in this json file but it displays me contents for source code.

Is it absolute necessary to have source code array along with coverage array OR can I reference the source code without adding source code?

Regards,
Amit