Test code coverage without source code?

Question

What tools are out there that can perform code coverage analysis at the machine code level rather than the source code level? I'm looking for a possible solution to perform fuzz testing on software that I do not have source code access.

Answer 1

I think the IBM Rational test coverage tools instrument object code.

Assuming you had such a tool, but no access to the source, what exactly would code coverage mean, other than 100%?

If you didn't have 100% coverage, you'd know you hadn't exercised something. But you would have no way of knowing what.

Answer 2

If you have the number of entry points (public methods), you can test the coverage for that. I don't know any tool for that though. Otherwise you would have to test the assembly code coverage, and I don't know if it is possible.

Answer 3

For compiled code (not Java), try Valgrind.

Answer 4

Old post... but my two cents.

If you have a bunch of jars and if you know what classes/methods you are using, you can instrument the jars with Emma and run your sample application against those jars.

In my case, I have jars which are actually proprietary components (to generate html code) which our company uses to build it's web-pages. We have a sample application that utilizes these components and a bunch of tests that are run against the sample app. I wrote an ant task to copy the maven dependencies to a directory, instrument them and run the tests against these instrumented jars. This task is invoked from the maven POM and is hence part of the build process.

Also, as part of the build process, we process the emma coverage data to produce a report. This report shows the classes and methods in the jar for which we do not have the source code! Hope this helps.