Can I get statistics for test cases steps inside robot framework?

Question

In order to optimize time of execution, i create some test cases dependent from each other i want to get metrics and statistics not only for each testcase and testsuite. But also i want to generate statistics and metrics for each step. Is that possible ? PS : I'm using team city for continuous integration.

Best regards,

Emna A.

Answer 1

Using robot framework api we can get TEST and KEYWORD Metrics

Reference:

Link

API:

• class robot.result.model.Keyword
• class robot.result.model.Test

Keyword Metrics Code:

# Keyword Metrics Code: (save following snippet as python file and execute)
from robot.api import ExecutionResult,ResultVisitor

result = ExecutionResult('output.xml')
result.configure(stat_config={'suite_stat_level': 2,
                              'tag_stat_combine': 'tagANDanother'})

class KeywordMetrics(ResultVisitor):

    def visit_keyword(self,kw):
        print "Keyword Name: " + str(kw.name) 
        print "Keyword Status: " + str(kw.status)
        print "Keyword Starttime: " + str(kw.starttime)
        print "Keyword Endtime: " + " " + str(kw.endtime)
        print "Keyword Elapsedtime (Sec): " + " " + str(kw.elapsedtime/float(1000))

result.visit(KeywordMetrics())

# Note:
# visit_keyword() returns userdefined keywords
# start_keyword() returns all the keywords (library and user defined)

Test Metrics Code:

# Test Metrics Code: (save following snippet as python file and execute)
from robot.api import ExecutionResult,ResultVisitor

result = ExecutionResult('output.xml')
result.configure(stat_config={'suite_stat_level': 2,
                              'tag_stat_combine': 'tagANDanother'})

class TestMetrics(ResultVisitor):

    def visit_test(self,test):
        print "Test Name: " + str(test.name) 
        print "Test Status: " + str(test.status)
        print "Test Starttime: " + str(test.starttime)
        print "Test Endtime: " + " " + str(test.endtime)
        print "Test Elapsedtime (Sec): " + " " + str(test.elapsedtime/float(1000))

result.visit(TestMetrics())

Robot framework Metrics project is implemented to show metrics result in HTML format with dashboard view.

Highlights

• Top 10 Tests Performance Bar Graph
• Top 10 Keyword Performance Bar Graph
• Pie Charts
• Keywords and Test Metrics in Tabular format

Robot framework Metrics Report ReadMe.MD

Answer 2

There's quite a few tools that you can use to achieve this. Making your own is also relatively easy, be it via Robot Framework's listener interface, or by post-interpretation of the test outputs.

You can use this tool to post-process an XML output and get statistics about each keyword. You may also want to complement it with this tool which basically generates a complete benchmark report