Reporting

#Reporting

After tests execution Jagger generates report. Reports could be generated in pdf(recommended) and html formats. Output format could be set using properties:

chassis.master.reporting.report.format=PDF
chassis.master.reporting.report.file.name=report.pdf

Report sections description

• Session summary
  • Session id. Unique identifier that could be used for session comparison.
  • Session's start and end time
  • Session's duration
  • Number of tasks
  • Number of tasks failures
  • Number of kernels that executed tasks.
• Session comparison. That section compares current section with chassis.engine.e1.reporting.session.comparison.baseline.session.id. Note: only tests with same termination strategy could be compared. If chassis.engine.e1.reporting.session.comparison.baseline.session.id=#IDENTITY or isn't present in environment.properties than session comparison section is skipped.
• Session status section is a summary of executed tests statuses. If one or more tests have failed invocations that section will have "fail indication".
• Tests summary table contains information about tests execution
  • Scenario name
  • Clock - count of threads in what test was executed.
  • Throughput (TPS) - number of executions per second.
• Details section
  • Version - manually edited field for tasks that tells if tests could be compared.
  • Samples - number of times test was executed.
  • Termination strategy - indicates what termination strategy was used for test (time or samples count).
  • Duration - tests duration
  • Average Latency(sec) - average invocation duration
  • StdDev Latency (sec) - latency's deviation
  • Number of failures
  • Success Rate - successInvocationsCount/totalInvocationsCount
• Scalability Plots - section contains throughput/thread count, latency/thread count plots.
• Test Details - contains more detailed plots of throughput/time, latency/time, total sys

Decision makers

For specifying rules for session comparison and session status treating

com.griddynamics.jagger.engine.e1.reporting.DefaultSessionComparisonDecisionMaker
com.griddynamics.jagger.engine.e1.reporting.DefaultSessionStatusDecisionMaker

could be used. If default strategy and strategy that could be achieved by spring configuration editing

    <bean id="sessionComparisonDecisionMaker"
          class="com.griddynamics.jagger.engine.e1.reporting.DefaultSessionComparisonDecisionMaker">
        <property name="warningDeviationThreshold"
                  value="${chassis.engine.e1.reporting.session.comparison.decision.maker.warning.deviation.threshold}"/>
        <property name="fatalDeviationThreshold"
                  value="${chassis.engine.e1.reporting.session.comparison.decision.maker.fatal.deviation.threshold}"/>
        <property name="description"
                  value="Status is based on throughput. Session status is a status of the worst test."/>
    </bean>
 
    <bean id="sessionStatusDecisionMaker"
          class="com.griddynamics.jagger.engine.e1.reporting.DefaultSessionStatusDecisionMaker">
        <property name="description"
                  value="Status is based on success rate. Session status is a status of the worst test."/>
    </bean>

is not enough than com.griddynamics.jagger.engine.e1.reporting.DecisionMaker should be implemented.

Configuring plots list

It's possible to configure plots list that should be present in final report. For more information see JAGGER_HOME/chassis/configuration/configuration/reporter/monitoring.reporting.conf.xml Bean name is monitoringPlotGroups