Performance Test Report

Overview

Each row in the Overview tab displays the data for one specific request. Its name is derived from the action name and may have been modified using request filter and transformation rules.

When hovering over the request name, a pop-up will show how many distinct URLs have been summarized under this name and will list up to 10 examplary URLs. Note that any fragment part (the "hash") has been stripped from the URL before calculating that data as the fragment part is not transferred to the server anyway.

The Count column of the table shows the total number of executions (Total), the calculated executions per second (1/s) and per minute (1/min) as well as projections or calculations of the executions per hour (1/h) and per day (1/d).

The Errors column shows the total number of errors that occurred during the loading of a request (Total). The error count does not include errors detected during the post validation of the received data. Typical error situations are HTTP response codes indicating server-side errors (5xx), timeouts, or connection resets.

The Runtime column shows the median and the arithmetic mean, the minimum and maximum runtime encountered, and the standard deviation of all data within that series. The Runtime Segmentation shows several runtime segments and the number of requests within the segment’s definition.

If the runtime of the test case is shorter than a displayed time period, for instance, if the test runtime was 30 minutes and the time period is given in hours, then the numbers are a linear projection. That means they show a possible outcome of a longer test run, assuming load and application behavior remain the same.

The Runtime Segmentation column helps you evaluate SLA and test success definitions. You can change the setup in the report generator properties.

Bandwidth

The Bandwidth tab shows the incoming and outgoing bandwidth per request. XLT measures on socket level. Thus, actual incoming and outgoing data is recorded. XLT does not analyze or modify that data when taking the measurements.

The Bytes Sent column comprises all data sent out of the application including overhead such as HTTP(S) headers and SSL protocol data. The Bytes Received column includes all received data and the connected overhead. As the data is measured right above the socket level and before it gets decoded by the application, any compressed traffic is measured as seen by the network layer. The data size you see is not the expanded data as seen by the DOM parser.

Network Timing

The Network Timing tab displays all low-level network timing data that has been measured on socket level. Each measurement point includes the minimum and maximum times and the mean of all gathered data. The following measurement points are shown:

• Connect Time: Time needed to establish a connection to the other system. Note that when you use keep-alive semantics during testing, the connect time will mainly be 0 except for the first request of a transaction.
• Send Time: Time needed to send the request to the other system. Depending on the payload and the network speed, this data often amounts to zero or very small values.
• Server Busy Time: Time spent waiting from sending the last bytes to receiving the first bytes.
• Receive Time: Time spent receiving data from the first to the last bytes received.
• Time to First Bytes: Total time from the connection start until the first bytes are received. Includes Connect, Send, Server Busy, and Receive Time.
• Time to Last Bytes: Total time from the connection start until the last bytes are received. This is the time needed to connect, send, and receive data. Often it is called network runtime. In contrast, the request runtime includes the network runtime and the application time needed to process header and protocol information and transfer the data from socket level to the application level.

Charts

The Summary charts comprise all data across requests to allow for a quick overview on request problems and performance. The tabs in this chart section contain single charts that refer to one request.

In the following, all chart sections below visualize this information:

• Overview: moving average and individual requests
• Averages: median, mean, and moving average only
• Counts/s: number of finished requests per second
• Distribution: runtime segmentation
• Network: incoming traffic and moving average

Please note that the charts display compressed information which means that any kind of information has been mapped to chart pixels. Such a pixel can stand for any time period, depending on the total test duration. Therefore, the information you see is taken from several events within that time period and signifies a certain value range, defined by the minimum and the maximum value measured.

If the upper border of a chart is highlighted, the chart display has been capped.

*) Numbers may be projected

**) Numbers are estimated using HyperLogLog and can be off by up to 0.5%, but only for distinct counts larger than 100,000. A difference of up to 2% can occur for distinct counts larger than 1,000,000.