XLT – Quick Start Guide

Introduction

About this document

This document gives you a short overview of XLT itself, its features, and how it can be used to perform regression and load tests in a comfortable, platform-independent approach. It describes the basic steps of writing web tests, running the tests, and finally rolling it out to perform a load test.

To get the most out of this document and to be able to work effectively with XLT, you should have some basic knowledge about web technologies, the Java programming language, and JUnit4. You definitely don’t need to be an expert but being familiar with these topics will help you better understand the next sections.

What is XLT?

XLT stands for Xceptance LoadTest. XLT offers a comfortable way to write and run regression and load tests for web applications. Nearly every software that provides access via HTTP/HTML can be tested using XLT. There is also extensive JavaScript support for testing applications that use Web 2.0 technologies. XLT supports not only web testing but also SQL tests, RCP-based application tests or any other test that should run as a load or performance test in a distributed manner. Since XLT is written in Java it can be run on any platform that supports Java.

XLT Script Developer

The easiest way to start is with the XLT Script Developer. It operates as a Firefox add-on and provides an easy-to-use user interface for developing and running test cases and entire test suites. XLT Script Developer can record the page flow while navigating through a website and features a wide range of validations on the web page.

XLT Framework

Scripting test cases using the Script Developer is easy, but keep in mind that you are limited by the strictly linear approach and the set of available commands.

The XLT Framework provides different approaches for writing test cases in Java. Starting with Java code automatically generated from the recorded Script Developer test cases, you can extend your test suite by using the XLT API. This features a higher level command scripting API with a very intuitive syntax as well as the WebDriver API and also the lower level XLT Action API based on HtmlUnit.

XLT API translates all of your test scenarios into JUnit4 tests. The principles of JUnit4 and its annotations are used to implement and tag test cases. This way, each XLT test is a JUnit test, so XLT tests can be executed just like any other unit test within an existing build process.

Installation Instructions

System Requirements

Hardware

Typically, XLT requires a CPU running at 1.5 GHz at least and about 1 GB of RAM. XLT can run on slower hardware but this depends on the tests themselves. The default installation requires about 85 MB of hard disk space, but load test results might require more free space, so at least 1 GB of free disk space is recommended.

Software

XLT runs on any operating system for which a Java 6 (or higher) virtual machine is available, such as Microsoft Windows, Linux, Oracle Solaris, HP-UX, or Mac OS X. We recommend using Oracle JDK if it is available for your platform. XLT will also run in JVMs provided by other vendors (e.g. BEA, HP, IBM), but this has not been tested extensively. In order to view the HTML load test reports generated by XLT, a modern web browser is required, such as Firefox 3 or 4, Chrome, Internet Explorer 8, Opera 10 or Safari 4. Please note that JavaScript needs to be enabled to fully utilize all functionality. If you plan to use the XLT Script Developer, you will need Firefox 3.6 or Firefox 4.

Installing XLT

The installation of XLT is simple. Just unzip the XLT archive to a file system location of your choice. The root directory is part of the archive, so you do not need to create it separately. Please note that although spaces in the path are supported by XLT, it is easier to code tests when the path is free of them.

On Unix-like systems, you need to grant execution rights to all files in the bin directory that end in .sh:

chmod u+x <XLT>/bin/*.sh

Next, copy the license XML file, which you should have received separately, to the directory <XLT>/config. Note that a Basic License of XLT does not require the installation of a license file. In this case, XLT is not limited in time or functionality, but the number of virtual users is restricted to five. Also notice that in this case the Basic License Terms apply. See the file <XLT>/doc/license.html for more information.

Finally, make sure that the executable directory of your Java installation is listed in your PATH environment variable so that the XLT start scripts can find the virtual machine runtime.

In order to install the XLT Script Developer, an extension to the Firefox browser, you need to carry out the following steps:

  1. Start Firefox.
  2. Click File > Open File....
  3. Navigate to the <XLT>/tools directory and select the .xpi file. The Add-On installation dialog should appear.
  4. Click Install to finish the installation of the Script Developer extension.

Alternatively, drag the .xpi file and drop it onto the Firefox window to install it.

Uninstalling XLT

Before uninstalling, make sure that you have backed up all the test results and test reports that you want to save. To uninstall XLT simply delete its installation directory. If you installed the Script Developer Firefox extension, use the Firefox Add-On dialog to remove the extension.

XLT Script Developer

For more detailed information on the Script Developer, please read the full XLT User Manual.

Introduction

XLT Script Developer is used to create script test case. They are based on a simple syntax and a reduced set of operations, which makes them a perfect fit for non-programmers. No other tool is necessary to create, edit, and manage basic script test cases, except the Script Developer, which is an extension to Firefox.

Script Developer is a recording tool. That means the tester simply uses the application under test. All interactions with the application are recorded in the background and stored to an XML script file. While recording, you can also perform validations as well as using appropriate assert commands. Any recorded value can later be extracted out of the script into a test data file to separate test data from script code. Scripts may also be exported as ordinary Java code.

Via the Script Developer, script files can be replayed in Firefox at any time to quickly check whether the test case still runs successfully.

XLT Script Developer XLT Script Developer

Basic Script Developer Settings

Before you can start, make sure the Script Developer Firefox extension is installed correctly. Open the Script Developer window, either via the Firefox Tools menu or via clicking the XLT icon in the status bar.

Next you need to tell the Script Developer where your test project is located on disk. In the Script Developer window, open the configuration dialog via clicking XLT Script Developer|Settings and configure the settings appropriately:

Getting Familiar With The Script Developer

The Script Developer main window is divided into two parts: the script library and the work area.

On the left-hand side of the window you can find the script library containing all test cases and modules existing in your test suite.

Right click on the left hand side of the window to show the context menu, which you can use to manage the scripts. This includes creating and deleting scripts as well as renaming them or managing meta data.

In order to edit the commands contained inside a test case script, you need to open the script inside the script editor tab by double-clicking it. The script editor lists all commands contained in a script. Use the context menu to manipulate the command list.

Finally, there is a tool bar with the usual functionality, like Reload and Save, but also with the record and play-back icons.

Creating Test Scripts

Recording

The easiest and preferred way to create test scripts is to record the steps a human user takes to interact with a web application.

  1. Open the web page you want to start with in a Firefox tab (and make sure this tab remains the active tab, i.e. the foreground tab).
  2. Switch to the Script Developer and create a new test case via the script library context menu. Provide a meaningful name. As a result, an empty script editor tab opens.
  3. Click the Start Recording icon in the tool bar to start recording.
  4. Switch back to your web page and start using it. All your interactions with the page will be recorded.
  5. You can also validate the correctness of page by using assert commands.
    • Open the Firefox context menu and choose XLT Script Developer (available only while recording). A sub menu opens.
    • Choose an appropriate assertion from the sub menu:
      • assertTextPresent - check that the selected text appears on the page (you have to select the respective text before)
      • assertTitle - check that the page title matches the current page title
      • assertElementPresent - check that an element is present on the page (you have to open the Firefox context menu directly on the element in question)
      • assertText - check that an element is present on the page and that the element has a certain text (you have to select the respective text before)
      • and many more ....
    • For each assertion, there is also a version that checks whether the respective condition is not true. Note that you can record as many assertions as you like, one after the other.
  6. Continue interacting with the web page as defined by your test scenario.
  7. Once you are done with the test scenario, switch back to the Script Developer and stop recording by clicking the Stop Recording icon.
  8. Do not forget to save the new script by clicking the Save icon or using the Ctrl+S shortcut.

As you may have noticed, the Script Developer also inserts so-called Actions while recording. An action is a sequence of steps that belong together. For example, filling in the inputs of a form, submitting the form by clicking the submit button, and checking the resulting page with some assertions is a typical action. Actions are primarily used to break the page flow down into reusable atomic steps and to give those steps a name. Action execution times are measured and reported while running load tests. The Script Developer names actions rather generically, but you can rename them later on.

Please note that actions are not to be used to structure your code visually since they are not comments. Furthermore, keep in mind to give your actions a meaningful name (e.g. Search , Browse or AddToCart).

Executing

Once you are finished with a test script, it can be executed inside the browser.

  1. Open the respective test case script in a script tab or activate the tab if it is already open.
  2. Click the Play tool bar icon.

Inside browser tab, elements highlighted in orange shows to the assert and highlighted in yellow refers to normal web page flow commands being executed respectively. Inside script editor tab, the commands are marked with a special status icon. As long as the command (or module) is executing, the status icon will be yellow. Once the command was finished, the icon turns either green in case of success, or red in case of an error. Note that script execution stops immediately if a command fails.

You can adjust the replay speed using the slider control. Sometimes it may be hard for you to recognize what the script is doing. In this case, reduce the replay speed or if you do not want to follow the interaction but want to have the result as quick as possible, you may increase it. Note that the actual commands are not executed slower or faster. The slider influences only the time the elements are highlighted.

While executing a test script most commonly used tool bar controls and context menu options are:

Editing

Test scripts can be modified at any time. Often enough, this becomes necessary to fix mistakes made during recording the test scenario. It is too easy to click the wrong link or to forget to add an assertion. In the first case, just go back and continue with your test scenario. The unwanted steps can be deleted later. In the second case, manually add the appropriate assertion commands after recording has finished. You might modify a script even while recording. Switch to the Script Developer window, make the necessary changes, return to the web page and continue recording.

The following changes can be made, all accessible via the context menu in the script editor tab (item = command or module or action):

To change the properties of a test script (e.g. its description) right-click the script in the library and choose Edit Details.

To make a call to an existing module, you simply insert the module into your test script. To do so, put the cursor at the right position and choose Insert | Module from the context menu. A dialog will pop up where you can choose the module to call and specify the parameter values to pass to the module. After this, the module call is embedded into your script and can be treated/edited as any other command.

Performing Load Tests

For more detailed information on Load Tests, please read the full XLT User Manual.

Typically, a distributed load generation environment is needed to generate enough load. For this we need a cluster of test machines. Install XLT on all these load machines.

Load generation environment Load generation environment

Load Test Environment Configuration

Before you can start the load test, some configuration work needs to be done. The configuration of XLT load generation environment itself is discussed in the next section, followed by an explanation of the configuration of your test suite.

The below mentioned property files are used to configure the main components of the XLT load generation runtime:

Agent Controller Configuration

Inside agent controller configuration file you can define the following properties.

Port Number

Port number, the agent controller is listening on. Default is 8500. You can pick any free port number, but make sure that the corresponding master controller entry matches that number. Also ensure, that the firewall rules in place allow unrestricted communication. The used protocol is https. If you would like to run more than one agent controller per machine, make sure that all controllers use different port numbers.

com.xceptance.xlt.agentcontroller.port = <portnumber>
Key Store Credentials

The credentials, your key store is encrypted with. You only have to change this, if your Java key store password has been changed from the default.

com.xceptance.xlt.agentcontroller.keystore.password = <password>
com.xceptance.xlt.agentcontroller.keystore.key.password = <password>
Agent Controller Logging

For the configuration of the agent controller logging facility, you can adjust these properties. These settings only affect the agent controller output and do not change the logging of your test code. Most of the time, a change here is not required.

log4j.rootLogger = info, console, file
log4j.appender.console = org.apache.log4j.ConsoleAppender
log4j.appender.console.layout = org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern = [%d{HH:mm:ss,SSS}] %-5p [%t] - %m%n ..............

For more information about log4j settings also see Apache Log4j API Docs

Master Controller Configuration

The second configuration file contains the properties of the master controller.

Test Suite Location

To determine which test suite should be used for the load test, you have to specify the location of the test suite relative to your XLT installation. It will be uploaded to the agent controllers from there.

com.xceptance.xlt.mastercontroller.testSuitePath = <location>
e.g. com.xceptance.xlt.mastercontroller.testSuitePath = samples/testsuite-pebble

When running on and from Windows, make sure that you use the correct encoding for backslashes, because property file format uses \ to quote other special characters, so you have to quote the \ with a \ to ensure its original meaning, e.g. c:\\test\\mysuite.

Depending on the operating system you are using, it can be necessary to specify an absolute path to the test suite instead of a relative one.

Update Interval

Defines how often the master controller updates the status of currently running load test.

com.xceptance.xlt.mastercontroller.ui.status.updateInterval = <time in seconds>
Status Display

Whether to display detailed status information for each simulated test user or not. If set to false, status information will be aggregated into one line per user type and vice versa. If you have a lot of test users running, it can be helpful to set this to false, otherwise you might become overwhelmed by the amount of information presented. This property will not change the data collection and final data presentation. This is a display property only.

com.xceptance.xlt.mastercontroller.ui.status.detailedList = <true/false>
Agent Controller Locations

This property lists the locations of the agent controllers, the master controller should use. 

com.xceptance.xlt.mastercontroller.agentcontrollers.<id>.url = <url> 
com.xceptance.xlt.mastercontroller.agentcontrollers.<id>.weight = <weight>

You can use any name for the <id> part of the property. Recommended are name and number combinations, such as ac1 for the first agent controller or blade01-02 for the second agent controller on the first blade. Make sure that the agent controller IDs differ from each other, otherwise a later entry in the file will overwrite the previous one.

In order to use load machines of different power together in a load cluster, you may specify a “weight” for each agent controller (defaults to 1 if not set). This value influences the automatic distribution of virtual users across the load machines. A machine with a weight of 3 will get 3 times the load of a machine with a weight of 1. 

com.xceptance.xlt.mastercontroller.agentcontrollers.ac1.url = https://localhost:8500
com.xceptance.xlt.mastercontroller.agentcontrollers.ac1.weight = 1
com.xceptance.xlt.mastercontroller.agentcontrollers.ac2.url = https://localhost:8501
com.xceptance.xlt.mastercontroller.agentcontrollers.ac2.weight = 3
Master Controller Logging

You can set a different logging behavior for the master controller. This can help to solve problems and also provides information in case of support inquiries.

log4j.rootLogger = debug, file
log4j.appender.console = org.apache.log4j.ConsoleAppender
log4j.appender.console.layout = org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern = [%d{HH:mm:ss,SSS}] %-5p [%t] - %m%n ......

Test Suite Configuration

The test suite itself is configured independent of the master controller. All properties are read from the <test-suite>/config directory. All files and most important properties are explained in detail in the Test Suite and Framework Configuration section of this document.

This section will discuss only the settings relevant to load testing.

Default Configuration – default.properties

Result directory location

Specifies the directory location where you want to store load test results. Normally it is not necessary to change this. 

com.xceptance.xlt.result-dir = <directory path>
Error Behavior

Specifies the framework behavior in case of an error – whether the framework should abort a transaction if any of the following error occurs:

com.xceptance.xlt.stopTestOnHttpErrors.page = <true/false>
com.xceptance.xlt.stopTestOnHttpErrors.embedded = <true/false>
com.xceptance.xlt.stopTestOnJavaScriptErrors = <true/false>
com.xceptance.xlt.maxErrors = <number of errors per agent controller>
Think Times

To specify the think time between two subsequent actions or transactions, use these properties. If a random think time is needed, set the deviation to a value greater than 0. It specifies the maximum deviation from think time in milliseconds. The respective value is added or subtracted from think time is generated with a pseudo-random, uniform distribution.

com.xceptance.xlt.thinktime.action = <time in [ms]>
com.xceptance.xlt.thinktime.action.deviation = <time in [ms]>
com.xceptance.xlt.thinktime.transaction = <time in [ms]>
com.xceptance.xlt.thinktime.transaction.deviation = <time in [ms]>

For example :

com.xceptance.xlt.thinktime.action = 100
com.xceptance.xlt.thinktime.action.deviation = 50
com.xceptance.xlt.thinktime.transaction = 0
com.xceptance.xlt.thinktime.transaction.deviation = 0

This will set the action think times between 50 and 150 ms and no transaction think time at all.

The deviation has to be smaller than the specified base think time.

Test Project Configuration – project.properties

To configure your test project you’ll have to edit the file named project.properties.

Test Properties File

XLT permits to prepare and use multiple test.properties files for easy maintenance of test setups. This makes switching between test setups easier and avoids configuration errors. This property does not allow the use of a path-specific file name. The test definition files reside in the same directory as the project.properties file. 

com.xceptance.xlt.testPropertiesFile = <filename>.properties
Test Class Mapping

test case mapping – specify Java classes mapped to the appropriate test IDs (fully qualified names). Note you can map the same class to multiple load test names if needed.

com.xceptance.xlt.loadtests.<name>.class = <fully qualified class name>

For example:

com.xceptance.xlt.loadtests.TVisitor.class = com.xceptance.xlt.samples.tests.TVisitor
com.xceptance.xlt.loadtests.TJSVisitor.class = com.xceptance.xlt.samples.tests.TJSVisitor
Test Class Specific Settings

Project specific settings – you can also define any test case specific settings as well, using the following syntax. 

<fully qualified name>.<property-name> = <value>

For example:

com.xceptance.xlt.samples.tests.blog-url = http://localhost:8080/pebble/
com.xceptance.xlt.samples.tests.TAuthor.username = username
com.xceptance.xlt.samples.tests.TAuthor.password = password
com.xceptance.xlt.samples.tests.webdriver.TAuthor.write-count = 2

Load Test Profile Configuration – test.properties

Test-run-specific settings – you can also configure an (optional) property file which contains the settings specific to a certain load test run. You can define more than one test property file, such as test-target-load.properties and test-2x-target-load. This way, many configurations can be defined and prepared in advance and used as needed. You switch between these files by changing com.xceptance.xlt.testPropertiesFile in the project.properties file.

Load test profile configurations are done inside test.properties file, where you define the test case name, number of virtual users and all other load test specific settings, which are to be run in parallel agents using the following syntax. 

com.xceptance.xlt.loadtests.<testID>.<setting> = <value>

For <testID> use any proper name. The supported values for <setting> are:

For example:

com.xceptance.xlt.loadtests = TAuthor
com.xceptance.xlt.loadtests.TAuthor.users = 5
com.xceptance.xlt.loadtests.TAuthor.iterations = 100
com.xceptance.xlt.loadtests.TAuthor.arrivalRate = 3600
com.xceptance.xlt.loadtests.TAuthor.initialDelay = 0
com.xceptance.xlt.loadtests.TAuthor.warmUpPeriod = 30s
com.xceptance.xlt.loadtests.TAuthor.measurementPeriod = 10m 0s

All time period values can be specified in one of the following formats (without the quotes):

If you want to run several test cases simultaneously, add the test case names to com.xceptance.xlt.loadtests separated by space:

For example:

com.xceptance.xlt.loadtests = TAuthor TVisitor TCrawler
com.xceptance.xlt.loadtests.TAuthor.users = 5
com.xceptance.xlt.loadtests.TVisitor.users = 3
com.xceptance.xlt.loadtests.TCrawler.users = 4 
.........

The following figure shows the relationship between the various load test profile settings.

Load Test Profile Configuration Load Test Profile Configuration

A load test starts by generating a constant load based on the number of concurrent users where the maximum load is limited by the number of users. If an “arrivalRate” is given the system generates load with a cluster-wide constant arrival rate. Note that an arrival rate is only supported for non-iteration based load tests.

In order to minimize discrepancies that could be caused by applications and other systems starting up and not yet operating at an optimal level, an initialDelay and warmUpPeriod can be defined which is the time we will give the load to level out before taking measurements.

The “rampUpPeriod” setting dictates the time that it will take for the load to increase to 100% after the initial delay. If you run an arrival rate test, this will only lift the ceiling of the users permitted to run. Due to the nature of the arrival rate test, this might not necessarily mean that the users will run at this point in time.

The test will be measured during the “measurementPeriod”. To ensure a predictable load until the very end of the measurement period, a shutdownPeriod can be set in which the users will continue to run but the measurements have been stopped. This permits a fading out of the tests.

In case you want to modify the behavior of the logging facility of the load test agents, the test suite configuration directory contains a file named log4j.properties, which can be changed to meet your needs.

To launch the Java Virtual Machine that runs the agent with additional parameters, specify them in a file named jvmargs.cfg.

Run the Load Test

Running the load test consists of two steps:

  1. Running the agent controllers,
  2. Running the master controller.

Running the Agent Controllers

To start the agent controllers open a command line window/console and type the following sequence of commands:

cd <XLT>/bin
./agentcontroller.sh

Windows users have to use the appropriate .cmd file which is located in the same directory.

The agent controller will start running on the specified port. The output will look like this:

- Using "C:\Users\AppData\Local\Temp\vfs_cache" as temporary files store.
- Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
- jetty-6.1.19
- Started SslSocketConnector@0.0.0.0:8500

Running the Master Controller

Make sure all agent controllers are running on all respective load test machines before starting the master controller. The master controller cannot be started if the agent controllers are not running. Also check that the test suite has been compiled successfully to avoid errors when uploading the test suite.

Master controller can be started in different modes. Recomended mode for beginners is -auto.

Auto Mode

This mode automatically executes a typical sequence of steps to be executed when running a load test. To start XLT master controller in this operating mode, use the following command line:

Unix based systems: 

cd <XLT>/bin
./mastercontroller.sh -auto

Windows: 

cd <XLT>\bin
mastercontroller.cmd -auto

If the test suite files were uploaded and the load agents started successfully, XLT automatically refreshes the agent status regularly. Once the test has finished, the test results are downloaded and XLT master controller quits.

If the command is followed by the option -report then additionally a load test and performance report is generated automatically after downloading all results is finished 

cd <XLT>\bin
mastercontroller.cmd -auto -report

To abort the test prematurely, press CTRL-C to terminate the master controller. This terminates all running agents as well and triggers the download of all test results generated so far. This also means, that it is not possible to disconnect the master controller from the test cluster while keeping the load test running.

For long running load tests, it is recommended to run the test without the auto option, because this allows a disconnect from the test as well as inhibits accidental test termination.

Acknowledgments