Features
XLT is the product of all our long-term experience in test automation and performance testing. It also makes use of well-known technologies such as WebDriver, Apache HttpClient, and HtmlUnit.
Known Technologies
XLT utilizes what you already know: Java, JUnit4, Selenium WebDriver. Absolutely platform-independent.
Because it takes time to learn new things and because you have established tools and processes, we built XLT on well-known and tested foundations.
All this gives you the tools to get the most out of XLT. It integrates well, you can extend it well, and you won't have a hard time mastering it.
- Java is the scripting language when scripting is required. Use your favourite IDE, profilers, and debuggers.
- JUnit allows easy integration into test automation processes.
- Result data is delivered as CSV for optional post-processing.
- Reports are HTML for easy sharing.
- Reports are rendered from XML via XSL for easy adjustments of content and layout.
- Reports feature an XML data file for machine-based evaluation.
- The command line helps you automate everything quickly.
- WebDriver is used for functional testing as well as client-performance testing.
And last but not least, XLT is platform-independent. Write on Windows, run on Linux. Compose on Mac, deploy on Windows. The possibilities are endless.
Deployment & Cloud
XLT is easy to install and start, no OS dependencies whatsoever. Ready-to-use Amazon-EC2 images are available.
XLT is platform-independent and you just have to unzip it. If you need ready-to-go cloud images, we provide Amazon Web Services EC2-AMIs free of charge.
You can deploy XLT to any cloud provider and use any OS that has support for Java 7 or 8. We strongly recommend Unix server environments for running large tests.
XLT runs HTTPS-based communication. You can use proxies, authentication, and your proven tools to deploy and run XLT. Our goal was to keep it as simple as possible, to make it as flexible as possible.
Awesome Reporting
Out-of-the-box reporting that is just right. No configuration work, all numbers at a glance. Comparison and trend reporting included.
Are you tired of configuring your reports manually? Are the provided data-sets just not what you want or do they hide crucial details? We‘ve got you covered! XLT provides best-practice reports out of the box, based on over a decade of load and performance testing experience.
Load Test Reports
The XLT performance report includes everything you need to know about your application.
Besides the obvious data such as runtime totals, averages, and medians, you can find projections for hourly or daily traffic. It also features service-level-agreement numbers, lists the configuration of your test as well as all errors and problems, and includes links to the result browser for quick analysis. Open an example here.
Comparison Reports
View here. If you run a test again and you want to know if the later round was in fact better, you can generate a comparison report.
Color codes and percentages help you quickly determine what changed between the test runs.
Trend Reports
View here. A trend report shows how performance developed over many test runs. It also uses color codes to visualize the changes.
The trend report visualizes two performance trends: first, a baseline comparison showing you the performance changes compared to your first test run; and second, a comparison to the previous run telling you what happened at a specific point in time.
Instant Feedback
Realtime monitoring capabilties give you instant insights into progress and health of your running load test.
XLT realtime monitoring gives you quick and realible insights into your running tests by utilizing known technologies.
By supporting the well known and established open source tools Graphite and Grafana, XLT enables you to get all the needed insights into running tests instantly.
You can also correlate XLT data with all your infrastructure data, if you use Graphite for your monitoring already.
Easily Adjustable
Add new data sources, modify the report content and layout, build powerful test suites as frameworks for even more testing.
A good tool has to be adjustable to the environment it is used in. Thus, XLT supports many ways to be extended or modified to fit your needs even better.
Reports
All reports can be adjusted in content and layout, because we deliver the CSS and the render templates as part of XLT. So if you need custom layouts of reports, you are covered. You want to have your own summary of data? Not a problem. You want to reduce the data? Fine as well.
Custom and External Data
During a load test, XLT logs a large amount of data relevant to the test run. Nevertheless, sometimes it comes in handy to log additional information about the system under test. For this purpose, XLT provides custom values.
The XLT report generator lets you include external data files in the report. This can be used if it’s impossible to directly access the external data source during the load test runtime.
Use Java
If there is anything that is not part of XLT, you can use Java to make it happen. Of course there are limits, because you can influence the measurements or limit the scaling, but with that in mind, a lot of things are possible.
Made for Automation
Java as script language, open data formats, command lines, JUnit test principle, WebDriver support, and much more.
XLT is made to be integrated into all kinds of automation processes. No matter if it‘s just a shell script or a full build process.
We built XLT in a way that perfectly gets along with your preferred IDE, the command line, build processes, or just with a little bit of shell scripting. Engineers need things to work smoothly and quickly. We have that in mind.
Load Profiles
Arrival rate or steady user rate, ramp-ups, and flexible profiles over the course of a test run. Realistic or predictable, your choice.
XLT supports a very flexible setup of load profiles ranging from steady user rates to arrival rates. Compose long-running test setups to reflect what happens during a day, absolutely freely configurable.
Load Profile Features
- Arrival rate model for realistic simulations.
- Steady user rate for predictable measurements.
- Ramp-up for gradually increasing traffic.
- Free configuration to compile challenging tests.
Freely Configurable
Do you need to have a test where, after a slow ramp-up of low traffic, the load jumps within a minute to peak traffic, stays on that level for 10 minutes to then come back in waves? Well, XLT lets you do so. No limits. It works for arrival rate-based tests as well as for user-based tests.
User Scenarios
Unlimited number of user scenarios that can be part of a test. Each scenario can have a different configuration or load profile.
Reality is versatile and your test should reflect that. You can set up as many different scenarios as you need and run them together.
Each scenario can be everything from a single pageview to a long-running call center operation. You can give each scenario an individual configuration that includes the load profile. The start of scenarios can be delayed in relation to other scenarios if needed.
See Everything
The result browser precisely shows you what happens in a scenario. It visualizes any failure situation for a post-mortem analysis as well.
The result browser is your tool to visually check the loaded pages and debug problems during test execution.
During script development, the result browser shows you all details about your recently executed script, so you can make sure that you covered everything and that the requests match the real traffic.
When executing WebDriver tests, the right side displays the web page as a screenshot. When executing with XLT's built-in HtmlUnit, the right side is an estimated rendering of the downloaded data.
Imagine a long-running test and you see three errors during the execution but thousands of transactions went just fine. XLT provides deep insights into the moments when the application misbehaves.
When errors occur during load testing, the last moments of the transaction are saved, including the downloaded data and all technical details. You can thus see what was sent and received. When you use the request-ID feature of XLT, you even get a unique ID you can match against your server logs.
Verify Everything
Assertions permit you to verify everything in detail to ensure that your test is valid.
A test is only as good as its verifications. XLT supports and encourages as much validation as possible to really make sure that the pages loaded match the expectations.
You can use the full syntax of assertions known from JUnit and Selenium. If you like Hamcrest, include it, XLT gives you that flexibility. XLT also provides access to all sub-requests of a page, so you can run even more detailed validations. The XLT Action API allows you to include validations before and after each activity.
Scalable
Run load tests from any number of machines, different locations, and with nearly unlimited users.
You can utilize an unlimited number of machines for your load testing. Scale up across datacenters easily. XLT orchestrates the load distribution, uploads, and downloads on its own.
Scale up XLT per machine by bringing up more processes for perfect hardware utilization. Scale up across machines easily by adding more locations. XLT uses differential and concurrent up- and downloads for maximum speed.
The load is automatically distributed across all agents. If you use heterogenous hardware, you can attach a weight to each machine to indicate a different speed or size.
Installing XLT is easy and only takes a few minutes. It leverages HTTPS and authentication for protecting your agents. Bind XLT to any port you wish.
Preconfigured Amazon Web Service Images (AMIs) get you started quickly on Amazon EC2. Just start it and you are good to go.
Configurable
Set timeouts, keep-alive, compression, image loading, permitted hosts, thinktimes, error handling, and much more.
To match the behavior with your testing needs, lots of parameters can be adjusted.
XLT offers a richt set of options to modify the behavior so that all kinds of different configurations can be tested. For example:
- Keep-alive
- Timeouts
- Content compression
- Header fields
- Thinktimes
- Content loading
- Error handling
- Maximum error count
- Permitted hosts
- User agent
If XLT does not provide a property for a certain parameter, it can also be influenced programmatically.
Archivable
Open formats for reports and results to archive them. Go back later for reanalysis. Test configurations are stored with the reports.
XLT uses open formats for results and reports. Perfectly suited for storing them anywhere.
Result data and reports are separated so you can easily make a decision if you want to store a lot of data or just the final outcome. Stored result data can be turned into reports anytime again on any machine.
Reports and results store your full test configuration. Repeating the exact same run days or weeks later is a matter of seconds.
Shareable
Test suites and configurations can be shared easily. Reports are plain HTML and perfect for quick sharing.