Lightning

Use cases

JMeter is a great non-functional testing tool. However, on its own, it struggles in two particular areas: automatic analysis of aggregated performance characteristics and integration with CI/CD servers.

Lightning integrates JMeter non-functional tests with your CI/CD server. It analyses results of your JMeter tests and provides your server with unequivocal information allowing it to make autonomous decision whether to pass or fail the build - with no human involvement needed.

Lightning is available as a standalone JAR, AWS Lambda function, Gradle and Maven plugins. All versions accept same input and provide the same functionality. Format of the input and output will vary. You can use the one you prefer.

Why Lightning

Benefits of using Lightning include:

Lightning allows you to make a decision on performance build results, based on very broad range of client-side statistics. This is much more than just average response times. You can measure: throughput, median response time, maximum response time, n-th percentile response time, response time standard deviation and percentage of failed transactions. Learn more about available test types.
There is no need to configure anything on CI/CD server to run the above tests. You can have your first Lightning test running in minutes.
Lightning understands you may have different SLA for different transactions types and allows you to define different thresholds for every transaction type.
Lightning is not only about client-side statistics. It also gives you ability to measure server-side metrics (CPU, memory, network usage etc) and fail the build if certain metrics go above defined threshold. Or below. Your JMeter tests are no longer pure performance tests, they are now powerful non-functional tests. This lets you test load balancing and other non-functional characteristics of your system.
No disk space considerations - there is no need for extremely verbose JMeter results in XML format. Lightning processes JMeter results in CSV format with just a few columns.
No security concerns - your data is not uploaded to any 3rd party servers.
Lightning integrates seamlessly with CI/CD servers using "language" which they understand - exit code and JUnit XML report.
Lightning is open source and free for commercial and non-commercial use.
If you use Jenkins, you can set the build name to something more meaningful than the defaults. Build name can now include number of passed and failed tests.
If you use TeamCity, you will benefit from build statistics charts.
If all you need is number of executed an failed JMeter transactions, Lightning can check that for you as well.

Issues

Raise issues using GitHub issue tracker and find more information.

License

Lightning is released under the MIT license.

Our friends