Performance Benchmarking
A key consideration for integration architects involved in planning and deploying messaging-based systems is the risk and cost associated with system performance. Unfortunately, the runtime performance and bottlenecks of a future production system are not easily simulated in the lab, but carefully planned benchmark testing should be able to validate whether the proposed solution will scale and perform appropriately, given hardware plans. This information can substantially reduce operational risk or, at least, provide early warning that alternative solutions may be required.
Before you make the considerable investment of running a performance benchmark, you should be clear on the purpose of the testing. This may be one or more of the following:
- Planning future system capacity needs
- Comparing the efficiency of various integration design options
- Comparing the cost/benefit of enterprise messaging providers
This purpose will guide the decisions and priorities of the performance evaluation project and will help keep the testing and analysis focused as the project proceeds.
Equally important is to ensure that the tests you run are a realistic assessment of the value of each alternative in terms of the performance, scalability and costs you can actually expect to encounter. There are several areas that are particularly important in ensuring the value of performance tests:
- Mimicking current or future system loads
- Simulating the topology of the messaging infrastructure, including endpoints, routing paths and services
- Avoiding artificial or unnecessary constraints on the potential implementation
- Prioritizing those components and requirements that have a high impact on performance
The paper, Benchmarking Enterprise Messaging Systems, discusses planning, building and executing a performance benchmark to meet your goals.
In addition to the whitepaper, our Test Harness (zip) includes source code and documentation for a comprehensive array of point-to-point and pub / sub benchmarks. The Test Harness includes a simple Java utility for measuring variations in disk speed across different hardware platforms.
You may also be interested in
