A compare tool: Download and analyze the runtime performance result from SAT

I once participated one project which needs to do bench-marking about how much performance gain we can benefit after we move the whole CRM system on top of HANA.

we plan to do it via steps below：

1. Run one test report via SAT on both HANA based and non-HANA based system

2. export trace result( xml format ) of HANA system locally, and import it into non-HANA system, and do comparison in SAT.

Soon we find our plan is only feasible and could never be put into practice:

1. The size of the trace file generated by SAT is huge. One trace with 1.9 second would lead to a trace file with size = 147MB.

In several of our scenarios we have execution time with 20 seconds, which leads to an xml file with 1GB. When the runtime increases, the SAT export function does not work – an memory consumption runtime exception occurred.

2. We only care about the performance of the Function module CRM_PRODUCT_GETLIST2 which we are responsible for. For some reasons we could not directly call this FM in our test report, but have to use BOL query instead. As a result, lots of executions deep in BOL – Genil callstacks are also traced, which we are not interested at all. This is actually a pain-point of SAT export function – it does not allow you to only export part of trace result which you are really interested with.

3. The compare functionality in SAT does not work for me. I always got the following runtime error.

So I developed a tool for our project, which eliminates the pain pointed mentioned above:

1. it could allow the user to only download the part of the trace which they are interested, which greatly reduces the trace file size. For example, from 147 MB to 84KB.

2. I implemented the compare functionality myself.

The tool is very easy to use:

(1) trace your application as usual using SAT.

(2) execute report ZHANA_PRODUCT_SAT_TOOL, download your trace file locally as xml file.

Specify the line from which you would to export.

For example in your trace result you have call stack like:

A

B

C

D

E

F

and you specify C, then the tool will export the trace for you which contains C, D, E, F.

Execute the report and the xml will be saved locally.

And the tool will also display the trace result in ALV:

3. trace your application once again and save the second trace result file by repeating step1 and step2. Now you have two xml trace files which are ready for comparison.

4. execute report ZHANA_PRODUCT_SAT_COMPARE.Sorry that I didn’t change the parameter description here. HN1 means the trace file which you expect the performance is better, and Q2U is that with poorer performance. Green threshold 50 means when the execution time of HN1 is 50% faster than Q2U, the comparison result row will display a green light to inform you. Yellow Threshold = 20 means if HN1 is at least 20% faster than Q2U but still less than 50% faster than Q2U, a yellow light will be displayed. A red light is displayed, if Q2U is faster than HN1 instead.

The final comparison result looks like below:

if you double click on a given row, it will automatically show you the exact source code:

Note:

1. Comparison means the two trace xml files being compared must come from the application traced by SAT with exactly the same input parameter. If each time you trace your application with different parameter, it is possible that the process execution flow would also be different, which makes the comparison meaningless.

2. Since you expect by double clicking on the trace result, it will automatically navigate to the exact source code. Suppose you are executing the compare report in a system C comparing the trace file exported from system A and system B. And system C does not have the class / FM involved in the two trace files. So please always execute the comparison report in either of the system where you perform the SAT trace.

This tool consists of a report ZHANA_PRODUCT_SAT_TOOL for downloading SAT trace result, a report ZHANA_PRODUCT_SAT_COMPARE for comparing the downloaded trace xml file, and three utility classes ZCL_CRM_HANA_TOOL, ZCL_CRM_HANA_XML_TOOL and ZCL_CRM_HANA_COMPARE_TOOL. I put their source code into attachment. I think they will give you an example how to access the trace result done by SAT via programming. The complete source code could be found from here.