Multi threading in the BusinessWorks Mapper and the BusinessWorks Container Edition Mapper
An interesting feature introduced in BusinessWorks 6.11 and BusinessWorks Container Edition 2.10 is the capability to leverage multi-threading in the Mapper activity itself which allows to reduce significantly the execution times of complex mappings.
This is useful when applying complex mapping rules to large lists of objects in context where processing times matters, whether for managing synchronous APIs or processing large batches of data.
Configuration to set-up to leverage multi threading in the BusinessWorks Mapper
This can be done by simply adding the following two attributes to a for-each statement:
threads : this defines how much threads will be used in the Mapper activity to process the list available in the Mapper input
contextsize : this defines the number of elements from the input list that will be processed at once
For example:
xsl:for-each select=”$ParseXml/tns3:Order” threads=”4" contextsize=”100">
In the example above 4 threads will be used to process the input list and elements from the input list are process by batch of 100 (a thread process 100 elements, another thread process 100 other elements and etc).
It is recommend to keep the number of threads to a relatively low value, like 2 or 4, and in every cases lower than the number of cores available in the runtime environment.
The context size has to be adjusted depending on the size of the input list and taking into account the general recommendation to use batches of a significant size to avoid doing a lot of context switching, generally values between 100 and a few hundred’s are recommended.
The XSLT Source editor is displayed by clicking on the ‘Edit Tab’ button of the Mapper has shown below:
Test of multi threading in the BusinessWorks Mapper
I created a test case with relatively complex mapping rules applied to a large data set to test the behavior of this new feature.
The behavior observed with the default mapper configuration is the following:
. Processing time is 46 seconds
. CPU usage while processing is around 11–12 % (one core on a 8 cores machines)
The behavior observed with the mapper configured to use multi-threading is the following:
. Processing time is 14 seconds (divided by more than 3)
. CPU usage while processing is around 60 % (approximately 5 cores on a 8 cores machine)
Overall the results are showing that multi-threading in the Mapper significantly reduces processing time to the cost of an higher cpu usage during the processing which is expected.
Reference elements
Extract from BusinessWorks 6.11 and BusinessWorks Container Edition 2.10 release notes:
