VTD-XML Benchmark Report for Version 2.3

(Part III) XPath Evaluation Performance

Objective

This articles reports the measurement of XPath evaluation performance for VTD-XML version 2.3 and compares with Xerces 2.7.1 DOM and XPath functions bundled with JDK 1.6 (which is Xalan), as well as Jaxen 1.1.

Methodology Discussion

The benchmark code performs the following steps:

Read XML file into an in-memory buffer.
Parse the document once.
Evaluate a single pre-compiled XPath expression for a large number of iterations.

All benchmark programs first loop thru the XPath evaluation code a number of iterations so the server JVM compile them into native code to obtain optimal performance, before the real measurement of the average latency of subsequent XPath evaluations.

Test Setup

Hardware

Processor: Core2 Duo T9300 2.5GHz (6MB L2 integrated cache).
Memory: 3GB 800Mhz FSB

Software

Windows Vista Business
JDK version 1.60; default set to server JVM.
XML parsers: Xerces DOM version 2.7.1and XPath bundled with JDK 1.5 (Xalan), Xerces DOM 2.7.1 and Jaxen, VTD-XML 2.3

The XML files

Three XML files of similar structure, but different sizes, are used for the test.

<?xml version="1.0"?>
<purchaseOrder orderDate="1999-10-20">
    <shipTo country="US">
        <name>Alice Smith</name>
        <street>123 Maple Street</street>
        <city>Mill Valley</city>
        <state>CA</state>
        <zip>90952</zip>
    </shipTo>
    <billTo country="US">
        <name> Robert Smith </name>
        <street>8 Oak Avenue</street>
        <city>Old Town</city>
        <state>PA</state>
        <zip>95819</zip>
    </billTo>
    <comment>Hurry, my lawn is going wild!</comment>
    <items>
        <item partNum="872-AA">
            <productName>Lawnmower</productName>
            <quantity></quantity>
            <USPrice>148.95</USPrice>
            <comment>Confirm this is electric</comment>
        </item>
        <item partNum="926-AA">
            <productName>Baby Monitor</productName>
            <quantity>1</quantity>
            <USPrice>39.98</USPrice>
            <shipDate>1999-05-21</shipDate>
        </item>
       ...
    </items>
</purchaseOrder>

The respective file sizes are:

"po_small.xml" ---- 6780 bytes
"po_medium.xml" ---- 112,238 bytes
"po_big.xml" ----- 1,219,388 bytes

The following XPath expressions are used for the test

/*/*/*[position() mod 2 = 0]
/purchaseOrder/items/item[USPrice<100]
/*/*/*/quantity/text()
//item/comment
//item/comment/../quantity
/purchaseOrder/items/item[USPrice > 100]/comment/text()
/*/*/*/* [position() mod 2 = 0]

The Benchmark Code Comparison

DOM Code	Jaxen Code	VTD-XML Code
Document d = null; k = total; d = parser.parse(bais); while (k > 0) { NodeList nodeList = (NodeList) xPathExpression.evaluate(d, XPathConstants.NODESET); k--; } long l=0,lt=0; for (int j = 0; j < 10; j++) { l = System.currentTimeMillis(); for (int i = 0; i < total; i++) { NodeList nodeList = (NodeList) xPathExpression.evaluate(d, XPathConstants.NODESET); } long l2 = System.currentTimeMillis(); lt = lt + (l2 - l); } System.out.println(" average XPath latency ==> " + ((double) (lt) / total / 10) + " ms");	XPath expression = new org.jaxen.dom.DOMXPath(xpe); System.out.println("====> update_jaxen1 ==>"+xpe); //parse and xpath eval Document d = null; d = parser.parse(bais); k = total; while (k > 0) { List results = expression.selectNodes(d); k--; } long l=0,lt=0; for (int j = 0; j < 10; j++) { l = System.currentTimeMillis(); for (int i = 0; i < total; i++) { List results = expression.selectNodes(d); } long l2 = System.currentTimeMillis(); lt = lt + (l2 - l); } System.out.println(" average XPath latency ==> " + ((double) (lt) / total / 10) + " ms");	k=total; VTDNav vn = null; while(k>0){ while((i1=ap.evalXPath())!=-1){ } ap.resetXPath(); k--; } for (int j=0;j<10;j++){ l = System.currentTimeMillis(); for(int i = 0;i<total;i++) { while((i1=ap.evalXPath())!=-1){ } ap.resetXPath(); } long l2 = System.currentTimeMillis(); lt = lt + (l2 - l); //System.out.println("j ==> "+j); } System.out.println(" average XPath latency ==> "+ ((double)(lt)/total/10) + " ms");

Results

Notice that the table includes # of nodes returned from XPath evaluation.

Latency Number Comparison

/*/*/*[position() mod 2 = 0]

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.7328	0.0229	0.005
po_medium.xml (112,238 bytes)	3.5045	0.28	0.0593
po_big.xml (1,060,823 bytes)	35.09	2.925	0.621
po_huge.xml (9,907,759 bytes)	397.59	26.09	6.04

/purchaseOrder/items/item[USPrice <100]

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.845	0.037	0.027
po_medium.xml (112,238 bytes)	5.069	0.6	0.512
po_big.xml (1,060,823 bytes)	50.39	8.53	5.465
po_huge.xml (9,907,759 bytes)	559.415	77.692	46.583

/*/*/*/quantity/text()

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.746	0.059	0.0195
po_medium.xml (112,238 bytes)	3.981	1.02	0.325
po_big.xml (1,060,823 bytes)	42.28	11.975	3.645
po_huge.xml (9,907,759 bytes)	541.795	129.522	29.913

//item/comment

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.849	0.203	0.052
po_medium.xml (112,238 bytes)	5.32	3.689	0.945
po_big.xml (1,060,823 bytes)	62.61	52.578	10.284
po_huge.xml (9,907,759 bytes)	602	1018.6	83.574

//item/comment/../quantity

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.854	0.23	0.061
po_medium.xml (112,238 bytes)	5.773	3.894	1.04
po_big.xml (1,060,823 bytes)	69.323	56.19	11.83
po_huge.xml (9,907,759 bytes)	630.58	1042.33	101.09

/*/*/*/* [position() mod 2 = 0]

	JDK(ms)	Jaxen (ms)	VTD-XML(ms)
po_small.xml (6,780 bytes)	0.802	0.0822	0.0225
po_medium.xml (112,238 bytes)	4.402	1.349	0.324
po_big.xml (1,060,823 bytes)	49.91	16.995	3.582
po_huge.xml (9,907,759 bytes)	897.95	158.02	31.915

/purchaseOrder/items/item[USPrice > 100]/comment/text()

	JDK (ms)	Jaxen (ms)	VTD-XML (ms)
po_small.xml (6,780 bytes)	0.846	0.0531	0.0331
po_medium.xml (112,238 bytes)	5.425	0.936	0.6
po_big.xml (1,060,823 bytes)	58	12.626	7.388
po_huge.xml (9,907,759 bytes)	571.23	116.35	63.428

Graphical View