Just a quick thank you for cElementTree. I am writing some code to do some ad hoc matching between elements of two XML files. The total file size is almost 8MB. As I develop the matching program I process the files over and over. I can read both files and iterate all their elements in the blink of an eye.

One of the files is abstracted from a 24MB file. I used cElementTree to create the abstract, too. I tried dom4j but it choked. Text editors choke on the file too. cElementTree ate it up and was hungry for more.

Thanks!

"Is it possible to use ElementTree to add processing instructions?"

Not really; if you want comments and PI:s to appear before the root element, you currently have to print them out yourself...

(I'm working on a new writer with better support for various things, including "invisible" elements which lets you add comments and PI:s on the document level, but it's not ready for release.)

Is it possible to use ElementTree to add processing instructions? I'm trying to add a reference to a stylesheet to my generated XML as follows:

root = Element("projectlist")
pi = ProcessingInstruction("xml-stylesheet","""type="text/xsl" href="projects.xslt" """)
root.insert(0,pi)

This inserts the PI under the root node; what I want is to insert it *before* the root node. I tried inserting it into an ElementTree instance, but it doesn't have an insert method and only seems to know about its root (i.e. no PIs that might preceed it) -- any suggestions?

Thanks!

Ramon

"Has the write() code been optimized at all in cElementTree?"

I'm afraid not; it uses the ElementTree serialization code. With cET 1.0 out of the door, better serialization code (for both libraries) is high on my list.

(note that the XML files you included are semantically identical, from an infoset perspective. computers should not care about the transformation...)

"But it was a pretty nice little piece of deceptive code, don't you think?"

Well it was certainly quite an optimization! Mixing the deceptive generator expression with timeit was especially sly. You should haul it out again on April Fool's Day.

Has the write() code been optimized at all in cElementTree? I'm seeing very nice speedups compared with libxml2 when parsing and processing, but I lose them all when I serialize out to disk. I also note that in a simple parse/write test, cElementTree does not appear to write out the identical XML file (adds in liberal namespace info to nodes). For example, given this GraphML file as input:

<?xml version="1.0" encoding="UTF-8"?>
<graphml xmlns="http://graphml.graphdrawing.org/xmlns/graphml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:y="http://www.yworks.com/xml/graphml">
  <graph id="G" edgedefault="directed">
 <node id="foo"/>
</graph>
</graphml>

cElementTree.write() will produce the following:

<ns0:graphml xmlns:ns0="http://graphml.graphdrawing.org/xmlns/graphml">
  <ns0:graph edgedefault="directed" id="G">
 <ns0:node id="foo" />
</ns0:graph>
</ns0:graphml>

Any suggestions?

"I think there's a problem with your latest benchmark"

You're 100% correct. The benchmark is flawed, on purpose (hence the evil laugh ;-). But it was a pretty nice little piece of deceptive code, don't you think?

[Had to edit my post -- had pasted the same timing twice, now fixed.]

Edited by author 01-26-2005 03:35 PM
I think there's a problem with your latest benchmark: it looks like you're only measuring the time it takes to create the generator object. It doesn't look like your code is executing. If you wrap the generator in a list(), to force execution, the timing takes longer. (I'm using the tiny samples/simple.xml from the cElementTree zip, below.)

(dn@change)(02:22P)
(%:~/src/cElementTree-1.0-20050126)- python2.4 -m timeit -s "import cElementTree" "matches = (elem.get('value') for event, elem in cElementTree.iterparse('samples/simple.xml') if elem.get('name') == 'reselectApi')"
10000 loops, best of 3: 23.9 usec per loop

(dn@change)(02:23P)
(%:~/src/cElementTree-1.0-20050126)- python2.4 -m timeit -s "import cElementTree" "matches = list(elem.get('value') for event, elem in cElementTree.iterparse('samples/simple.xml') if elem.get('name') == 'reselectApi')"
10000 loops, best of 3: 116 usec per loop

http://www.intel.com/products/notebook/pro...celeron_m/index.htm
says that Celeron M processors go only up to 1.5 GHz, and 8600s are delivered with Celeron M or Pentium M.

I seem to have too much time at my hands to actually care about that;-). On the other hand, google is just too handy...

The article says: "It's a Centrino 1.7GHz, which is about equivalent to a P4-3GHz"

Unfortunately, this is unlikely to be true, and you can see that from the Pystone results. It may be the case that Centrino bundles use Pentium-M CPUs, but for all I know (and care) it could be a Celeron on the motherboard. Anyway, apart from all the power-saving horseplay that could go on, the main difference between Pentium-M and P4 is probably the cache - again, from my own limited knowledge of the field and my P2-266-based perspective of the world.

Just another note to the pystone numbers: When I read the Uche's article this morning, I computed the pystone for my desktop machine, and although the CPU is less powerful (1.4 GHz Athlon), the came out higher. On my ThinkPad T41p, which has 512MiB of RAM and a 1.7GHz Pentium-M and therefore comparable to his Dell notebook, I get the following numbers:
Pystone(1.1) time for 50000 passes = 1.33
This machine benchmarks at 37594 pystones/second

Python is 2.3.4. Either my notebook is magically enhanced (which it of course is), Debian is outrageously faster than FC3 (true, too) or something is wrong about his numbers.
Anyway, keep up the good work, I look forward to adding (c)ElementTree packages to Debian soon and using it in my own software!

Edited by author 01-24-2005 12:34 PM
"are we to conclude that this means namespace declarations which come into and go out of scope on the same element will be nested appropriately"

That's the intention, at least. I guess I have to add some more tests to make sure that this is always true...

I love the new iterparse function, but one thing about the example you give bothers me. The SAX documentation warns that the order in which namespaces start and end may not be nested "properly" with respect to each other. I see that the 'end-ns' event from iterparse just returns None, are we to conclude that this means namespace declarations which come into and go out of scope on the same element will be nested appropriately?

Edited by author 01-22-2005 09:49 AM
cElementTree in the standard library? why not; it wouldn't be the first thing I've written that makes it into the library... but I'm not going to lobby for that myself; I usually leave such things to enthusiastic users (that was a hint ;-)

thanks Timothy for that tip. I am aware of the Python Properties file, and frequently use it to alternate between different Python versions on my PC. In this case, though, I had updated to 2.4 and forgot to also update that Properties file.

S