XML documents should not be large
XML.com: Decomposition, Process, Recomposition :-
So my first line of advice has always been: don’t go processing gigabyte files in XML formats. I have been working with XML for about 8 years. I have used XML in numerous ways with numerous tools for numerous purposes. I have never come across a natural need for an XML file more than a few megabytes. There may be terabytes of raw data in a system, but if you’re following XML (and classic data design) best practices, this should be structured into a number of files in some sensible way.
I’m saving this hear so I can remember where it is when one of my customers complains about poor performance or memory issues with the Oracle XDK.
It happens about once a month, and 9 times out of 10 they have huge XML files. The problem is that they don’t think they have a huge XML file and think 1Gb is a reasonable size, and give spurious arguments like “a powerful database like Oracle should be able to handle this much data”.
I have always told them that 1Mb is a more sensible limit, but have never been able to find a solid link to back myself up.
August 5th, 2004 at 10:51
Your milage will most certainly vary, but I start shifting about nervously when the relational databases I use start hitting a gig, let alone XML files