Dear Gary,
and thanks, Jean-Marc, for the feedback.
I'm evaluating BaseX to use in a project that involves 10s of TB of xml data. Our goal is to be able to return a small amount of data from a query within a few seconds. Is BaseX capable of handling this much data?
BaseX can in fact be used to handle data in the size of terabytes. Some database statistics are listed in our Wiki [1], but in most cases, a single database should not exceed 10-50 gigabytes (the optimal size is always dependent on the actual structure of your input data). An important thing to mention here is that BaseX allows you to address any number of databases with a single XQuery expression, so the limit is not as restrictive as it may sound.
My initial evaluation of 1GB of data met our requirements. But, when I evaluated BaseX using about 20GB of data I get an out of memory message when I run a query.
This is something that shouldn’t happen of course. Could you please provide us with the query that triggered the error?
Thanks, Christian