Hi Gary

I am a BaseX team member, but have you sized up your JVM to run "Big Data" ?

See the files BaseX/basex*.bat. I am currently using set VM=-Xmx6g

Hope this helps

Jean-Marc

2013/12/6 Huband, Gary W *HS GWH2SJ@hscmail.mcc.virginia.edu

I'm evaluating BaseX to use in a project that involves 10s of TB of xml data. Our goal is to be able to return a small amount of data from a query within a few seconds. Is BaseX capable of handling this much data?

My initial evaluation of 1GB of data met our requirements. But, when I evaluated BaseX using about 20GB of data I get an out of memory message when I run a query.

I'm running Windows 7 Pro on a Core i7, 8GB memory, and using the BaseX GUI.

Thanks for your time

Gary

---

BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Dear Gary,

and thanks, Jean-Marc, for the feedback.

I'm evaluating BaseX to use in a project that involves 10s of TB of xml data. Our goal is to be able to return a small amount of data from a query within a few seconds. Is BaseX capable of handling this much data?

BaseX can in fact be used to handle data in the size of terabytes. Some database statistics are listed in our Wiki [1], but in most cases, a single database should not exceed 10-50 gigabytes (the optimal size is always dependent on the actual structure of your input data). An important thing to mention here is that BaseX allows you to address any number of databases with a single XQuery expression, so the limit is not as restrictive as it may sound.

My initial evaluation of 1GB of data met our requirements. But, when I evaluated BaseX using about 20GB of data I get an out of memory message when I run a query.

This is something that shouldn’t happen of course. Could you please provide us with the query that triggered the error?

Thanks, Christian

[1] http://docs.basex.org/wiki/Statistics