http://www.cs.ucdavis.edu/research/tech-reports/2009/CSE-2009-12.pdf
I remembered I had read this paper as well. They claimed great performance with their methods of using MapReduce. Quite an interesting read.
Eric
2011/11/16 Lukas Lewandowski Lukas.Lewandowski@uni-konstanz.de
Hi Eric,
We did already some tests with Hadoop (HDFS and MapReduce). Some ideas are quite good input for a possible future distribution architecture of BaseX. All interesting ideas and approaches are welcome.
Lukas
Am 16.11.2011 um 18:13 schrieb Eric Murphy:
Lukas,
Also, some thought should be given to what sort of distributed file system should be used for distributed BaseX. Maybe you could look at HDFS so you can get a full perspective on what a distributed solution would be, top to bottom.
Obviously the system requirements for distributed computing are very different than your normal server, so all of that should be considered in how the data is queried.
This looks like fun research, good luck.
Eric
On Wed, Nov 16, 2011 at 11:43 AM, Lukas Lewandowski < Lukas.Lewandowski@uni-konstanz.de> wrote:
Hi Eric,
Thanks for the links to the papers about distributed XQuery. Currently we are evaluating different approaches to distribute and query XML data. Basically, we are interested in using such an approach directly out of XQuery. The complexity for XQuery developers should be as small as possible, therefore we prefer solutions which do not extend XQuery specification with complex hierarchical structures. The "Distributed XQuery-based integration and visualization of multimodality brain mapping data" approach looks interesting and straightforward. We do more research in distribution approaches within the next weeks.
regards Lukas
On Monday, November 14, 2011 17:32 CET, Christian Grün < christian.gruen@uni-konstanz.de> wrote:
Hi Eric,
nice to have your update!
I've been seeing quite a bit of chatter about cloud computing and distributed systems at work of recent. I think over the next few
years,
there is going to be a dire need for Distributed XQuery with our
government
customers. They have massive amounts of XML data and it keeps growing.
That's good to know; it's also reflected in the feedback we're getting on BaseX. We have some students working on getting BaseX in the cloud; it will be interesting to get their opinion on DXQ.. @Lukas, Dirk, Lukas: could you have a look at the two papers referenced by Eric, and give some feedback?
http://www.cs.washington.edu/homes/kmorton/paper.pdf http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2636687/
Regarding the Android port: do you think your code base is already complete enough to be published, or to be given away?
Thanks, Christian