Christian,

For example, I have big load of urls to crawl. Urls can be split to multiple chunks. Basex can start multiple function calls in parallel that deals one chunk.

After functions finished, we can merge results. This is basic way.

I think that we can benefit excellent concurrency feature of clojure language. basex can call clojure functions but I don't know how

On Thu, Oct 24, 2013 at 6:33 PM, Christian Grün <cg@basex.org> wrote:

Hi Erol,

I can also imagine various use cases in which multithreading would be
beneficial. Have you already thought about ways how multithreaded
execution could work in XQuery?

Christian
____________________________________

2013/10/24 Erol Akarsu <eakarsu@gmail.com>:

> I am working on crawling data. I can see it is taking a lot of time because
> xquery is pulling
> web pages one by one. But popular crawlers like apache Nutch has
> multithreading support so it can open multiple socket connections to
> multiple sites so that it can crawl very fast.
> Can we make add multithreading support basex? So it can run multiple xquery
> functions in parallel
> and then merge results?
> Initially, merging results can be done manually. At least, if we can
> distribute loads to different xquery functions, that would be great.
>
> Erol Akarsu