Hello,
I am building a Django application using BaseX as its database, using a custom model layer with the Python API.
Currently, I am using a single Session instance to connect to the BaseX server, but I am facing problems when multiple queries are sent at the same time.
I didn't check how the client API works in details, but is it made in such a way that only one query at a time can be sent? Should I use a new Session instance for every single query sent to the server?
Thanks in advance.
Regards,
Michael Clement
Hi Michael,
I didn’t check how the client API works in details, but is it made in such a way that only one query at a time can be sent? Should I use a new Session instance for every single query sent to the server?
exactly, that’s the way our sessions work: If you want to run queries in parallel, you’ll have to create multiple sessions. In the use cases we are familiar with, this works out fine, because creating a new session is a pretty light-weight operation.
Hope this helps, Christian
One use case where this might be a problem is when you want all of your queries compiled at your application start time (using QueryProcessor::compile()). This is a speed/CPU load issue.
Please tell me I'm wrong it this is done automagically for modules stored in the database, but my impression is it doesn't happen until you call compile or execute (which calls compile).
I'm working in Java and plan to use Apache Commons Pool to store a bunch of sessions with associated start-time-compiled queries for the threads of my website requests. Can anyone see any problems with this approach?
On Fri, Apr 12, 2013 at 1:30 AM, Christian Grün christian.gruen@gmail.comwrote:
Hi Michael,
I didn’t check how the client API works in details, but is it made in
such a
way that only one query at a time can be sent? Should I use a new Session instance for every single query sent to the server?
exactly, that’s the way our sessions work: If you want to run queries in parallel, you’ll have to create multiple sessions. In the use cases we are familiar with, this works out fine, because creating a new session is a pretty light-weight operation.
Hope this helps, Christian _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
One use case where this might be a problem is when you want all of your queries compiled at your application start time (using QueryProcessor::compile()). This is a speed/CPU load issue.
This is something we want to tackle within the query processor architecture: query modules that are requested multiple times will automatically be cached and reused. As you may have seen, however, parsing and compilation time is usually pretty fast unless the number of linked library modules isn’t that extensive.
It would be interesting to hear if you (or someone else) has already encountered a real bottleneck of the current client/server architecture or the XQuery parsing/compilation step?
Christian ___________________________
On Fri, Apr 12, 2013 at 5:03 PM, Stu Salsbury stu.salsbury@gmail.com wrote:
Please tell me I'm wrong it this is done automagically for modules stored in the database, but my impression is it doesn't happen until you call compile or execute (which calls compile).
I'm working in Java and plan to use Apache Commons Pool to store a bunch of sessions with associated start-time-compiled queries for the threads of my website requests. Can anyone see any problems with this approach?
On Fri, Apr 12, 2013 at 1:30 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Michael,
I didn’t check how the client API works in details, but is it made in such a way that only one query at a time can be sent? Should I use a new Session instance for every single query sent to the server?
exactly, that’s the way our sessions work: If you want to run queries in parallel, you’ll have to create multiple sessions. In the use cases we are familiar with, this works out fine, because creating a new session is a pretty light-weight operation.
Hope this helps, Christian _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Thanks, Christian. I have not run into anything yet. In fact, I'm still deciding between Marklogic and BaseX and this is one area that I want to make sure I understand before I take the plunge. I haven't even looked at how I would handle web request threads or compiled query caching in Marklogic yet, so I'm still pretty naive. I appreciate your response. It looks like I read the situation right. I might be over-thinking the parse/compile overhead -- I'm underexposed to interpreted languages on servers.
Aside: the race is close, with BaseX in the lead. BaseX supports Xquery Update Facility and Java function imports, Marklogic has rollbacks and schema awareness (to what extent I don't yet know). I need to figure out how/if I can efficiently integrate BaseX with Saxon as a middle tier so that they can share DOM and SAX objects. The servlex project is interesting: https://code.google.com/p/servlex/
On Sat, Apr 13, 2013 at 1:58 PM, Christian Grün christian.gruen@gmail.comwrote:
One use case where this might be a problem is when you want all of your queries compiled at your application start time (using QueryProcessor::compile()). This is a speed/CPU load issue.
This is something we want to tackle within the query processor architecture: query modules that are requested multiple times will automatically be cached and reused. As you may have seen, however, parsing and compilation time is usually pretty fast unless the number of linked library modules isn’t that extensive.
It would be interesting to hear if you (or someone else) has already encountered a real bottleneck of the current client/server architecture or the XQuery parsing/compilation step?
Christian ___________________________
On Fri, Apr 12, 2013 at 5:03 PM, Stu Salsbury stu.salsbury@gmail.com wrote:
Please tell me I'm wrong it this is done automagically for modules
stored in
the database, but my impression is it doesn't happen until you call
compile
or execute (which calls compile).
I'm working in Java and plan to use Apache Commons Pool to store a bunch
of
sessions with associated start-time-compiled queries for the threads of
my
website requests. Can anyone see any problems with this approach?
On Fri, Apr 12, 2013 at 1:30 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Michael,
I didn’t check how the client API works in details, but is it made in such a way that only one query at a time can be sent? Should I use a new Session instance for every single query sent to the server?
exactly, that’s the way our sessions work: If you want to run queries in parallel, you’ll have to create multiple sessions. In the use cases we are familiar with, this works out fine, because creating a new session is a pretty light-weight operation.
Hope this helps, Christian _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Michael CLEMENT -
Stu Salsbury