First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu newsrini@gmail.com wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command '*create db hello*' and then run the command "*xquery doc('hello')*", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün christian.gruen@gmail.comwrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu newsrini@gmail.com wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (:
function
not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects.
While
this query runs, BaseX queues all other xqueries (including read
xqueries on
other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running
on
other databases become unresponsive and our application looks frozen
until
this update query is done.From reading old email archives here, I see
that
BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to
lock
only the database that is being updated and not the entire server?
Thanks, Srini
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
It looks like the commands I highlighted in gmail were appended with an asterisks at either ends. Please ignore the asterisks in the commands in my previous post. Also fyi, I do see the db 'hello' in basex list of databases after the first command ran.
Thanks, Srini
On Tue, Nov 13, 2012 at 1:00 AM, Srinivasan Muthu newsrini@gmail.comwrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command '*create db hello*' and then run the command "*xquery doc('hello')*", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün <christian.gruen@gmail.com
wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu newsrini@gmail.com wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are
setup
for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (:
function
not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects.
While
this query runs, BaseX queues all other xqueries (including read
xqueries on
other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are
running on
other databases become unresponsive and our application looks frozen
until
this update query is done.From reading old email archives here, I see
that
BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to
lock
only the database that is being updated and not the entire server?
Thanks, Srini
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Srinisivan,
it's true that the semantics of the doc() function in BaseX has changed: a database must now contain exactly one document in order to be detected by doc(). If you want to open a database, I recommend db:open() instead, which will additionally be a little faster, as it does not check for files in the file system.
However, I agree that the error message has been too inadequate. This has changed in the latest stable snapshot (the address of which is… [1]).
Christian
[1] http://files.basex.org/releases/latest/ ___________________________
On Tue, Nov 13, 2012 at 10:00 AM, Srinivasan Muthu newsrini@gmail.com wrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command 'create db hello' and then run the command "xquery doc('hello')", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu newsrini@gmail.com wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Christian,
We figured this eventually and made some changes to our code to create db with a simple root node example: create db blah </root> and that worked. Thanks for confirming.
I have another issue that I would like to report and am going to open a new thread of conversation to keep the context clean.
Thanks, Srini
On Tue, Nov 13, 2012 at 3:55 PM, Christian Grün christian.gruen@gmail.comwrote:
Hi Srinisivan,
it's true that the semantics of the doc() function in BaseX has changed: a database must now contain exactly one document in order to be detected by doc(). If you want to open a database, I recommend db:open() instead, which will additionally be a little faster, as it does not check for files in the file system.
However, I agree that the error message has been too inadequate. This has changed in the latest stable snapshot (the address of which is… [1]).
Christian
[1] http://files.basex.org/releases/latest/ ___________________________
On Tue, Nov 13, 2012 at 10:00 AM, Srinivasan Muthu newsrini@gmail.com wrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we
run
to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command 'create db hello' and then run the command "xquery doc('hello')", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu newsrini@gmail.com wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and
have
several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'}
{string($desktopId)},
'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive
until
this update query is done. Due to this even the queries that are
running
on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor
to
lock only the database that is being updated and not the entire server?
Thanks, Srini
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Srinivasan Muthu