First of all, great work you guys, thanks much. We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function: declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () }; This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery. Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server? Thanks, Srini
Hi Srini, in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do. Hope this helps, Christian [1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________ On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5. For example if I open BaseX GUI and run the command '*create db hello*' and then run the command "*xquery doc('hello')*", I get the following error: Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist. The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help. Thanks, Srini On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün <christian.gruen@gmail.com>wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
It looks like the commands I highlighted in gmail were appended with an asterisks at either ends. Please ignore the asterisks in the commands in my previous post. Also fyi, I do see the db 'hello' in basex list of databases after the first command ran. Thanks, Srini On Tue, Nov 13, 2012 at 1:00 AM, Srinivasan Muthu <newsrini@gmail.com>wrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command '*create db hello*' and then run the command "*xquery doc('hello')*", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün <christian.gruen@gmail.com
wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Srinisivan, it's true that the semantics of the doc() function in BaseX has changed: a database must now contain exactly one document in order to be detected by doc(). If you want to open a database, I recommend db:open() instead, which will additionally be a little faster, as it does not check for files in the file system. However, I agree that the error message has been too inadequate. This has changed in the latest stable snapshot (the address of which is… [1]). Christian [1] http://files.basex.org/releases/latest/ ___________________________ On Tue, Nov 13, 2012 at 10:00 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command 'create db hello' and then run the command "xquery doc('hello')", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and have several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Christian, We figured this eventually and made some changes to our code to create db with a simple root node example: create db blah </root> and that worked. Thanks for confirming. I have another issue that I would like to report and am going to open a new thread of conversation to keep the context clean. Thanks, Srini On Tue, Nov 13, 2012 at 3:55 PM, Christian Grün <christian.gruen@gmail.com>wrote:
Hi Srinisivan,
it's true that the semantics of the doc() function in BaseX has changed: a database must now contain exactly one document in order to be detected by doc(). If you want to open a database, I recommend db:open() instead, which will additionally be a little faster, as it does not check for files in the file system.
However, I agree that the error message has been too inadequate. This has changed in the latest stable snapshot (the address of which is… [1]).
Christian
[1] http://files.basex.org/releases/latest/ ___________________________
On Tue, Nov 13, 2012 at 10:00 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
Thanks Christian. I downloaded the latest build but the basic xQuery we run to check if database (document) exists fails. I am not sure if something changed in the xml data path configuration between 7.3 and 7.5.
For example if I open BaseX GUI and run the command 'create db hello' and then run the command "xquery doc('hello')", I get the following error:
Error: Stopped at line 1, column 3 in C:/Tools/basex_75/file: [FODC0002] Resource 'hello' does not exist.
The same set of commands work fine in 7.3. This sound very basic so I am guessing I am missing something simple here. Appreciate your help.
Thanks, Srini
On Sun, Nov 11, 2012 at 6:44 AM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi Srini,
in both cases, using the latest stable snapshot of BaseX 7.5 [1] may give you better results: first of all, updates have been sped up in this version; next, it provides a DBLOCKING flag in the .basex configuration file [2], which addresses your second problem, and which can be set to true in order to activate database locking. Note, however, that this flag is still experimental, which is why we can't give any guarantee yet if it will do what it's supposed to do.
Hope this helps, Christian
[1] http://files.basex.org/releases/latest/ [2] http://docs.basex.org/wiki/Configuration ___________________________
On Sun, Nov 11, 2012 at 8:31 AM, Srinivasan Muthu <newsrini@gmail.com> wrote:
First of all, great work you guys, thanks much.
We use BaseX 7.3 in client/server mode for concurrency purposes and
have
several databases. We have an updating function which updates the XML database after the initial creation to make sure the attributes are setup for faster querying. Here is the function:
declare updating function local:processVMs($dbName as xs:string){ let $doc := doc($dbName) let $vms := $doc/root/proj/VM return if (empty($vms/@desktopId)=true()) then ( for $vm in $vms let $desktopId := local:get-desktop-id-from-vm($vm) (: function not provided here:) return insert node (attribute {'desktopId'} {string($desktopId)}, 'text', <VM/>) into $vm) else () };
This function takes a minute to run when there are about 3000 objects. While this query runs, BaseX queues all other xqueries (including read xqueries on other databases too) and thereby rendering the server unresponsive until this update query is done. Due to this even the queries that are running on other databases become unresponsive and our application looks frozen until this update query is done.From reading old email archives here, I see that BaseX transaction monitor will queue all requests when update query is processed as multiple dbs can be updated using one xQuery.
Is this really true and is there any way to hint transaction monitor to lock only the database that is being updated and not the entire server?
Thanks, Srini
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
participants (2)
-
Christian Grün -
Srinivasan Muthu