Hi all, In order to have a "dynamic backend” implementation to REST-XQ invocations we used the fn:function-lookup() function but we are experiencing a strange behavior if the target looked-up function is an updating function that for example adds a resource to a database.
Putting the provided files in the webapp folder of a fresh new BaseX latest (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the .basex and with a database named DB already created inside) and invoking the REST function that calls directly the backend function with [1] the resource is correctly created on the database, by invoking the REST function that performs the dynamic lookup by using the fn:function-lookup() with [2] it completes without errors but the resource is not added to the DB. In our implementation the implementation modules are stored in the repo but the same behavior can be reproduced with “direct” function call as per my example.
Is there any limitation in the looked-up function related to updating databases?
Regards,
Mauro
[1] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/store", <joe/> ) [2] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/dyn-store", <joe/> )
Dear Mauro,
The combination of higher-order functions and MIXUPDATE led to a wrong assignment of static properties of your updating expression. I have made the compilation step more rigid: If a dynamic function call is found, and if mixing updates is enabled, the function will now always be tagged as an updating function. Feel free to check out the latest stable snapshot [1].
If you want to stick with 9.3.2, you can prefix the function lookup call with an additional "updating" keyword:
updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
Cari saluti all’ Italia; our hearts are with you, Christian
[1] http://files.basex.org/releases/latest/
Hi all, In order to have a "dynamic backend” implementation to REST-XQ invocations we used the fn:function-lookup() function but we are experiencing a strange behavior if the target looked-up function is an updating function that for example adds a resource to a database.
Putting the provided files in the webapp folder of a fresh new BaseX latest (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the .basex and with a database named DB already created inside) and invoking the REST function that calls directly the backend function with [1] the resource is correctly created on the database, by invoking the REST function that performs the dynamic lookup by using the fn:function-lookup() with [2] it completes without errors but the resource is not added to the DB. In our implementation the implementation modules are stored in the repo but the same behavior can be reproduced with “direct” function call as per my example.
Is there any limitation in the looked-up function related to updating databases?
Regards,
Mauro
[1] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/store",
<joe/> ) [2] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/dyn-store", <joe/> )
Dear Christian, Thank you so much, it also worked very well in version 9.3.1 that is the one we are using at the moment. We plan to move to the next stable release when it will be available.
Thanks also for your words; all the best to all: to Italy and also to people all over the world.
Mauro
Il giorno 31 mar 2020, alle ore 14:14, Christian Grün christian.gruen@gmail.com ha scritto:
Dear Mauro,
The combination of higher-order functions and MIXUPDATE led to a wrong assignment of static properties of your updating expression. I have made the compilation step more rigid: If a dynamic function call is found, and if mixing updates is enabled, the function will now always be tagged as an updating function. Feel free to check out the latest stable snapshot [1].
If you want to stick with 9.3.2, you can prefix the function lookup call with an additional "updating" keyword:
updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
Cari saluti all’ Italia; our hearts are with you, Christian
[1] http://files.basex.org/releases/latest/
Hi all, In order to have a "dynamic backend” implementation to REST-XQ invocations we used the fn:function-lookup() function but we are experiencing a strange behavior if the target looked-up function is an updating function that for example adds a resource to a database.
Putting the provided files in the webapp folder of a fresh new BaseX latest (9.3.2) downloaded instance (adding "MIXUPDATES = true” setting in the .basex and with a database named DB already created inside) and invoking the REST function that calls directly the backend function with [1] the resource is correctly created on the database, by invoking the REST function that performs the dynamic lookup by using the fn:function-lookup() with [2] it completes without errors but the resource is not added to the DB. In our implementation the implementation modules are stored in the repo but the same behavior can be reproduced with “direct” function call as per my example.
Is there any limitation in the looked-up function related to updating databases?
Regards,
Mauro
[1] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/store",
<joe/> ) [2] http:send-request( <http:request method="POST"> <http:body media-type="application/xml"/> </http:request>, "http://localhost:8984/dyn-store", <joe/> )
Dear Christian, About this problem we discover another issue that is causing us some troubles. In the same scenario of calling RESTXQ updating functions discovered dynamically in a basex with MIXUPDATES turned on: if a reading function is called on the same DB while an updating function is storing some content in the DB the DB will be corrupted. We performed some tests and some times we get an OutOfMemoryError exception, some time an ArrayIndexOutOfBoundsException with the result that some “rubbish” data gets stored into the DB (e.g. standalone attributes). I’ll provide you a SSCE [1] and the inserting function [2] we used to confirm the problem. During the save it is sufficient to call the read via rest several times with [3] to cause the problem. We tested on both last release (adding the “updating” prefix before the looked up function) and latest nightly build.
All my best,
Mauro
[1] test.xqm file
module namespace _ = 'urn:test';
declare %rest:path("save") %output:method("json") %output:media-type('application/json') %rest:POST("{$resource}") function _:create($resource as node()) { function-lookup(xs:QName("db:add"), 3)("test", $resource, random:uuid()) };
declare %rest:path("readall") %output:method("json") %output:media-type('application/json') %rest:GET function _:readall() { <json type="object">{ for tumbling window $in-window in function-lookup(xs:QName("db:open"), 1)("test") start $first at $s when $s = 1 end $last at $e when $e - $s = 10 - 1 return $in-window }</json> };
[2] distinct-values(for $i in 1 to 10000 let $res := http:send-request(<http:request method="post"> <http:body media-type="application/fhir+json"/> </http:request>, "http://localhost:8984/save", '{ "resourceType": "Encounter", "status" : "finished" }') return string($res[1]/@status))
[3] http://localhost:8984/readall
Il giorno 31 mar 2020, alle ore 14:14, Christian Grün christian.gruen@gmail.com ha scritto:
Dear Mauro,
The combination of higher-order functions and MIXUPDATE led to a wrong assignment of static properties of your updating expression. I have made the compilation step more rigid: If a dynamic function call is found, and if mixing updates is enabled, the function will now always be tagged as an updating function. Feel free to check out the latest stable snapshot [1].
If you want to stick with 9.3.2, you can prefix the function lookup call with an additional "updating" keyword:
updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
Cari saluti all’ Italia; our hearts are with you, Christian
Hi Mauro,
Mi dispiace di averti fatto aspettare.
Your test case was very helpful for reproducing the issue. The problem was caused by the combination of mixupdates and function-lookup. I hope that the issue is resolved with the latest stable snapshot [1] (it’s time to give it a TLA… maybe LSS?).
Ciao, Christian
[1] http://files.basex.org/releases/latest/
On Mon, Apr 27, 2020 at 1:04 PM Mauro Mugnaini themaxx76@gmail.com wrote:
Dear Christian, About this problem we discover another issue that is causing us some troubles. In the same scenario of calling RESTXQ updating functions discovered dynamically in a basex with MIXUPDATES turned on: if a reading function is called on the same DB while an updating function is storing some content in the DB the DB will be corrupted. We performed some tests and some times we get an OutOfMemoryError exception, some time an ArrayIndexOutOfBoundsException with the result that some “rubbish” data gets stored into the DB (e.g. standalone attributes). I’ll provide you a SSCE [1] and the inserting function [2] we used to confirm the problem. During the save it is sufficient to call the read via rest several times with [3] to cause the problem. We tested on both last release (adding the “updating” prefix before the looked up function) and latest nightly build.
All my best,
Mauro[1] test.xqm file
module namespace _ = 'urn:test';
declare %rest:path("save") %output:method("json") %output:media-type('application/json') %rest:POST("{$resource}") function _:create($resource as node()) { function-lookup(xs:QName("db:add"), 3)("test", $resource, random:uuid()) };
declare %rest:path("readall") %output:method("json") %output:media-type('application/json') %rest:GET function _:readall() { <json type="object">{ for tumbling window $in-window in function-lookup(xs:QName("db:open"), 1)("test") start $first at $s when $s = 1 end $last at $e when $e - $s = 10 - 1 return $in-window }</json> };
[2] distinct-values(for $i in 1 to 10000 let $res := http:send-request(<http:request method="post"> <http:body media-type="application/fhir+json"/> </http:request>, "http://localhost:8984/save", '{ "resourceType": "Encounter", "status" : "finished" }') return string($res[1]/@status))
[3] http://localhost:8984/readall
Il giorno 31 mar 2020, alle ore 14:14, Christian Grün christian.gruen@gmail.com ha scritto:
Dear Mauro,
The combination of higher-order functions and MIXUPDATE led to a wrong assignment of static properties of your updating expression. I have made the compilation step more rigid: If a dynamic function call is found, and if mixing updates is enabled, the function will now always be tagged as an updating function. Feel free to check out the latest stable snapshot [1].
If you want to stick with 9.3.2, you can prefix the function lookup call with an additional "updating" keyword:
updating fn:function-lookup(fn:QName('urn:ns:impl2', 'store'), 1)($resource)
Cari saluti all’ Italia; our hearts are with you, Christian
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Mauro Mugnaini