Re: [basex-talk] Local variables holding retrieved data from BaseX
Hi Teena (cc @basex-talk), first of all, thanks for giving us details on your data and queries. To complete this: do you have a complete version of your query that shows us what are the values of $context, $project and $scopeName? As an example, I’m wondering what are the ancestor nodes of the <scopes> elements.
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { if (exists($project) = true()) then ( $project/scopes/scope[@name=$scopeName]/item ) else () };
The following query is equivalent, by the way (this is the beauty of XPath): declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { $project/scopes/scope[@name=$scopeName]/item }; If all your data is stored in the same database, it may be faster to open the database within the function. One more optimization could be to directly access the text index: declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { db:attribute("name-of-your-db", $scopeName, "name")/parent::scope }; You could also check out the latest stable snapshot of BaseX 7.8 [1], which inlines functions and may thus do this automatically.
When we call the first function in another function and use it for further computation, it is always slower by 40 seconds (when compared to second variation of the function). What could be causing this?
Difficult to say; we probably need to see your complete function call.
Are we still referencing to baseX document when we do following instead of getting a local copy? let $items := foo:get-node-v1($context, $project, $scopeName)
Yes, the returned nodes still reference nodes in the database. Hope this helps, Christian [1] http://files.basex.org/releases/latest/
Christian, Thanks for your prompt reply. I had tried db:attribute for quicker look-up but the bottle-neck for me is the usage part and not the retrieval part. Retrieval is quick but good to know that it has been taken care of in latest stable version of BaseX. There are several joins due to complexity of data happening in the query that use this data coming from foo:get-node-v1(all the three parameters that are passed to this function are small objects.) To give you a quick overview of the query I have: : : let $aNodes := foo:get-a($context) let $bNodes := foo:get-b($context) let $scopeValues := foo:get-node-v1($context, $project, $scopeName) for $aNode in $aNodes for $bNode in $bNodes[<predicate>] : : let $val := $foo:another-function($scopeValues, $bNode) foo:another-function is basically filtering $scopeValues relevant to $bNode. We have a high number of $aNodes and $bNodes. If I use the second variation wherein the data content is exactly same (only difference is that is local copy now) then my xquery performs way better. Is there a way to get a local copy of the database node so that we do not reference the node in the database for further usage in the xquery (any other cleaner way instead of iterating over the data and creating a local copy that foo:get-node-v2 is doing)? Thanks, Teena On Thu, Dec 12, 2013 at 12:24 PM, Christian Grün <christian.gruen@gmail.com>wrote:
Hi Teena (cc @basex-talk),
first of all, thanks for giving us details on your data and queries. To complete this: do you have a complete version of your query that shows us what are the values of $context, $project and $scopeName? As an example, I’m wondering what are the ancestor nodes of the <scopes> elements.
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { if (exists($project) = true()) then ( $project/scopes/scope[@name=$scopeName]/item ) else () };
The following query is equivalent, by the way (this is the beauty of XPath):
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { $project/scopes/scope[@name=$scopeName]/item };
If all your data is stored in the same database, it may be faster to open the database within the function. One more optimization could be to directly access the text index:
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { db:attribute("name-of-your-db", $scopeName, "name")/parent::scope };
You could also check out the latest stable snapshot of BaseX 7.8 [1], which inlines functions and may thus do this automatically.
When we call the first function in another function and use it for further computation, it is always slower by 40 seconds (when compared to second variation of the function). What could be causing this?
Difficult to say; we probably need to see your complete function call.
Are we still referencing to baseX document when we do following instead of getting a local copy? let $items := foo:get-node-v1($context, $project, $scopeName)
Yes, the returned nodes still reference nodes in the database.
Hope this helps, Christian
Is there a way to get a local copy of the database node so that we do not reference the node in the database for further usage in the xquery (any other cleaner way instead of iterating over the data and creating a local copy that foo:get-node-v2 is doing)?
You could try the XQuery Update transform expression [1]: copy $a := <your-result/> modify () return $a [1] http://docs.basex.org/wiki/XQuery_Update#transform
Thanks! On Thu, Dec 12, 2013 at 1:20 PM, Christian Grün <christian.gruen@gmail.com>wrote:
Is there a way to get a local copy of the database node so that we do not reference the node in the database for further usage in the xquery (any other cleaner way instead of iterating over the data and creating a local copy that foo:get-node-v2 is doing)?
You could try the XQuery Update transform expression [1]:
copy $a := <your-result/> modify () return $a
participants (2)
-
Christian Grün -
Teena Gulabani