Re: [basex-talk] Performing queries with a local function takes too much time
Hi Serdar (cc to the list), Your query amounts to more than thousands of index requests, and this eventually makes the query faster than when being executed sequentially. You can enforce sequential execution for the last for clause by using "eq" instead of "=". Next, you should factor out the content id path; otherwise, it will be evaluated thousands of times as well. The following query will be executed a bit faster: let $PlCts := //PlCt[environment = 'AIR']/@id for $PlGeTy in //PlGeTy[isOfPlCt/@href = $PlCts] for $PlSpTy in //PlSpTy[isOfPlGeTy/@href = $PlGeTy/@id] let $ContId := $PlSpTy/isOfCont/@href/string() for $Cont in //Cont[@id eq $ContId] return <done>{ $PlGeTy, $PlSpTy, $Cont }</done> Ideally, the query optimizer would detect that it's counterproductive to rewrite the last for clause to an index expression. However, decisions like this very much depend on the input data, and there are no simple heuristics for that. Hope this helps, Christian On Wed, Jun 17, 2015 at 1:39 PM, serdar ceran <bsceran@gmail.com> wrote:
Hi Christian,
I have also another optimization issue on a query as I mentioned in Update three section. Could you help on this one?
Thanks,
Serdar
On Wed, Jun 17, 2015 at 12:57 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Serdar,
Thanks for the hint. We have added a new GitHub issue to remember your request.
Best, Christian
[1] https://github.com/BaseXdb/basex/issues/1160
On Wed, Jun 17, 2015 at 11:18 AM, serdar ceran <bsceran@gmail.com> wrote:
Hi, BaseX team. I have a problem; Performing queries with a local function takes too much time to get the result.
Please have a look at the Stackoverflow for more details.
Best regards;
Serdar CERAN
-- Best regards;
-- B. Serdar CERAN
Yaltes Company Specialist Software Engineer Tel: +905423993073
Hi Serdar, I forgot one more optimization: You can take advantage of maps in XQuery 3.1 and cache repeatedly accessed values in advance: let $Conts := map:merge(//Cont/map { @id/string(): . }) let $PlCts := //PlCt[environment = 'AIR']/@id for $PlGeTy in //PlGeTy[isOfPlCt/@href = $PlCts] for $PlSpTy in //PlSpTy[isOfPlGeTy/@href = $PlGeTy/@id] for $Cont in $Conts($PlSpTy/isOfCont/@href) return <done>{ $PlGeTy, $PlSpTy, $Cont }</done> If you are wondering what's going on in the first line of code.. Here is a more verbose writing: map:merge( for $Cont in //Cont return map { $Cont/@id/string(): $Cont } ) You can even take this further and create maps for all accessed ids in advance, as shown in the query below. However, please note that... * this may take a lot of memory if your document contains a large number of entries * it may get slower than the original query if you only a few of the cached entries * as you see below, you'll have to group entries if there is more than one entry with the same key. Maps in XQuery are a powerfull feature. It may take some time to get to know them, but it's really worth the trouble.. Have fun, Christian let $Conts := map:merge(//Cont/map { @id/string(): . }) let $PlGeTys := map:merge( for $value in //PlGeTy group by $key := $value/isOfPlCt/@href/string() return map { $key: $value } ) let $PlSpTys := map:merge( for $value in //PlSpTy group by $key := $value/isOfPlGeTy/@href/string() return map { $key: $value } ) for $PlCts in //PlCt[environment = 'AIR']/@id for $PlGeTy in $PlGeTys($PlCts) for $PlSpTy in $PlSpTys($PlGeTy/@id) for $Cont in $Conts($PlSpTy/isOfCont/@href) return <done>{ $PlGeTy, $PlSpTy, $Cont }</done>
participants (1)
-
Christian Grün