Hi Jean-Marc,
Might it be garbage collector stuff ?
That's exactly what it is.
2015-01-14 8:18 GMT+01:00 jean-marc Mercier <jeanmarc.mercier@gmail.com>:
Glory to our holy Leo for transmitting brahmavidya to shishya !!
Turning back to this optimization problem, this might be trickier since
execution time does not seem to behave linearly. For instance, consider
declare function local:test1($i){let $data_3 as item() *:=
fn:for-each((1 to 1000000*$i), function($a_0) as item()? { (2 * $a_0) }) return for $n in (1 to 2) return fn:count($data_3)};
declare function local:test2($i){count(fn:for-each((1 to 1000000*$i), function($a_0) as item()? { (2 * $a_0) }))};
Then execution time is ( the results are stable while repeating the experience )
prof:time(local:test1(7)) : 631.32 ms prof:time(local:test2(7)) : 451.32 ms prof:time(local:test1(8)) : 2550.17 ms prof:time(local:test2(8)) : 507.32 ms
2015-01-13 22:43 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Jean-Marc,
beforehand: The first issue you reported back to us recently [1] has been fixed by our functional guru Leo! A new snapshot is available [2].
Hi. Thanks for the FLWOR trick, behaving in constant time. However it
seems
to generates an unexpected 5X time overhead
Most probably this is because the results of for-each will be cached, while in the previous case (if no inlining takes place), all items will be iterated.
Hope this helps, Christian
[1] https://github.com/BaseXdb/basex/issues/1052 [2] http://files.basex.org/releases/latest/
On Tue, Jan 13, 2015 at 9:19 AM, jean-marc Mercier <jeanmarc.mercier@gmail.com> wrote:
Christian,
Hi. Thanks for the FLWOR trick, behaving in constant time. However it seems to generates an unexpected 5X time overhead, and I can't use it straightforwardly as workaround. Can you reproduce it, since this overhead might be due to eclipse ?
declare function local:test(){fn:for-each(1 to 10000000, function($a){2*$a})}; declare function local:test1(){count(local:test())}; declare function local:test2(){let $ data := local:test() return for $n in (1 to 10) return count($data)};
prof:time(local:test1()) ouput : 639.12 ms prof:time(local:test2()) ouput : 2431.67 ms
2015-01-13 1:59 GMT+01:00 Christian Grün <christian.gruen@gmail.com>:
Hi Jean-Marc,
interesting one; it seems that $data is inlined and thus evaluated more than once. This doesn't happen if a FLWOR expression is used:
let $data := fn:for-each ... for $n in 1 to $i return local:test2($data, $n)
Should be easy to resolve.
Thanks for your (always concise) examples, Christian
On Mon, Jan 12, 2015 at 9:08 PM, jean-marc Mercier <jeanmarc.mercier@gmail.com> wrote:
Hello,
I am facing some performances issues, probably due to unexpected
query
reevaluation. Might it be due to the optimizer ? Here is a code to test
declare function local:test($i){ let $data := fn:for-each(1 to 10000000, function($a){2*$a} ) (: let $data := xquery:eval("fn:for-each(1 to 10000000, function($a){2*$a} )") :) return (1 to $i) ! local:test2($data,.) };
declare function local:test2($data, $dummy) { count($data) };
(1 to 10) ! prof:time(local:test(.))
Output :
683.76 ms 1137.95 ms 1727.68 ms 2151.06 ms 2694.84 ms 3189.57 ms 3725.1 ms 4277.51 ms 4815.74 ms 5417.79 ms
Note that using xquery:eval (toggling the comment inside local:test). seems to perform correctly. However I can't use this trick as a workaround.
(1 to 10) ! prof:time(local:test(.)) Ouput :
2472.71 ms 825.61 ms 3013.14 ms 986.81 ms 2982.97 ms 778.24 ms 1195.33 ms 3280.7 ms 976.96 ms 913.69 ms