Christian,

Thx very much for your workaround. I did not succeed using the GUI and executing »SET INLINELIMIT 0«. However, it seems that there exists several way to set this parameter without the GUI. I tested successfully the two following methods:

- Set the parameter before the query as follow : (# db:INLINELIMIT 0 #) {fn:count($insert())} However, it is somehow tedious to write all queries that way. - Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local Options. I would prefer this method, would you have no objection.

I am not sure to understand well this parameter. Does it means that it is possible to inline function in XQUERY ? This is quite close to template mechanism programming !

Cheers,

Jean-Marc

2013/11/22 Christian Grün christian.gruen@gmail.com

Hi Jean-Marc,

the bug you encountered is due to a newly feature (called “function inlining”) we’ve introduced just recently. It will be fixed soon [2]. Until then, you can simply disable it by setting INLINELIMIT to 0 [1] This can e.g. be done by choosing »Command« in the dropdown box of the GUI main window, and executing the command »SET INLINELIMIT 0«.

Thanks for the reproducible code, Christian

[1] http://docs.basex.org/wiki/Options#INLINELIMIT [2] https://github.com/BaseXdb/basex/issues/796 ___________________________

On Fri, Nov 22, 2013 at 10:28 AM, jean-marc Mercier jeanmarc.mercier@gmail.com wrote:

...

Hi,

Within my environment, I am experiencing some frustrating behavior of the Xquery Basex engine. I've done my best to single out the problem in the following XQUERY code :

declare function local:wtf($x as item() ) {function() { $x}}; declare function local:dummy($f,$x) { if(fn:empty($f())) then local:wtf($x) else local:wtf(())};

declare variable $insert := local:dummy(function(){ () }, function() {

(),

...

()} ); fn:count($insert())

The issues are :

1. executing this code yields an "Improper use? Potential bug?" exception

(see details further on). 2) If I change the execution query to be fn:count(local:dummy(function(){ () }, function() { (), ()} )()) ( I just copy and pasted the value of $insert into the fn:count(..) ) then the

query

...

is executed normally, returning "1". 3) If I change the function declaration local:wtf as follow declare function local:wtf($x) {function() { $x}}; (I just removed "as item()"), then the query is executed normally, returning "1".

Can you reproduce it with another environment ?

Cheers,

Jean-Marc

Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.8 beta 6aeaebf Java: Oracle Corporation, 1.7.0_05 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.value.item.FuncItem.inlineExpr(FuncItem.java:308) at org.basex.query.func.DynFuncCall.optimize(DynFuncCall.java:58) at org.basex.query.func.DynFuncCall.compile(DynFuncCall.java:39) at org.basex.query.expr.Arr.compile(Arr.java:40) at org.basex.query.func.StandardFunc.compile(StandardFunc.java:60) at org.basex.query.MainModule.compile(MainModule.java:63) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:70) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:61) at org.basex.query.QueryContext.analyze(QueryContext.java:270) at org.basex.query.QueryContext.compile(QueryContext.java:253) at org.basex.query.QueryProcessor.compile(QueryProcessor.java:71) at org.basex.core.cmd.AQuery.query(AQuery.java:81) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:323) at org.basex.core.Command.execute(Command.java:92) at org.basex.server.LocalSession.execute(LocalSession.java:121) at org.basex.server.Session.execute(Session.java:37) at org.basex.core.Main.execute(Main.java:146) at org.basex.BaseX.<init>(BaseX.java:119) at org.basex.BaseX.main(BaseX.java:38)

---

BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

Dear Jean-Marc,

I fixed the bug, thank you for reporting it. You can get the current snapshot here: http://files.basex.org/releases/latest/

Am 22.11.2013 15:06, schrieb jean-marc Mercier:

Thx very much for your workaround. I did not succeed using the GUI and executing »SET INLINELIMIT 0«.

That only sets the setting for the current GUI session, not permanently.

• Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local

Options. I would prefer this method, would you have no objection.

It should be fine, but globally disables all inlining of functions. So just remember to remove it after you have a fixed version of BaseX.

I am not sure to understand well this parameter. Does it means that it is possible to inline function in XQUERY ? This is quite close to template mechanism programming !

BaseX inlines all functions (static ones and function items) that are * not recursive (or, for function items, don't contain themself), and * below a certain size, which can be changed through the `INLINELIIT`.

This means that introducing abstraction by encapsulating code in many small functions should have little to no runtime overhead. Together with higher-order functions, this makes quite general and abstract libraries feasible.

Cheers, Leo

Leo,

I have a remark and a question about inlining. - 1) this inlining mechanism is good, no contest about it. A drawback is that the stack trace is inefficient, hardening the debugging. Thus it is better to set INLINE = 0 in dev phase. 2) Could you be kind enough to point me out the link to your code at Basex repository ? Motivation : I would like to try evaluating whether inlining recursive functions is possible or not. You are welcome to suggestions !

Cheers

2013/11/22 jean-marc Mercier jeanmarc.mercier@gmail.com

Leo, Christian

...

I fixed the bug, thank you for reporting it. You can get the current

snapshot here: http://files.basex.org/releases/latest/ I just tested it with my lib code, it is now working fine, thx. You guys are reactive !

...

That only sets the setting for the current GUI session, not permanently.

Ok thx, this explains why I couldn't make it works with my environment.

...

BaseX inlines all functions (static ones and function items) etc..

That's really interesting. Very close to template programming as expression templating, but without having to write a single line of code. That could be of great utility if I try to write an algebra module.

Cheers,

Jean-Marc

2013/11/22 Leo Wörteler lw@basex.org

...

Dear Jean-Marc,

I fixed the bug, thank you for reporting it. You can get the current snapshot here: http://files.basex.org/releases/latest/

Am 22.11.2013 15:06, schrieb jean-marc Mercier:

Thx very much for your workaround. I did not succeed using the GUI and

...

executing »SET INLINELIMIT 0«.

That only sets the setting for the current GUI session, not permanently.

• Edit the file .basex, adding the line "INLINELIMIT = 0" after #Local

...

Options. I would prefer this method, would you have no objection.

It should be fine, but globally disables all inlining of functions. So just remember to remove it after you have a fixed version of BaseX.

I am not sure to understand well this parameter. Does it means that it

...

is possible to inline function in XQUERY ? This is quite close to template mechanism programming !

BaseX inlines all functions (static ones and function items) that are

• not recursive (or, for function items, don't contain themself), and
• below a certain size, which can be changed through the `INLINELIIT`.

This means that introducing abstraction by encapsulating code in many small functions should have little to no runtime overhead. Together with higher-order functions, this makes quite general and abstract libraries feasible.

Cheers, Leo

Hi Leo,

sorry for the delayed answer. I was looking to something smart to say over this topic, but nothing came :) Indeed, I don't have tested yet patterns like expression template to make any pertinent remark there.

you should probably also set `TAILCALLS` to -1

Thx for pointing this out.

This also means that unconditionally inlining recursive functions can send

the compiler into an infinite loop.

For C++ (the only interpreter I know a little), this is left under the user responsibility. When such infinite loop occurs, then the user payback its freedom spending a lot of time to blindly understand why its interpreter rose an exception. I don't know whether it is a good idea or not.

This is conceptually simpler, but would bloat the syntax tree and mostly

(?) not help much. See remark above.

In a first phase, having inlined version of fn:fold-left (maybe naming it as hof:fold-left would be safer ?) should be enough, and safer, since you should know in advance the call depth. Shall (INLINE = ?) control the depth call ?

I did not have a look to the code yet, but I was wondering how can you detect recursive calls ? The self recursive case seems easy to detect, but if I glue two functions together, it seems trickier.

Cheers

Jean-Marc

2013/11/24 Leo Wörteler lw@basex.org

Jean-Marc,

Am 23.11.2013 17:52, schrieb jean-marc Mercier:

A drawback is that the stack trace is inefficient, hardening the

...

debugging. Thus it is better to set INLINE = 0 in dev phase.

yes; you should probably also set `TAILCALLS` to -1 if you want the stack trace to always be accurate. As BaseX performs tail-call optimization [1], the stack will otherwise never contain more than a fixed number (currently 256) of tail-call frames.

Could you be kind enough to point me out the link to your code at Basex

...

repository ?

Sure, function inlining is triggered in StaticFuncCall [2] and DynFuncCall [3].

Motivation : I would like to try evaluating whether inlining recursive

...

functions is possible or not. You are welcome to suggestions !

The hard part about inlining recursive functions is figuring out when to stop. As they have a call to themselves in the function body, each time you inline them at least one new call is inserted. This also means that unconditionally inlining recursive functions can send the compiler into an infinite loop.

Inlining is indeed possible for all functions for which we can guarantee that the number of replacements will be finite. I'm currently implementing this for `fn:fold-left`, `fn:fold-right` and `fn:for-each` (basically loop unrolling) when the sequence is short and known at compile time. It would definitely be possible to implement this for user-defined functions, too, but we'd need to identify (a subset of) the correct ones.

Another option would be to always inline recursive functions up to a fixed depth. This is conceptually simpler, but would bloat the syntax tree and mostly (?) not help much.

What are your ideas? Leo

[1] http://en.wikipedia.org/wiki/Tail_call [2] https://github.com/BaseXdb/basex/blob/next/basex-core/ src/main/java/org/basex/query/func/StaticFuncCall.java#L71 [3] https://github.com/BaseXdb/basex/blob/next/basex-core/ src/main/java/org/basex/query/func/DynFuncCall.java#L57