Hi Dave,
I completely agree that the QueryProcessor class has not been tailored (..yet?..) to provide a clean API for external users. Your feedback is indeed very helpful. Some general remarks:
After looking at this a little more, I have to wonder if the QueryProcessor class is safe for external use. It contains private flags to indicate whether the QueryContext has been parsed or compiled, but these flags don't get reset on events that might invalidate the previous compilation such as changing the context or updating the database - the only case I can find where they do get reset is if the actual expression changes.
Currently, those expressions are mainly used to prevent duplicate parsing/compilations of a query. They are only invalidated by the query() method, which is only called by the XQJ API. In this case, it would be more correct to really invalidate the query context instead of just initializing the two flags. The query() method might disappear as soon as we have an external XQJ implementation that will render ours obsolete.
Because the parse() and compile() methods in the QueryProcessor class check the corresponding flag, it's actually impossible to re-compile the QueryContext from the QueryProcessor (without changing the expression).
The main reason why it's currently impossible to recompile an expression is that once an expression is compiled, it won't be possible to reset the expression to its initial state. For example, a query that has been rewritten to using an index cannot be reset to its initial expression tree. One advantage of the current approach is that the parsing/compilation steps are very fast; they'll surely get slower if we decide to create an additional compiled tree representation and preserve the original tree. Still, a clean separation would probably the way to go in future.
Digging deeper, it appears as though the methods that allow these changes aren't even really used within BaseX - QueryProcessor.context() is never called and QueryProcessor.bind() is only used from the QueryListener in the server. This suggests that these methods were added in an attempt to create a public API and perhaps aren't tested that much or haven't been revisited.
The context() method is currently used by the XQJ and the W3T APIs (located in the basex-api and basex-tests repositories); bind() is used at some other places. It's true that those methods are mainly used by the APIs (indeed I believe that BaseX doesn't have *any* unused methods except for some that are defined in the Performance class..).
My suggestion would be one of the following:
- Verify that under all conditions calling QueryProcessor.context(), QueryProcessor.bind(), etc. on a previously evaluated query and then evaluating again does not cause problems (though again, based on your explanation it sounds they should).
As indicated above, I think the main challenge here is to introduce two separate representations for parsed and compiled expressions. The related question is: how should the compiled representation look like? Should it include alternative rewritings for index access, which can be chosen at runtime if appropriate? Sounds exciting, but ambitious as well.
Once more.. Hope this helps ;) Christian