June 2015 - BaseX-Talk - mailman.uni-konstanz.de

thinking outloud
by Rob Stapper 17 Jun '15

17 Jun '15

Hi Christian, Elaborating on the issue below and just thinking out loud. Would it make sense having to opportunity to evaluate a function body lazily? Consider the following function: declare function item:exists ( $item , $existProcessor , $emptyValue ) { if ( exists( $item)) then $existProcessor( $item) else $emptyValue } ; Even when called with an existing: $item, $emptyValue will be evaluated what could be an expensive evaluation. Having the possibility of lazy evaluation of the function body would prevent this. I’ve two questions: - does the above make sense? - does it make enough sense to be implemented in basex? If implemented: for keeping the code clean, specifying this laziness should be done within the function, not within the function call. I tumbled on the %basex:inline-annotation for global variables. This would do very nicely for specifying laziness for evaluating function bodies. declare %basex:inline function item:exists ( $item , $existProcessor , $emptyValue ) { if ( exists( $item)) then $existProcessor( $item) else $emptyValue } ; Best, Rob Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: woensdag 13 mei 2015 12:36 Aan: Rob Stapper; BaseX Onderwerp: Re: [basex-talk] unexpected behaviour If I get it right, you would also expect '!' not to be output in the following query, right? declare function local:f($a, $b) { if($a) then $b else () }; local:f((), trace('!')) In this case, you'll have to wrap your trace call in a function: declare function local:f($a, $b) { if($a) then $b() else () }; local:f((), function() { trace('!') }) Best, Christian On Wed, May 13, 2015 at 11:44 AM, Rob Stapper <r.stapper(a)lijbrandt.nl> wrote: Hi Christian, This is the best I can do without losing the essence of the issue. Sorry about the import, forgot to remove that one. The clue is in the “Query Info”-window. It shows: Evaluating: f1: 1 Compiling: ……. So function “local:f1#1” is unnecessarely evaluated with value 1. Success, Rob Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: woensdag 13 mei 2015 11:13 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] unexpected behaviour Hi Rob, It usually takes me some time to understand new code, so could you possibly simplify the example as much as possible? For example, the import cannot be resolved, an empty squence is bound to $a, which means that one branch will never be called, and maybe you can simplify the partial function application as well.. Thanks in advance, Christian On Wed, May 13, 2015 at 9:55 AM, Rob Stapper <r.stapper(a)lijbrandt.nl> wrote: Hi Christian, Can you have a look at the attached program. Both items of the returned sequence of the program should do the same thing. Both do give the same, correct result: an empty sequence, but the first item unnecessarely triggers function: “local:f1#1” , while the second item doesn’t. This a pity since “local:f1#1” could be a time-consuming function. Cheers, Rob _____ <http://www.avast.com/> Afbeelding verwijderd door afzender. Avast logo Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. www.avast.com <http://www.avast.com/> _____ <http://www.avast.com/> Afbeelding verwijderd door afzender. Avast logo Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. www.avast.com <http://www.avast.com/> --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 2

Re: [basex-talk] Performing queries with a local function takes too much time
by Christian Grün 17 Jun '15

17 Jun '15

Hi Serdar (cc to the list), Your query amounts to more than thousands of index requests, and this eventually makes the query faster than when being executed sequentially. You can enforce sequential execution for the last for clause by using "eq" instead of "=". Next, you should factor out the content id path; otherwise, it will be evaluated thousands of times as well. The following query will be executed a bit faster: let $PlCts := //PlCt[environment = 'AIR']/@id for $PlGeTy in //PlGeTy[isOfPlCt/@href = $PlCts] for $PlSpTy in //PlSpTy[isOfPlGeTy/@href = $PlGeTy/@id] let $ContId := $PlSpTy/isOfCont/@href/string() for $Cont in //Cont[@id eq $ContId] return <done>{ $PlGeTy, $PlSpTy, $Cont }</done> Ideally, the query optimizer would detect that it's counterproductive to rewrite the last for clause to an index expression. However, decisions like this very much depend on the input data, and there are no simple heuristics for that. Hope this helps, Christian On Wed, Jun 17, 2015 at 1:39 PM, serdar ceran <bsceran(a)gmail.com> wrote: > Hi Christian, > > I have also another optimization issue on a query as I mentioned in Update > three section. > Could you help on this one? > > Thanks, > > Serdar > > > On Wed, Jun 17, 2015 at 12:57 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Serdar, >> >> Thanks for the hint. We have added a new GitHub issue to remember your >> request. >> >> Best, >> Christian >> >> [1] https://github.com/BaseXdb/basex/issues/1160 >> >> >> On Wed, Jun 17, 2015 at 11:18 AM, serdar ceran <bsceran(a)gmail.com> wrote: >> > Hi, BaseX team. I have a problem; Performing queries with a local >> > function >> > takes too much time to get the result. >> > >> > Please have a look at the Stackoverflow for more details. >> > >> > >> > Best regards; >> > >> > >> > Serdar CERAN >> > >> > > > > > > -- > Best regards; > > -- > B. Serdar CERAN > > Yaltes Company > Specialist Software Engineer > Tel: +905423993073 >

1 1

Replication, partition and the CAP theorem
by ziiro_94＠freenet.de 15 Jun '15

15 Jun '15

Hello, I consider BaseX in the context of BigData and try to compare it with other noSQL databases. Yet, there is no replication or partition mechanism implemented and BaseX is a database with a single point of failure. Am I right? Thus, my question is, how can I classify the System BaseX regarding the CAP-Theorem? Yours sincerely --- Alle Postfächer an einem Ort. Jetzt wechseln und E-Mail-Adresse mitnehmen! Rundum glücklich mit freenetMail

2 1

XQuery - math on elements within a sequence
by Luiz Matos 15 Jun '15

15 Jun '15

Hi everyone, I'm trying to execute a XQuery sum function in a multiplication of two XML elements, but it has been difficult to avoid the iteration in a sequence of elements. For example, considering this case: sample data: <Orders> <Order> <OrderKey>1</OrderKey> <LineItem> <LineNumber>1</LineNumber> <ExtendedPrice>70848.0000</ExtendedPrice> <Discount>0.0913</Discount> <ReturnFlag>A</ReturnFlag> <LineStatus>F</LineStatus> <ShipDate>1994-09-03</ShipDate> </LineItem> <LineItem> <LineNumber>2</LineNumber> <ExtendedPrice>64372.0000</ExtendedPrice> <Discount>0.0674</Discount> <ReturnFlag>A</ReturnFlag> <LineStatus>F</LineStatus> <ShipDate>1994-08-18</ShipDate> </LineItem> <LineItem> <LineNumber>3</LineNumber> <ExtendedPrice>64247.0000</ExtendedPrice> <Discount>0.0396</Discount> <ReturnFlag>R</ReturnFlag> <LineStatus>F</LineStatus> <ShipDate>1994-10-21</ShipDate> </LineItem> </Order> . . . </Orders> XQuery: <results> { for $lineitem in collection("tpch")/Orders/Order/LineItem let $returnflag := $lineitem/ReturnFlag let $linestatus := $lineitem/LineStatus where $lineitem/ShipDate <= "1998-08-31" group by $returnflag, $linestatus order by $returnflag, $linestatus return <record> <sum_disc_price>{sum($lineitem/ExtendedPrice*(1-$lineitem/Discount))}</sum_disc_price> </record> } </results> BaseX output: [XPTY0004] Item expected, sequence found: (element ExtendedPrice {...}, ...... My question: Considering this dataset model, how can I to write this query without change its semantics? I wouldn't want to use extra functions (e.g., fn:for-each), because I need to submit the same query in an limited xquery processor. Thanks in advance! -- Luiz Matos PhD student at Federal Fluminense University

2 1

Internal cache for often accessed node sets?
by Gioele Barabucci 15 Jun '15

15 Jun '15

Hello, thanks to the all the answers I received in the thread "Optimization of a slow query with `//`" I can now make most of my queries much faster. However, while rewriting these queries I noticed that almost all the queries in my application could be rewritten in the form /tei:TEI/tei:text/tei:body//tei:ELEM[PREDICATE] and I wondered if it was possible to tell BaseX (or to have it understand by itself) that it can cache and reuse the results of /tei:TEI/tei:text/tei:body//tei:ELEM and only apply PREDICATE on the saved set of results. Is something like this available in BaseX? I suppose this could have a big impact on my queries, because the base query executes in about 250 ms and most of the other unoptimized queries in about 280 ms. This means that with the case in place they should complete in 30 ms (or 50 ms in more realistic terms). I tried sending via HTTP a dummy request for `/tei:TEI/tei:text/tei:body//tei:ELEM` together with a `<option name='cachequery' value='true'/>` to warm up the cache. Unfortunately all I got back was a "Unknown option 'CACHEQUERY'" error (with BaseX 8.2.1). Any idea why? Regards, -- Gioele Barabucci <gioele(a)svario.it>

3 2

Re: [basex-talk] Optimization of a slow query with `//`
by Christian Grün 15 Jun '15

15 Jun '15

Hi Gioele, Queries with positional predicates such as the one that I posted in my last mail (in private, attached below) are now optimized as well. Best, Christian [1] http://files.basex.org/releases/latest/ On Mon, Jun 15, 2015 at 1:09 PM, Christian Grün <christian.gruen(a)gmail.com> wrote: > Hi Gioele, > >> I had a look at the info view but I fail to understand whether it is >> optimized or not. What should I look at? Sometimes I see a "Optimized Query" >> section, but it is not present for these two queries. > > I see. Yes, this means that the query is not rewritten. Instead, this > is one is optimized for index access, as the query info (and, > obviously, in this case the execution time) will show you: > > /descendant::tei:orth[text() = "arci"] > > I noticed that the query is not rewritten because of the positional > predicate (which are generally brain twisters in XQuery). Things look > better with the following query (but it may yield other results): > > /descendant::tei:orth > [text() = "arci"] > [ancestor-or-self::* > [@xml:lang][starts-with(@xml:lang, "san")] > ] > /parent::tei:form/parent::*[self::tei:entry or self::tei:re] > > The following query should take advantage of the index: > > for $n in /descendant::tei:orth[text() = "arci"] > where $n/ancestor-or-self::* > [@xml:lang][1][starts-with(@xml:lang, "san")] > return $n/parent::tei:form/parent::* > [self::tei:entry or self::tei:re] > > Best, > Christian

1 0

Optimization of a slow query with `//`
by Gioele Barabucci 15 Jun '15

15 Jun '15

Hello, I am working on an application that retrieves its data from a TEI XML file via BaseX. The following query lies at the core of this application but is too slow to be used in production: on a modern PC it requires about 600 ms to run over a 4MB file (1/10 of the complete dataset). Any suggestion on how to improve its performance (without changing the underlying TEI files) would be much appreciated. Here is the query: declare namespace tei='http://www.tei-c.org/ns/1.0'; /tei:TEI/tei:text/tei:body// *[self::tei:entry or self::tei:re] [./tei:form/tei:orth[. = "arci"] [ancestor-or-self::* [@xml:lang][1] [(starts-with(@xml:lang, "san"))] ] ] In human terms is should return all the `tei:entry` or `tei:re` that * have the word "arci" in their `/tei:form/tei:orth` element, * their nearest `xml:lang` attribute starts with 'san'. I made some tests and it turned out that the main culprit is the use of `//` in the first line. (_Main_ culprit, not the only one...) I use the `//` axis because I do not know what is the structure of the underlying TEI file. I expect BaseX to keep track of all the `tei:entry` and `tei:re` elements and their parents, so selecting the correct ones should be quite fast anyway. But the measurements disagree with my assumptions... What could I do to improve the performance of this query? Now, some remarks based on some small tests I have done: 1. Removing the [ancestor-or-self::*[....]] predicate slashes the run time in half, but the query is still way too slow. 2. Changing ./tei:form/tei:orth[. = "arci"] to ./tei:form[1]/tei:orth[1][. = "arci"] makes the query even slower. 3. changing `starts-with(@xml:lang, "san")` to `@xml:lang = 'san-xxx'` has a negligible effect. 4. Dropping the `[1]` from [@xml:lang][1] makes the whole query twice as fast. Regards, -- Gioele Barabucci <gioele(a)svario.it>

4 8

BaseX Mac refusing to quit
by Stuart Simon 14 Jun '15

14 Jun '15

I have BaseX 8.2.1 running on Mac OS X 10.10.3 Yosemite, BaseX GUI.app, release channel. My computer has a 2.2 GHz Intel processor and 4 GB of RAM. My problem is that I have tried all the standard ways to quit BaseX, but BaseX stubbornly refuses to quit; it only closes the active windows. Is there anything I'm doing wrong, or is this a bug? -- Stuart Simon stuart.sa(a)gmail.com

3 5

dba suggestions
by Andy Bunce 14 Jun '15

14 Jun '15

Hi Here are a couple of minor suggestions for dba enhancement/fixes (based on 8.2.1). I should get a PR together but that might be some time. 1. Login field tab order. Currently this is (1) name (2) <login bn> (3) password. Moving the login button to after the password would make sense to me as I could then tab through the fields. This could be done just be putting the button in the next row (or setting an explicit tab order if there is some reason for the current appearance.) 2. logs There is a little bug in the log sorting function. The log page is /dba/*logs*?... The field headers go to /dba/*log*?.. Also it would be nice if urls for the log display would use any supplied query parameter for initial filtering e.g going to /dba/logs?name=2015-06-09&resource=&sort=&loglist=&query=%2Fmyapp%2F would display the logs for 2015-06-09 containing /myapp/ /Andy

2 5

Re: [basex-talk] Multiple web applications with independent xq modules
by Ioan Fericel 11 Jun '15

11 Jun '15

Hi Christian, On 06/06/2015 02:39 PM, Christian Grün wrote: > Hi Ioan, > > Just recently, there has been some discussion on Tomcat and BaseX on > the mailing list. I think we managed to successully encounter various > of the problems with Tomcats in the latest snapshot [1], so I will be > interested in hearing more about your experiences with that version. Sorry for my english expressing, which unfortunately are not helped me to be well understood! I carefully watched to all recent discussions on the list, but what is of interest to us is different. I have seen that in these cases it is intended that each application to use a new instance of BaseX. And, on the same Tomcat, to use multiple separate instances of BaseX. And, in general, I have seen that BaseX is used dedicated for a single application (very suitable for embeded), and used separated instances for other applications. But in our case is exactly the opposite. We have some experience with web applications on Tomcat, made with JSP / Servlets and relational databases. The structure of the applications installed on web server looks like: -webapps -app1 -app2 -app3 Each application is developed and deployed separately, as an war archive, having everything it needs inside. We try to use BaseX in this way to access data directly in XML format. The structure should be something like: -webapps -basex -bxapp1 -xqrepo1 -bxapp2 -xqrepo2 -bxapp3 -xqrepo3 We managed to make this model functional only if all 3 repositories we have put them together: or directly under BaseX, or in a separate application and setting according to this the path in context-param/restxqpath in web.xml. But we want it so that each application to contain they repo. How? It would be possible to set in web.xml, to restxqpath, a list of several repo? Or from somewhere to be told from where to load such a list? Maybe this would be an easier solution. It implies some security problems or other constraints? We looked a bit by sources, but not have identified which part it handles this. Is welcomed any other idea. > Regarding CACHERESTXQ and .ignore: These features are already > available in that snapshot as well (but they will only be officially > announced with 8.3). Unless you don't build more complex RESTXQ > applications, you won't probably need to care about these features. > > Hope this helps, > Christian > > [1] http://files.basex.org/releases/latest/ > > We tried this last version and, indeed, for what we want does not help us. Many thanks for your understanding! Ioan

3 6