- BaseX-Talk - mailman.uni-konstanz.de

Optimization of a slow query with `//`
by Gioele Barabucci 15 Jun '15

15 Jun '15

Hello, I am working on an application that retrieves its data from a TEI XML file via BaseX. The following query lies at the core of this application but is too slow to be used in production: on a modern PC it requires about 600 ms to run over a 4MB file (1/10 of the complete dataset). Any suggestion on how to improve its performance (without changing the underlying TEI files) would be much appreciated. Here is the query: declare namespace tei='http://www.tei-c.org/ns/1.0'; /tei:TEI/tei:text/tei:body// *[self::tei:entry or self::tei:re] [./tei:form/tei:orth[. = "arci"] [ancestor-or-self::* [@xml:lang][1] [(starts-with(@xml:lang, "san"))] ] ] In human terms is should return all the `tei:entry` or `tei:re` that * have the word "arci" in their `/tei:form/tei:orth` element, * their nearest `xml:lang` attribute starts with 'san'. I made some tests and it turned out that the main culprit is the use of `//` in the first line. (_Main_ culprit, not the only one...) I use the `//` axis because I do not know what is the structure of the underlying TEI file. I expect BaseX to keep track of all the `tei:entry` and `tei:re` elements and their parents, so selecting the correct ones should be quite fast anyway. But the measurements disagree with my assumptions... What could I do to improve the performance of this query? Now, some remarks based on some small tests I have done: 1. Removing the [ancestor-or-self::*[....]] predicate slashes the run time in half, but the query is still way too slow. 2. Changing ./tei:form/tei:orth[. = "arci"] to ./tei:form[1]/tei:orth[1][. = "arci"] makes the query even slower. 3. changing `starts-with(@xml:lang, "san")` to `@xml:lang = 'san-xxx'` has a negligible effect. 4. Dropping the `[1]` from [@xml:lang][1] makes the whole query twice as fast. Regards, -- Gioele Barabucci <gioele(a)svario.it>

4 8

BaseX Mac refusing to quit
by Stuart Simon 14 Jun '15

14 Jun '15

I have BaseX 8.2.1 running on Mac OS X 10.10.3 Yosemite, BaseX GUI.app, release channel. My computer has a 2.2 GHz Intel processor and 4 GB of RAM. My problem is that I have tried all the standard ways to quit BaseX, but BaseX stubbornly refuses to quit; it only closes the active windows. Is there anything I'm doing wrong, or is this a bug? -- Stuart Simon stuart.sa(a)gmail.com

3 5

dba suggestions
by Andy Bunce 14 Jun '15

14 Jun '15

Hi Here are a couple of minor suggestions for dba enhancement/fixes (based on 8.2.1). I should get a PR together but that might be some time. 1. Login field tab order. Currently this is (1) name (2) <login bn> (3) password. Moving the login button to after the password would make sense to me as I could then tab through the fields. This could be done just be putting the button in the next row (or setting an explicit tab order if there is some reason for the current appearance.) 2. logs There is a little bug in the log sorting function. The log page is /dba/*logs*?... The field headers go to /dba/*log*?.. Also it would be nice if urls for the log display would use any supplied query parameter for initial filtering e.g going to /dba/logs?name=2015-06-09&resource=&sort=&loglist=&query=%2Fmyapp%2F would display the logs for 2015-06-09 containing /myapp/ /Andy

2 5

Re: [basex-talk] Multiple web applications with independent xq modules
by Ioan Fericel 11 Jun '15

11 Jun '15

Hi Christian, On 06/06/2015 02:39 PM, Christian Grün wrote: > Hi Ioan, > > Just recently, there has been some discussion on Tomcat and BaseX on > the mailing list. I think we managed to successully encounter various > of the problems with Tomcats in the latest snapshot [1], so I will be > interested in hearing more about your experiences with that version. Sorry for my english expressing, which unfortunately are not helped me to be well understood! I carefully watched to all recent discussions on the list, but what is of interest to us is different. I have seen that in these cases it is intended that each application to use a new instance of BaseX. And, on the same Tomcat, to use multiple separate instances of BaseX. And, in general, I have seen that BaseX is used dedicated for a single application (very suitable for embeded), and used separated instances for other applications. But in our case is exactly the opposite. We have some experience with web applications on Tomcat, made with JSP / Servlets and relational databases. The structure of the applications installed on web server looks like: -webapps -app1 -app2 -app3 Each application is developed and deployed separately, as an war archive, having everything it needs inside. We try to use BaseX in this way to access data directly in XML format. The structure should be something like: -webapps -basex -bxapp1 -xqrepo1 -bxapp2 -xqrepo2 -bxapp3 -xqrepo3 We managed to make this model functional only if all 3 repositories we have put them together: or directly under BaseX, or in a separate application and setting according to this the path in context-param/restxqpath in web.xml. But we want it so that each application to contain they repo. How? It would be possible to set in web.xml, to restxqpath, a list of several repo? Or from somewhere to be told from where to load such a list? Maybe this would be an easier solution. It implies some security problems or other constraints? We looked a bit by sources, but not have identified which part it handles this. Is welcomed any other idea. > Regarding CACHERESTXQ and .ignore: These features are already > available in that snapshot as well (but they will only be officially > announced with 8.3). Unless you don't build more complex RESTXQ > applications, you won't probably need to care about these features. > > Hope this helps, > Christian > > [1] http://files.basex.org/releases/latest/ > > We tried this last version and, indeed, for what we want does not help us. Many thanks for your understanding! Ioan

3 6

Re: [basex-talk] Problem with xi:include - What is the correct href path?
by Kristian Kankainen 10 Jun '15

10 Jun '15

Hello! This is not the most appropriate place to ask, but I'd just like to get a hint of the overall feeling of it. Since the XInclude support is relied upon that of Java, is there any hope it will become complete some day? I'd be satisfied with answers like "I'm positive the support will be completed, but it might take some two years" or "the development seems stalled and probably nothing will happen if noone interested in it makes an effort". Thank you Kristian 08.06.2015 15:33, Christian Grün kirjutas: > Dear Asxetos, > > BaseX relies on the XInclude support of Java, which is (unfortunately) > incomplete. Your example works locally when removing the xpointer > attribute (but it will include the full bar.xml resource). > Aditionally, please note that local or relative XInclude reference > cannot be resolved with DBA, because the file to be embedded will not > be available to the database server. > > If you are interested in getting more background information, please > check out another discussion on this issue on the mailing list [1]. > > Hope this helps, > Christian > > [1] https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg06197.ht… > > > On Mon, Jun 8, 2015 at 2:17 PM, Asxetos <pure.asxetos(a)gmail.com> wrote: >> Hello, >> >> I would like your help getting the following to work. >> I have document 'foo.xml' : >> >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> <?xml version="1.0" encoding="UTF-8"?> >> <?xml-model href="http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng" >> type="application/xml" >> schematypens="http://relaxng.org/ns/structure/1.0" ?> >> <TEI xmlns="http://www.tei-c.org/ns/1.0"> >> <teiHeader> >> <fileDesc> >> <titleStmt> >> <title>foo</title> >> </titleStmt> >> <publicationStmt> >> <ab></ab> >> </publicationStmt> >> <sourceDesc> >> <ab></ab> >> </sourceDesc> >> </fileDesc> >> </teiHeader> >> <text> >> <body> >> <div> >> <p>I am foo!</p> >> </div> >> <include xmlns="http://www.w3.org/2001/XInclude" href="bar.xml" >> xpointer="thebar"/> >> </body> >> </text> >> </TEI> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> >> >> and document 'bar.xml' : >> >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> <?xml version="1.0" encoding="UTF-8"?> >> <?xml-model href="http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng" >> type="application/xml" >> schematypens="http://relaxng.org/ns/structure/1.0" ?> >> <TEI xmlns="http://www.tei-c.org/ns/1.0"> >> <teiHeader> >> <fileDesc> >> <titleStmt> >> <title>bar</title> >> </titleStmt> >> <publicationStmt> >> <ab></ab> >> </publicationStmt> >> <sourceDesc> >> <ab></ab> >> </sourceDesc> >> </fileDesc> >> </teiHeader> >> <text> >> <body> >> <div xml:id="thebar"> >> <p>I am bar!</p> >> </div> >> </body> >> </text> >> </TEI> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> >> >> Using the administration interface (http://localhost:8984/dba) >> I create a new databse, add bar.xml and then try to add foo.xml >> This is what I get: >> >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> HTTP ERROR 400 >> >> Problem accessing /dba/add. Reason: >> >> Stopped at /Users/Asxetos/basex/webapp/dba/databases/resources/add.xqm, >> 91/10: >> [FODC0002] "foo.xml" (Line 22): An include with href 'bar.xml'failed, >> and no fallback element was found. >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> >> >> I am guessing that the href path is not correct. >> What should the value of the href attribute be? >> >> >> Thanks, >> Asxetos >> >> >> PS: notice the error message says "with href 'bar.xml'failed". It seems the >> 404 page template is missing a whitespace after the filename.

3 2

java.io.IOException: Stream Closed
by Ketill Fenne 10 Jun '15

10 Jun '15

Hi! >From time to time I see the following error: java.io.IOException: Stream Closed at java.io.RandomAccessFile.seek0(Native Method) at java.io.RandomAccessFile.seek(RandomAccessFile.java:557) at org.basex.io.random.TableDiskAccess.readBlock(TableDiskAccess.java:520) at org.basex.io.random.TableDiskAccess.readPage(TableDiskAccess.java:485) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:464) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:148) at org.basex.data.Data.kind(Data.java:307) at org.basex.query.value.node.DBNode$4.next(DBNode.java:349) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:58) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:36) at org.basex.query.expr.gflwor.For$1.next(For.java:79) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:95) at org.basex.query.MainModule$1.next(MainModule.java:114) at org.basex.server.ServerQuery.execute(ServerQuery.java:130) at org.basex.api.client.LocalQuery.cache(LocalQuery.java:48) at org.basex.api.client.Query.more(Query.java:76) An example of how I query and iterate: DBResult dbResult = null; LocalQuery results = null; try { xPath = xPath.indexOf("/") == -1 ? "/" + xPath : xPath; String myQuery = "for $doc in (" + getContentCollectionQuery(PPSCONTENT_DATABASE_NAME) + " ) " + xPath + createNameCondition(name, PPSCONTENT_DATABASE_NAME) + createDateCondition(version) + " " + " return (data($doc/topic/@id))"; dbResult = runQuery(myQuery); results = dbResult.getXmlResults(); String topic = null; if (results.more()) { topic = results.next(); } return topic; } catch (QueryException e) { PPLogger.getInstance().error("Error in getTopicForDocWithName: " + e.getMessage()); return null; } catch (IOException e) { PPLogger.getInstance().error("Error in getTopicForDocWithName: " + e.getMessage()); return null; } finally { if (results != null) results.close(); if (dbResult != null) dbResult.close(); } The error occurs randomly, but is always "triggered" by "results.more()" and I am not able to reproduce it at will. Is there something wrong or missing in how I iterate the results? I am using BaseX8.2.1 on windows 7. Kind regards Ketill Fenne

2 2

DBA and long-running queries
by Kristian Kankainen 10 Jun '15

10 Jun '15

I want to add a big (150mb) xml file as a resource to my database. The machine is behind a reverse proxy, so the easiest way would be through the DBA. I have the file locally on the same machine I run Tomcat on, but when I run the db:add() query, I get a timeout (504 Gateway Time-out 504 Gateway Time-out nginx/1.2.1). Is there a way of circumventing this? Perhaps instead of running the db:add() straight from the DBA, I should have it as a function in my XQrepo and trigger it somehow? Kristian

2 1

Doc mistake for map:get() for NaN key
by Amanda Galtman 09 Jun '15

09 Jun '15

Hi, I think the following is a mistake in the doc, but before changing the wiki, I wanted to double-check. The documentation for map:get says, "If the supplied key is the xs:float or xs:double value NaN, the function returns an empty sequence." BaseX 8.1 returns the value that corresponds to the NaN key (if present). The software behavior seems correct to me. For example: INPUT let $mymap := map:entry(xs:double('NaN'),'abc') return map:get($mymap,xs:double('NaN')) OUTPUT abc Thanks, Amanda

2 1

BaseX 8.2.1
by Christian Grün 09 Jun '15

09 Jun '15

Dear all, BaseX 8.2.1 has been released today. It contains bug fixes and, additionally, some new DBA features: - Code highlighting (thanks, James Ball!) - New panel for up- and downloading files - Queries and files are now stored in a temporary directory You can also test some of the features that will officially be available with 8.3, such as RelaxNG validation (see http://docs.basex.org/index.php?search=8.3 for more information). Have fun, Christian

1 0

Function to throw an exception
by cmarchand＠oxiane.com 09 Jun '15

09 Jun '15

Hello, Before doing some updates, I want to check many things. If those checks fail, I want to throw an error, to allow the calling program be notified of the error. I do not want to avoid the updates, but really want to throw an error. Is there such a function ? Christophe

2 2