November 2011 - BaseX-Talk - mailman.uni-konstanz.de

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

deadlock in BaseXServer 6.2.7
by Godmar Back 23 Jun '12

23 Jun '12

Hi, we're observing that our BaseXServer 6.2.7 deadlocks. The stack traces are here: http://top.cs.vt.edu/~gback/bx/deadlock-6.2.7/basex-deadlock.txt Could you take a look to see if these ring any bells? Has this problem been addressed in a more recent version? - Godmar

3 5

Whatever happened to DeepFS
by Andy Bunce 13 Dec '11

13 Dec '11

Hi, I was looking for this feature in version 7. https://mailman.uni-konstanz.de/pipermail/basex-talk/2011-January/000994.ht… says it was temporarily removed from the GUI but create fs [name] [path] gives an error in 7.0.1. Is this feature gone for good now? /Andy

5 8

cdata-section-elements collisions
by jidanni＠jidanni.org 04 Dec '11

04 Dec '11

declare option output:cdata-section-elements "description"; works great, except that I want it to work only on my //Folder/description's not my //Placemark/description's. I read http://www.w3.org/TR/2010/REC-xslt-xquery-serialization-20101214/index.html… and http://docs.basex.org/wiki/Serialization and no matter what I do...

2 2

namespace limit?
by C. M. Sperberg-McQueen 01 Dec '11

01 Dec '11

Greetings! I've just downloaded BaseX 7.0.2, and was very happy to see the checkbox for "Skip corrupt (non-well-formed) files", since it makes it much much easier to create a BaseX database of the XSD test suite. But my attempt to build the database still fails with the message "idI001.xsd" (Line 2): Too many different namespaces (limit: 256). Since document idI001.xsd only declares 5 namespaces, I guess the 256-namespace limit is for the database, not for an individual document. Is there any way to ease the limit, or do I have to contemplate making multiple databases, each covering just part of the test suite? Thank you! Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

3 7

Re: [basex-talk] ajax and http server - cross-domain, jsonp or proxy?
by Maud Ingarao 30 Nov '11

30 Nov '11

Hello, The support of jsonp would be very welcomed here, since several projects on tge-adonis use or will use the basex.tge-adonis.fr/rest service... For now, I went on with the proxy solution : http://hyperdonat.tge-adonis.fr/basex/basex.html Cheers, Maud On Sat, Oct 22, 2011 at 12:07 AM, < basex-talk-request(a)mailman.uni-konstanz.de> wrote: > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. Re: Performance of Basex 7.0 XQuery execution (Leo W?rteler) > 2. Re: Problems in using BaseX REST server - > IncompatibleClassChangeError javax.ws.rs.core.Response$Status > does not implement the requested interface (Pushkaraj Thorat) > 3. Re: ajax and http server - cross-domain, jsonp or proxy? > (Godmar Back) > 4. Re: ajax and http server - cross-domain, jsonp or proxy? > (Michael Seiferle) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 21 Oct 2011 14:12:50 +0200 > From: Leo W?rteler <lw(a)basex.org> > To: Constantine Hondros <Constantine.Hondros(a)tomtom.com> > Cc: basex-talk <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Performance of Basex 7.0 XQuery execution > Message-ID: <4EA161C2.3060908(a)basex.org> > Content-Type: text/plain; charset=UTF-8; format=flowed > > Dear Constantine, > > Am 20.10.2011 09:44, schrieb Constantine Hondros: > > Of course, I am really interested in more useful queries such as: > > > > //tns:poi[not(tns:id = preceding-sibling::tns:poi/tns:id)] > > this query basically compiles to two nested loops, the outer one > iterating over //tns:poi and the inner one over all preceding-siblings. > So it has an O(n^2) runtime. > > > I guess what I am asking is ... while most XPath queries I run are quick > to resolve, anything using preceding::, preceding-sibling:: or following::, > following-sibling:: essentially runs too slowly to be workable - at least, > on my sample dataset with about 100K sibling children of the root element. > > Well, that's an extreme example, 100'000^2 is a huge number of nodes to > traverse. I performed some tests with > a) 10'000 sibling elements inside the root: > <root> > <item id="1"/> > <item id="2"/> > [...] > <item id="10000"/> > </root> > > and the query //item[not(preceding-sibling::*)] > > b) 10'000 nested elements: > <root> > <item id="1"> > <item id="2"> > [...] > <item id="10000"/> > [...] > </item> > </item> > </root> > > and the query //item[not(ancestor::*)] > > The timings I get are almost identical, replacing "preceding" with > "following" or "ancestor" with "descendant" doesn't change much. > > > So is this any sort of known issue? Should I be looking at any sort of > configuration options? Is there any way to profile the execution of the > Xpath? > > My guess is that the number of nodes on the *-sibling-axes is usually > much bigger than on the vertical axes, so an expensive algorithm gets > unusable earlier. > > To return to your example query, you could drastically reduce its > runtime by only traversing the database once, comparing each node only > to the nodes with unique tns:ids found before. I'll use a left fold [1][2]: > > fold-left( > function($seq, $x) { > if($seq/tns:id = $x/tns:id) then $seq > else ($seq, $x) > }, > (), > //tns:poi > ) > > If you're OK with using not-yet-standard XQuery extensions, a map [3] > instead of a sequence can potentially be even faster: > > let $map := > fold-left( > function($map, $x) { > let $key := $x/tns:id > return if(map:contains($map, $key)) then $map > else map:new(($map, map:entry($key, $x))) > }, > map:new(), > //tns:poi > ) > for $key in map:keys($map) > return $map($key) > > But this won't preserve the relative ordering of the results. > > Hope that helped, > Cheers Leo > __________ > > [1] http://www.w3.org/TR/xpath-functions-30/#func-fold-left > [2] http://en.wikipedia.org/wiki/Fold_%28higher-order_function%29 > [3] http://docs.basex.org/wiki/Map_Functions > > > ------------------------------ > > Message: 2 > Date: Sat, 22 Oct 2011 01:40:23 +0530 > From: Pushkaraj Thorat <pushkaraj.thorat.devl(a)gmail.com> > To: Rositsa Shadura <rositsa.shadura(a)uni-konstanz.de> > Cc: basex-talk(a)mailman.uni-konstanz.de > Subject: Re: [basex-talk] Problems in using BaseX REST server - > IncompatibleClassChangeError javax.ws.rs.core.Response$Status does > not > implement the requested interface > Message-ID: > <CAH5ngE_gf655mnLuTCAjStd2X1cmWKJQk8xYTErMyFv4QnMYeQ(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > Thanks Rositsa! > It is bit late to reply, but indeed it is a good to know. > > Regards > Pushkaraj > > On Fri, Mar 4, 2011 at 3:01 AM, Rositsa Shadura < > rositsa.shadura(a)uni-konstanz.de> wrote: > > > Hi all, > > > > I've just managed accidentally to reproduce the error. I noticed that the > > class javax.ws.rs.core.Response.**Status is defined in both > > jsr311-api-1.0.jar and jersey-core-1.4.jar but only the class in the > > jersey-core JAR implements the interface > javax.ws.rs.core.Response.**StatusType. > > This is why it is important that the jersey-core JAR appears before the > > jsr311-api JAR in the classpath. When not the error occurs. > > > > Greetings, > > Rositsa > > > > > > On Thu, 20 Jan 2011 12:03:45 +0100, Michael Seiferle < > > michael.seiferle(a)uni-**konstanz.de <michael.seiferle(a)uni-konstanz.de>> > > wrote: > > > > Hi Pushkaraj, > >> > >> thanks in return for your response. > >> I will see if I can reproduce that in my VM; so far you are the first > one > >> to report this issue, I'll let you know what I can find. > >> > >> Kind regards > >> > >> Michael > >> > >> Am 19.01.2011 um 20:19 schrieb Pushkaraj Thorat: > >> > >> Thanks for your response Michael, > >>> > >>> I am using java 1.6 > >>> > >>> pushkaraj@toothless:~$ java -version > >>> java version "1.6.0_17" > >>> Java(TM) SE Runtime Environment (build 1.6.0_17-b04) > >>> Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode) > >>> pushkaraj@toothless:~$ > >>> > >>> Just to update, the rest server ran successfully on my windows machine. > >>> Not sure what problem faced on the linux(ubuntu) machine. I will debug > once > >>> I get some time. > >>> > >>> Thanks, Pushkaraj > >>> > >>> > >>> On Tue, Jan 18, 2011 at 3:46 AM, Michael Seiferle > <michael.seiferle@uni- > >>> **konstanz.de <michael.seiferle(a)uni-konstanz.de>> wrote: > >>> Dear Pushkaraj, > >>> > >>> I am deeply sorry for you having to mess around with that erroneous > >>> startup scripts. > >>> We fixed them in both the ZIP & DMG distribution, feel free to download > >>> them again. > >>> > >>> >> java.lang.**IncompatibleClassChangeError > >>> We have been trying to reproduce this error - but to no avail. > >>> Your error message might have to do something with your Java Version; > >>> please make sure you are running the latest version. > >>> > >>> Hope this helps! In case it does not, feel free to ask for more. > >>> > >>> Kind regards > >>> > >>> Michael > >>> > >>> Am 17.01.2011 um 22:00 schrieb Pushkaraj Thorat: > >>> > >>> > I am doing setup for REST enabled basex database with procedure > >>> mentioned in http://docs.basex.org/wiki/**JAX-RX_API< > http://docs.basex.org/wiki/JAX-RX_API>URL. > >>> > > >>> > Configuring & starting server and running failed due to class not > >>> found, but later I've modified the ./basexrest script as: (Included > all the > >>> jars in lib folder) > >>> > > >>> > ######START > >>> > #!/bin/bash > >>> > # Path to this script > >>> > PWD=`dirname $0` > >>> > # Paths to distributed files or source directories > >>> > BASEX=$PWD/BaseX.jar > >>> > BASEXAPI=$PWD/../lib/basex-**api.jar > >>> > # Classpath > >>> > LIB=$PWD/../lib > >>> > LIB_CP="" > >>> > for JAR_FILE in $LIB/*.jar > >>> > do > >>> > LIB_CP="$LIB/$JAR_FILE:$LIB_**CP" > >>> > done > >>> > CP=$BASEX:$BASEXAPI:$LIB_CP > >>> > # Options for virtual machine > >>> > VM=-Xmx1g > >>> > # Run REST server > >>> > java -cp "$CP" $VM org.basex.api.jaxrx.**JaxRxServer "$@" > >>> > > >>> > ######END > >>> > > >>> > after this, server started successfully, hit the URL - > >>> http://localhost:8984/basex/**jax-rx< > http://localhost:8984/basex/jax-rx>I am getting following exception: > >>> > > >>> > java.lang.**IncompatibleClassChangeError: Class > >>> javax.ws.rs.core.Response$**Status does not implement the requested > >>> interface javax.ws.rs.core.Response$**StatusType > >>> > at > com.sun.jersey.spi.container.**ContainerResponse.getStatus(** > >>> ContainerResponse.java:548) > >>> > at com.sun.jersey.spi.container.**ContainerResponse$** > >>> CommittingOutputStream.**commitWrite(ContainerResponse.**java:156) > >>> > at com.sun.jersey.spi.container.**ContainerResponse$** > >>> CommittingOutputStream.write(**ContainerResponse.java:133) > >>> > at org.basex.io.BufferedOutput.**flush(BufferedOutput.java:50) > >>> > at org.basex.io.PrintOutput.**flush(PrintOutput.java:126) > >>> > at org.basex.data.XMLSerializer.**cls(XMLSerializer.java:206) > >>> > at org.basex.data.Serializer.**close(Serializer.java:219) > >>> > at org.basex.api.jaxrx.BXJaxRx$1.**code(BXJaxRx.java:96) > >>> > at org.basex.api.jaxrx.BXCode.**run(BXCode.java:53) > >>> > at org.basex.api.jaxrx.BXOutput.**write(BXOutput.java:53) > >>> > at com.sun.jersey.core.impl.**provider.entity.** > >>> StreamingOutputProvider.**writeTo(**StreamingOutputProvider.java:**71) > >>> > at com.sun.jersey.core.impl.**provider.entity.** > >>> StreamingOutputProvider.**writeTo(**StreamingOutputProvider.java:**57) > >>> > at com.sun.jersey.spi.container.**ContainerResponse.write(** > >>> ContainerResponse.java:299) > >>> > > >>> > >>> > >>> ______________________________**_________________ > > BaseX-Talk mailing list > > BaseX-Talk(a)mailman.uni-**konstanz.de <BaseX-Talk(a)mailman.uni-konstanz.de > > > > https://mailman.uni-konstanz.**de/mailman/listinfo/basex-talk< > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk> > > >

1 0

Re: [basex-talk] ajax and http server - cross-domain, jsonp or proxy?
by Maud Ingarao 30 Nov '11

30 Nov '11

Hello everybody, I am very new at BaseX http server, and at jQuery, and at this mailing list too :) Concerning the error you got Colin : thrown error: [Exception... "Access to restricted URI denied" code: "1012"... I suddenly have the same problem though basex and my files have the same domain : http://hyperdonat.tge-adonis.fr/basex/basex.html (The basex server is here : http://basex.tge-adonis.fr/rest) I had other problems before this with my script, but not this one, and I don't know what I did that produced it... Maud On Fri, Oct 21, 2011 at 12:00 PM, < basex-talk-request(a)mailman.uni-konstanz.de> wrote: > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. Re: Performance of Basex 7.0 XQuery execution (Christian Gr?n) > 2. Re: Performance of Basex 7.0 XQuery execution (Jan Vl?insk? (CAD)) > 3. Clustering/replicating? (Pascal Heus) > 4. Re: ajax and http server - cross-domain, jsonp or proxy? > (colin mcenearney) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 20 Oct 2011 15:43:50 +0200 > From: Christian Gr?n <christian.gruen(a)gmail.com> > To: Constantine Hondros <Constantine.Hondros(a)tomtom.com> > Cc: "basex-talk(a)mailman.uni-konstanz.de" > <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Performance of Basex 7.0 XQuery execution > Message-ID: > <CAP94bnPYC2abLTiQJkBPt1_WquH6ix8OQeU6HnJtxCa+cp-5Vg(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > Hi Constantine, > > > The XPath I showed was just a simple example of using the > preceding-sibling axis. > > I guessed so already.. Thanks for persisting ;) > > > I guess what I am asking is ... while most XPath queries I run are quick > to resolve, anything using preceding::, preceding-sibling:: or following::, > following-sibling:: essentially runs too slowly to be workable - at least, > on my sample dataset with about 100K sibling children of the root element. > > > > So is this any sort of known issue? Should I be looking at any sort of > configuration options? Is there any way to profile the execution of the > Xpath? > > It's true that BaseX is mostly optimized for descendant and ancestor > operations. In particular, the preceding/preceding-sibling axes will > always be slower, as our storage does not contain backward pointers to > preceding nodes. I'm pretty sure, though, that there are ways to > either optimize relevant queries or the internal optimizer strategies, > but we'll probably have to spend some more time on analyzing the > relevant bottlenecks.. > > Feel free to ask for more, > Christian > > > ------------------------------ > > Message: 2 > Date: Thu, 20 Oct 2011 19:20:29 +0200 > From: Jan Vl?insk? (CAD) <jan.vlcinsky(a)cad-programs.com> > To: Christian Gr?n <christian.gruen(a)gmail.com> > Cc: "basex-talk(a)mailman.uni-konstanz.de" > <basex-talk(a)mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Performance of Basex 7.0 XQuery execution > Message-ID: > <CAAq9E_fb5KymrtL=xosGsH=BuP4aWpgip0OWxDJUAHANqqO1Ng(a)mail.gmail.com > > > Content-Type: text/plain; charset="utf-8" > > Hi Constantine > Just my side note. > As far as I remember, I found this sort of performance discussion with most > the XQuery processors I have tested (Sedna, Zorba, Saxon, eXist...). Each > time related to sibling, preceding etc. and each time explaining, that > indices are not containing pointers for tracking this relationship. My > feeling is, this could lead to much bigger index file and slower > performance > when adding documents and indexing. > My conlcusion is - optimizing this sort of queries has its cost and mostly > it is better (and somehow possible) to avoid these constructions. > > With best regards > > Jan > > *Ing. Jan Vl?insk?* > CAD programy > Slune?nicov? 338/3, 734 01 Karvin? R?j, Czech Republic > tel: +420-597 602 024; mob: +420-608 979 040 > skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com > http://cz.linkedin.com/in/vlcinsky > > > > On 20 October 2011 15:43, Christian Gr?n <christian.gruen(a)gmail.com> > wrote: > > > Hi Constantine, > > > > > The XPath I showed was just a simple example of using the > > preceding-sibling axis. > > > > I guessed so already.. Thanks for persisting ;) > > > > > I guess what I am asking is ... while most XPath queries I run are > quick > > to resolve, anything using preceding::, preceding-sibling:: or > following::, > > following-sibling:: essentially runs too slowly to be workable - at > least, > > on my sample dataset with about 100K sibling children of the root > element. > > > > > > So is this any sort of known issue? Should I be looking at any sort of > > configuration options? Is there any way to profile the execution of the > > Xpath? > > > > It's true that BaseX is mostly optimized for descendant and ancestor > > operations. In particular, the preceding/preceding-sibling axes will > > always be slower, as our storage does not contain backward pointers to > > preceding nodes. I'm pretty sure, though, that there are ways to > > either optimize relevant queries or the internal optimizer strategies, > > but we'll probably have to spend some more time on analyzing the > > relevant bottlenecks.. > > > > Feel free to ask for more, > > Christian > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk(a)mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >

1 1

Query / Iteration on server side.
by Fabian Hickert 29 Nov '11

29 Nov '11

Hi all, I was playing with the python examples of basex version 6.7.1-1 from the debian-wheezy repository. I've used the corresponding BaseXClient.py to establish a connection/add/update/query stuff and so on, it worked very well. Now I've noticed the new 7.0.2 version and now I've got problems with the query/iterate operation. In the 6.7 example one starts a query and iterates over the results, using the more method, which sends \x01 followed by the query-result id. But the examples of 7.0.2 don't behave like this anymore, instead they cache the complete result on the client side. I've tried to rebuild the old 6.7 behaviour with the commands shown here, http://docs.basex.org/wiki/Server_Protocol but I wasn't successful. Is is still possible to query and iterate on the server side instead, of caching the complete result on the client side? Am I doing things the wrong way? If I do, could you please point me to the right direction or examples, documentation? Best regards Fabian -- Fabian Hickert <fabian.hickert(a)GONICUS.de> (System Engineer) * GONICUS GmbH * Moehnestrasse 11-17 * D-59755 Arnsberg * Tel.: +49 (0) 29 32 / 9 16 - 0 * Fax: +49 (0) 29 32 / 9 16 - 242 * http://www.GONICUS.de * http://twitter.com/gonicus *Sitz der Gesellschaft: Moehnestrasse 11-17 * D-59755 Arnsberg *Geschaeftsfuehrer: Rainer Luelsdorf, Alfred Schroeder *Vorsitzender des Beirats: Juergen Michels *Amtsgericht Arnsberg * HRB 1968

2 2

"declare copy-namespaces no-preserve, inherit; " must come first
by jidanni＠jidanni.org 29 Nov '11

29 Nov '11

"declare copy-namespaces no-preserve, inherit;" must come first: $ cat e declare option output:omit-xml-declaration "no"; declare copy-namespaces no-preserve, inherit; <z/> $ basex e Stopped at line 2, column 8 in /tmp/e: [XPST0003] Unexpected end of query: 'copy-namespaces...'. $ cat f declare copy-namespaces no-preserve, inherit; declare option output:omit-xml-declaration "no"; <z/> $ basex f <?xml version="1.0" encoding="UTF-8"?> <z/> $ apt-cache policy basex basex: Installed: 7.0.2-1

2 1

Question about the Fulltext search!
by Truong An Nguyen 29 Nov '11

29 Nov '11

Hi, I've two questions about the fulltext search. 1) I've created fulltext i for my BaseX doesn't understand ftcontains. When I try a CONTAINS query in a 1 Gigabyte Database, using text index is slower than without text index. How can I use fulltext index rightly? 2) Does BaseX supports regular expression in the fulltext extension? If yes, could you please give me an example. Thanks Cheers, An

5 20