April 2011 - BaseX-Talk - mailman.uni-konstanz.de

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

deadlock in BaseXServer 6.2.7
by Godmar Back 23 Jun '12

23 Jun '12

Hi, we're observing that our BaseXServer 6.2.7 deadlocks. The stack traces are here: http://top.cs.vt.edu/~gback/bx/deadlock-6.2.7/basex-deadlock.txt Could you take a look to see if these ring any bells? Has this problem been addressed in a more recent version? - Godmar

3 5

Problems in using BaseX REST server - IncompatibleClassChangeError javax.ws.rs.core.Response$Status does not implement the requested interface
by Pushkaraj Thorat 21 Oct '11

21 Oct '11

Hello, I am doing setup for REST enabled basex database with procedure mentioned in http://docs.basex.org/wiki/JAX-RX_API URL. Configuring & starting server and running failed due to class not found, but later I've modified the ./basexrest script as: (Included all the jars in lib folder) ######START #!/bin/bash # Path to this script PWD=`dirname $0` # Paths to distributed files or source directories BASEX=$PWD/BaseX.jar BASEXAPI=$PWD/../lib/basex-api.jar # Classpath LIB=$PWD/../lib LIB_CP="" for JAR_FILE in $LIB/*.jar do LIB_CP="$LIB/$JAR_FILE:$LIB_CP" done CP=$BASEX:$BASEXAPI:$LIB_CP # Options for virtual machine VM=-Xmx1g # Run REST server java -cp "$CP" $VM org.basex.api.jaxrx.JaxRxServer "$@" ######END after this, server started successfully, hit the URL - http://localhost:8984/basex/jax-rx I am getting following exception: java.lang.IncompatibleClassChangeError: Class javax.ws.rs.core.Response$Status does not implement the requested interface javax.ws.rs.core.Response$StatusType at com.sun.jersey.spi.container.ContainerResponse.getStatus(ContainerResponse.java:548) at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.commitWrite(ContainerResponse.java:156) at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:133) at org.basex.io.BufferedOutput.flush(BufferedOutput.java:50) at org.basex.io.PrintOutput.flush(PrintOutput.java:126) at org.basex.data.XMLSerializer.cls(XMLSerializer.java:206) at org.basex.data.Serializer.close(Serializer.java:219) at org.basex.api.jaxrx.BXJaxRx$1.code(BXJaxRx.java:96) at org.basex.api.jaxrx.BXCode.run(BXCode.java:53) at org.basex.api.jaxrx.BXOutput.write(BXOutput.java:53) at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71) at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57) at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:299) Your help is appreciated. Thanks, Pushkaraj

3 6

adding zip contents – bug or feature?
by Imsieke, Gerrit, le-tex 25 May '11

25 May '11

When I add a directory to a database in 6.6.1 using the plain *.xml createfilter, and when there's a zip file below this directory, the zip file's contents will be indexed, too, as if they reside in the server's working directory. If this is a feature, how can I switch it off? Gerrit

4 14

Re: [basex-talk] Renaming of documents
by Leo Wörteler 18 May '11

18 May '11

Hi Thomas, I could reproduce the bug and we'll fix it soon. Your query however looks a bit strange to me: Am 21.04.2011 14:31, schrieb Thomas Kaltofen: > insert node collection('db/docb')/rootB > after collection('db/doca')/rootA Your collection looks something like this: > collection(db) > | > |-- document(doca) > | | > | `-- <rootA .../> > | > `-- document(docb) > | > `-- <rootB .../> Your query doesn't create a new document node (as XQuery Update doesn't operate on document level) but inserts the <rootB .../> element directly into the first document, which then has two(!) root elements: > collection(db) > | > |-- document(doca) > | | > | |-- <rootA .../> > | | > | `-- <rootB .../> > [...] That's allowed in XQuery Update, but isn't valid XML any more and I can't think of a scenario where it's actually desirable. Is that what you wanted to do? If not, could you explain in more detail what you intended? Cheers Leo

3 3

Possible to store query on the server side, but payload (XML data) in HTTP post, using jax-rx?
by Computer Software Developer 09 May '11

09 May '11

Hi, Using the run parameter in a jax-rx request, one can specify a query contained in a file stored on the server side. Using HTTP Post, one can specify the xquery and the payload in the HTTP request as XML. If the XML is generated on the client side to be stored on the server using HTTP post, can one execute an updating xquery stored on the server side to store this generated XML on the server? In other words, (how) can one run xqueries stored on the server side with an XML payload submitted using the HTTP post? Thanks.

2 10

Re: [basex-talk] What happened to the DeepFS menu? Was it removed?
by John D. Mitchell 08 May '11

08 May '11

Howdy, Just upgraded to the latest Base-x v6.6.1 and the create command no longer accepts "fs" as one of the types. I've been using DeepFS and so this is a bit of an issue. :-) Note also that "mab" is listed in the auto-complete list but is also excluded. Thanks, John

5 4

collections in 6.5.1
by C. M. Sperberg-McQueen 05 May '11

05 May '11

Thanks to Christian Grün's prompt response to my question about attributes, I upgraded to Basex 6.5.1 the other day. And I've run into an unexpected behavior. I have several versions of the Unicode database in XML (the Unicode consortium started shipping XML versions with 5.1.0, and I've created XML documents with the information I need for all the earlier versions); they are all in directory ~/2011/Unicode. But when I ask to create a new database in the GUI, giving that directory as the path and accepting the default pattern of *.xml, the only document in the resulting database appears to be the small schemas.xml file that nXML mode placed in one of the subdirectories of ~/2011/Unicode when I edited an XSLT stylesheet there. What I was expecting was that all the XML documents in that subtree of the file directory would be added -- I think that was the behavior in earlier versions. Has something changed? Or have I gotten out of practice and done something wrong? I hope this is clear; I can try to explain further if needed. Thanks! -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

3 6

Clustering of BaseX servers
by Erol Akarsu 29 Apr '11

29 Apr '11

Hi, Have we thought on clustering basex servers so we can partition xml documents? Here, I am only interested in a global partitioning: Let's say if we have xml document like this, I would like to partition RECORDS content so that each host will have equal number of RECORD elements. Then, we need to aggregate results of individual hosts. Can we implement this simple clustering framework with Basex? <RECORDS> <RECORD> ..... </RECORD> <RECORD> ..... </RECORD> </RECORDS> On Mon, Apr 11, 2011 at 12:35 PM, Erol Akarsu <eakarsu(a)gmail.com> wrote: > Ok, > > I was able to run full text search with another XML Database. > > I am primarily interested in how Basex will play with big XML file > Wikipedia. > > Actually, Database create of wikipedia is fine. But when I add full tet > search and indexes for it, it always throws out of memory exception error. > > I have changed -Xmx with 6GB that is still not enough to generate indexes > for Wikipedia. > > Can you help me on how to generate indexes with a machine that case 6GB > for Basex process? > > Thanks > > Erol Akarsu > > > > On Sun, Apr 10, 2011 at 4:07 AM, Andreas Weiler < > andreas.weiler(a)uni-konstanz.de> wrote: > >> Hi, >> >> the following query could work for you: >> >> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> for $i in doc("enwiki-latest-pages-articles")//w:sitename >> return $i[. contains text "Wikipedia"]/.. >> >> -- Andreas >> >> Am 09.04.2011 um 20:43 schrieb Erol Akarsu: >> >> Hi >> >> I am having difficulty in running full text operators. This script gives >> siteinfo below >> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> let $d := fn:doc ("enwiki-latest-pages-articles") >> return ($d//w:siteinfo)[1] >> >> But return $d//w:siteinfo[w:sitename contains text 'Wikipedia'] does NOT >> give same node >> Why "contains" ft operator behave incorrectly? I remember it was working >> fine. I just dropped and recreated database and turn all indexes. Can you >> help me? >> Query info is here: >> >> Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> Compiling: >> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >> - adding text() step >> - optimizing descendant-or-self step(s) >> - removing path with no index results >> - pre-evaluating (())[1] >> - binding static variable $res >> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >> - binding static variable $d >> - adding text() step >> - optimizing descendant-or-self step(s) >> - removing path with no index results >> - simplifying flwor expression >> Result: () >> Timing: >> - Parsing: 0.46 ms >> - Compiling: 0.42 ms >> - Evaluating: 0.17 ms >> - Printing: 0.1 ms >> - Total Time: 1.15 ms >> Query plan: >> <sequence size="0"/> >> >> >> >> <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" >> http://www.w3.org/2001/XMLSchema-instance"> >> <sitename>Wikipedia</sitename> >> <base>http://en.wikipedia.org/wiki/Main_Page</base> >> <generator>MediaWiki 1.17wmf1</generator> >> <case>first-letter</case> >> <namespaces> >> <namespace key="-2" case="first-letter">Media</namespace> >> <namespace key="-1" case="first-letter">Special</namespace> >> <namespace key="0" case="first-letter"/> >> <namespace key="1" case="first-letter">Talk</namespace> >> <namespace key="2" case="first-letter">User</namespace> >> <namespace key="3" case="first-letter">User talk</namespace> >> <namespace key="4" case="first-letter">Wikipedia</namespace> >> <namespace key="5" case="first-letter">Wikipedia talk</namespace> >> <namespace key="6" case="first-letter">File</namespace> >> <namespace key="7" case="first-letter">File talk</namespace> >> <namespace key="8" case="first-letter">MediaWiki</namespace> >> <namespace key="9" case="first-letter">MediaWiki talk</namespace> >> <namespace key="10" case="first-letter">Template</namespace> >> <namespace key="11" case="first-letter">Template talk</namespace> >> <namespace key="12" case="first-letter">Help</namespace> >> <namespace key="13" case="first-letter">Help talk</namespace> >> <namespace key="14" case="first-letter">Category</namespace> >> <namespace key="15" case="first-letter">Category talk</namespace> >> <namespace key="100" case="first-letter">Portal</namespace> >> <namespace key="101" case="first-letter">Portal talk</namespace> >> <namespace key="108" case="first-letter">Book</namespace> >> <namespace key="109" case="first-letter">Book talk</namespace> >> </namespaces> >> </siteinfo> >> >> On Mon, Apr 4, 2011 at 7:31 AM, Erol Akarsu <eakarsu(a)gmail.com> wrote: >> >>> I imported wikipedia xml into basex and tried to search it. >>> >>> But searching it takes longer. >>> >>> I tried to search one element that is first child of whole document and >>> it took 52 sec. >>> I know the XML file is very big 31GB. How can I optimize the search? >>> >>> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >>> >>> let $d := fn:doc ("enwiki-latest-pages-articles")//w:siteinfo >>> return $d >>> >>> Database info: >>> >>> > open enwiki-latest-pages-articles >>> Database 'enwiki-latest-pages-articles' opened in 778.49 ms. >>> > info database >>> Database Properties >>> Name: enwiki-latest-pages-articles >>> Size: 23356 MB >>> Nodes: 228090153 >>> Height: 6 >>> >>> Database Creation >>> Path: /mnt/hgfs/C/tmp/enwiki-latest-pages-articles.xml >>> Time Stamp: 03.04.2011 12:29:15 >>> Input Size: 30025 MB >>> Encoding: UTF-8 >>> Documents: 1 >>> Whitespace Chopping: ON >>> Entity Parsing: OFF >>> >>> Indexes >>> Up-to-date: true >>> Path Summary: ON >>> Text Index: ON >>> Attribute Index: ON >>> Full-Text Index: OFF >>> > >>> >>> >>> Timing info: >>> >>> Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >>> Compiling: >>> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >>> - optimizing descendant-or-self step(s) >>> - binding static variable $d >>> - removing variable $d >>> - simplifying flwor expression >>> Result: element siteinfo { ... } >>> Timing: >>> - Parsing: 1.4 ms >>> - Compiling: 52599.0 ms >>> - Evaluating: 0.28 ms >>> - Printing: 0.62 ms >>> - Total Time: 52601.32 ms >>> Query plan: >>> <DBNode name="enwiki-latest-pages-articles" pre="5"/> >>> >>> >>> >>> Result of query: >>> >>> <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" >>> http://www.w3.org/2001/XMLSchema-instance"> >>> <sitename>Wikipedia</sitename> >>> <base>http://en.wikipedia.org/wiki/Main_Page</base> >>> <generator>MediaWiki 1.17wmf1</generator> >>> <case>first-letter</case> >>> <namespaces> >>> <namespace key="-2" case="first-letter">Media</namespace> >>> <namespace key="-1" case="first-letter">Special</namespace> >>> <namespace key="0" case="first-letter"/> >>> <namespace key="1" case="first-letter">Talk</namespace> >>> <namespace key="2" case="first-letter">User</namespace> >>> <namespace key="3" case="first-letter">User talk</namespace> >>> <namespace key="4" case="first-letter">Wikipedia</namespace> >>> <namespace key="5" case="first-letter">Wikipedia talk</namespace> >>> <namespace key="6" case="first-letter">File</namespace> >>> <namespace key="7" case="first-letter">File talk</namespace> >>> <namespace key="8" case="first-letter">MediaWiki</namespace> >>> <namespace key="9" case="first-letter">MediaWiki talk</namespace> >>> <namespace key="10" case="first-letter">Template</namespace> >>> <namespace key="11" case="first-letter">Template talk</namespace> >>> <namespace key="12" case="first-letter">Help</namespace> >>> <namespace key="13" case="first-letter">Help talk</namespace> >>> <namespace key="14" case="first-letter">Category</namespace> >>> <namespace key="15" case="first-letter">Category talk</namespace> >>> <namespace key="100" case="first-letter">Portal</namespace> >>> <namespace key="101" case="first-letter">Portal talk</namespace> >>> <namespace key="108" case="first-letter">Book</namespace> >>> <namespace key="109" case="first-letter">Book talk</namespace> >>> </namespaces> >>> </siteinfo> >>> >>> >>> >>> Thanks >>> >>> Erol Akarsu >>> >>> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> >

1 0

Converting String to node inside XQuery
by Sony Vijay 26 Apr '11

26 Apr '11

Hi, I am wondering if there is a way to convert a String into a Node within XQuery. For example, consider the xquery let $node := copy $c := <entry> <title>entry title</title> <content>entry content</content> </entry> modify ( return replace value of node $entry/title with 'new entry title', return replace value of node $entry/content with 'new entry content' ) return $c return replace node doc('doc1')/entry with $c The above xquery throws the error: [XPTY0019] Context node required for entry; xs:string found. How do I fix this ? Thanks, Sony

3 8