June 2011 - BaseX-Talk - mailman.uni-konstanz.de

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

deadlock in BaseXServer 6.2.7
by Godmar Back 23 Jun '12

23 Jun '12

Hi, we're observing that our BaseXServer 6.2.7 deadlocks. The stack traces are here: http://top.cs.vt.edu/~gback/bx/deadlock-6.2.7/basex-deadlock.txt Could you take a look to see if these ring any bells? Has this problem been addressed in a more recent version? - Godmar

3 5

Problems in using BaseX REST server - IncompatibleClassChangeError javax.ws.rs.core.Response$Status does not implement the requested interface
by Pushkaraj Thorat 21 Oct '11

21 Oct '11

Hello, I am doing setup for REST enabled basex database with procedure mentioned in http://docs.basex.org/wiki/JAX-RX_API URL. Configuring & starting server and running failed due to class not found, but later I've modified the ./basexrest script as: (Included all the jars in lib folder) ######START #!/bin/bash # Path to this script PWD=`dirname $0` # Paths to distributed files or source directories BASEX=$PWD/BaseX.jar BASEXAPI=$PWD/../lib/basex-api.jar # Classpath LIB=$PWD/../lib LIB_CP="" for JAR_FILE in $LIB/*.jar do LIB_CP="$LIB/$JAR_FILE:$LIB_CP" done CP=$BASEX:$BASEXAPI:$LIB_CP # Options for virtual machine VM=-Xmx1g # Run REST server java -cp "$CP" $VM org.basex.api.jaxrx.JaxRxServer "$@" ######END after this, server started successfully, hit the URL - http://localhost:8984/basex/jax-rx I am getting following exception: java.lang.IncompatibleClassChangeError: Class javax.ws.rs.core.Response$Status does not implement the requested interface javax.ws.rs.core.Response$StatusType at com.sun.jersey.spi.container.ContainerResponse.getStatus(ContainerResponse.java:548) at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.commitWrite(ContainerResponse.java:156) at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:133) at org.basex.io.BufferedOutput.flush(BufferedOutput.java:50) at org.basex.io.PrintOutput.flush(PrintOutput.java:126) at org.basex.data.XMLSerializer.cls(XMLSerializer.java:206) at org.basex.data.Serializer.close(Serializer.java:219) at org.basex.api.jaxrx.BXJaxRx$1.code(BXJaxRx.java:96) at org.basex.api.jaxrx.BXCode.run(BXCode.java:53) at org.basex.api.jaxrx.BXOutput.write(BXOutput.java:53) at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71) at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57) at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:299) Your help is appreciated. Thanks, Pushkaraj

3 6

(Attribute with) namespace problem.
by Huib Verweij 14 Jul '11

14 Jul '11

Hi guys, I'm having a weird thing with BaseX, it's probably something I do wrong, but I can't find what it is so I am asking your help. I am PUTting a XML document to the JAX-RX server: ================================================ PUT /basex/jax-rx/lexus/2e7a78c8-fdce-488b-993e-019ad812bc04 HTTP/1.1 Content-Type: text/xml; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= User-Agent: Jakarta Commons-HttpClient/3.1 Host: localhost:8489 Content-Length: 699099 <?xml version="1.0" encoding="UTF-8"?> <lexus id="uuid:2e7a78c8-fdce-4993e-019ad812bc04"> <meta version="1.0"> <name>Lexicon</name> <description>lexicon</description> <note>[This lexicon was imported by demo on 2011-06-24T12:48:02.194+02:00.] </note> <owner ref="uuid:2c9090a20ad6bf2e009f096563"/> <users> <user ref="uuid:2c9090a20ad6bf2e01f096563"> <permissions> <read>true</read> <write>true</write> </permissions> </user> </users> <schema> <container admin-info="" description="" id="MmM5MDkwYTIxNjdjMjFkNzAxMTY4MDQ2OTg5MTRhMDE=" mandatory="true" multiple="false" name="lexicon" note="" type="lexicon"> <container admin-info="" description="Represents a word, a multi-word expression, or an affix in a given language" id="uuid:5f5be48f-089b-4da3-9493-ea302b082fb7" note="" type="lexical-entry"> <container admin-info="" description="" id="uuid:09775cfd-acaf-46cd-98f8-ca878386d133" mandatory="false" multiple="true" name="lexemeGroup" note="" type="container"> <container xmlns:dcr="http://www.isocat.org/ns/dcr" admin-info="http://www.isocat.org/datcat/DC-1325" dcr:datcat="http://www.isocat.org/datcat/DC-1325" description="Minimal unit of language which : has a semantic interpretation and embodies a distinct cultural concept. " id="uuid:a8eb40c4-2068-462d-946b-031a74202a68" mandatory="false" multiple="true" name="lexeme" note="" type="data"/> ================================================ etc. Note the xmlns:dcr declaration. (I am using tcpmon to see what's sent and received. So tcpmon is sitting just in between my app and the JAX-RX server.) When I request the document I am getting something I do not expect, the xmlns:dcr seems to be replaced with a xmlns:xhtml and therefor I receive an error when processing the jax-rx result: The query: ================================================ POST /basex/jax-rx/ HTTP/1.1 Content-Type: application/query+xml; charset=UTF-8 Authorization: Basic YWRtaW46YWRtaW4= User-Agent: Jakarta Commons-HttpClient/3.1 Host: localhost:8489 Content-Length: 673 <?xml version="1.0" encoding="UTF-8"?> <query> <text> (: Returns lexus/meta element :) element lexus { attribute id { 'uuid:2e7a78c8-fdce-4993e-019ad812bc04' }, collection('lexus')/lexus[@id eq 'uuid:2e7a78c8-fdce-4993e-019ad812bc04']/meta } </text> <parameter name="output" value="media-type=text/xml; omit-xml-declaration=no"/> </query> ================================================ and the result: ================================================ HTTP/1.1 200 OK Content-Type: text/xml;omit-xml-declaration=no Content-Length: 15285 Server: Jetty(6.1.25) <jax-rx:results xmlns:jax-rx="http://jax-rx.sourceforge.net"><jax-rx:result><lexus id="uuid:2e7a78c8-fdce-4993e-019ad812bc04"> <meta version="1.0"> <name>Lexicon</name> <description>lexicon</description> <note>[This lexicon was imported by demo on 2011-06-24T12:48:02.194+02:00.]</note> <owner ref="uuid:2c9090a20ad6bf2e010b22509f096563"/> <users> <user ref="uuid:2c9090a20ad6bf2e010b22509f096563"> <permissions> <read>true</read> <write>true</write> </permissions> </user> </users> <schema> <container admin-info="" description="" id="MmM5MDkwYTIxNjdjMjFkNzAxMTY4MDQ2OTg5MTRhMDE=" mandatory="true" multiple="false" name="lexicon" note="" type="lexicon"> <container admin-info="" description="Represents a word, a multi-word expression, or an affix in a given language" id="uuid:5f5be48f-089b-4da3-9493-ea302b082fb7" note="" type="lexical-entry"> <container admin-info="" description="" id="uuid:09775cfd-acaf-46cd-98f8-ca878386d133" mandatory="false" multiple="true" name="lexemeGroup" note="" type="container"> <container xmlns:xhtml="http://www.w3.org/1999/xhtml" admin-info="http://www.isocat.org/datcat/DC-1325" dcr:datcat="http://www.isocat.org/datcat/DC-1325" description="Minimal unit of language which : has a semantic interpretation and embodies a distinct cultural concept. " id="uuid:a8eb40c4-2068-462d-946b-031a74202a68" mandatory="false" multiple="true" name="lexeme" note="" type="data"/> ================================================ I vaguely remember having seen this before, but I can't find it in the mailing list archive, so sorry if I missed an obvious answer, and please help me anyway. Hartelijke groet, Huib Verweij.

3 5

Release BaseX 6.7
by Christian Grün 30 Jun '11

30 Jun '11

Hi all, once more, we are glad to announce a new version of BaseX. Beside continuous tweaks, fixes, minor changes and alignments with the XQuery 3.0 specification, it offers the following enhancements: Main Features: [ADD] Native EXPath Packaging system support: http://docs.basex.org/wiki/Packaging [ADD] Client/server event notification: http://docs.basex.org/wiki/Events [ADD] Persistent document index added to speed up access to large collections XQuery: [ADD] New database and full-text functions: http://docs.basex.org/wiki/Database_Functions http://docs.basex.org/wiki/Full-Text_Functions [ADD] Event function added to fire events [MOD] Index optimizations, better cost estimations Commands: [ADD] Glob syntax added to database commands: http://docs.basex.org/wiki/Commands [ADD] New commands added: REPLACE, RENAME, REPO DELETE/INSTALL/LIST, CREATE/DROP EVENT [MOD] BACKUP optimized, renamed to CREATE BACKUP Next, beside our local trainings, we are also offering a two- and three-day course on XML, XQuery and BaseX now in Germany's renowned LinuxHotel: http://www.linuxhotel.de/kurs/xquery/basex. We are looking forward to your feedback, have fun, Christian, BaseX Team

1 0

BaseX on Wikipedia
by Michael Seiferle 30 Jun '11

30 Jun '11

Dear List, BaseX is now on Wikipedia, we kindly encourage you to extend our article, contribute or provide valuable feedback. You can find the article here: http://en.wikipedia.org/wiki/BaseX If you look hard enough, you might even see a hint for the release date of our upcoming version, now that's peer pressure ;-). Kind regards Michael

1 0

Database format changed?
by Huib Verweij 30 Jun '11

30 Jun '11

Hi, we're getting this error [FODC0004] from our BaseX database out of the blue on our test machine. 14:25:38.236 [127.0.0.1:39513] QUERY(0) declare namespace lexus="http://www.mpi.nl/lat/lexus"; declare function lexus:lexicon($lexus as node()) as node()* { element lexicon { $lexus/lexicon/@*, $lexus/meta } }; let $id := 'uuid:0190305e-730b-4a39-b4c0-2962f09bc01d' let $lexus := collection('lexus')/lexus[@id eq $id] let $schema := $lexus/meta/schema return element result { $schema, lexus:lexicon($lexus) } OK 14:25:38.236 [127.0.0.1:39513] QUERY(0) OK 14:25:38.300 [127.0.0.1:39510] INIT(0) Error: Stopped at line 5, column 48: [FODC0004] The database format has changed; please create a new database. 14:25:38.300 [127.0.0.1:39510] CLOSE(0) OK AFAICT nothing has changed, nobody copied over database files or accessed the database through another interface or something like that. Any ideas on what can cause such a message? When was the last time the format did change? What can we do about it (other than do as it says, assuming it's not actually a feature)? Thanks, Huib Verweij.

3 2

Problems with bigger amounts of data
by Mathias K 28 Jun '11

28 Jun '11

Hello everyone! My name is Mathias. I'm using BaseX for an university project where we are creating a publication database. Right now we have 25 GB xml data spread over 180k documents. Ultimately I want to be able to perform Xquery searches on this data, possibly even full text. I'd like to know whether you think that BaseX is at all suitable for this amount of data. If yes, how would I add these files to the database optimally? If I use the BaseX GUI to add the folder an OutOfMemoryException is produced shortly after starting the process. Even providing more RAM (~7 GB via -Xmx7000M) only delays this. I haven't looked at the code but it appears as though all file contents are stored in RAM and are only written to hard disk at the end, which would at least explain the huge amounts of memory BaseX consumes. Since the GUI can't handle the files I wrote an importer myself which consecutively adds single files via the "Add" command. This seems to work without memory excess. However, it is taking ages to add all 180.000 files this way (several hours, haven't completed it yet). Maybe it's just further delaying the overflow since it's so slow. Also, this might just be my subjective feeling, but adding files seems to get slower as the database grows. Is there some kind of duplicate check going an that could be in the way? If yes, is there a way just to bulk insert all the data I got without checks? I'd be grateful for any thoughts on this! Thanks in advance, Mathias

2 4

Packaging API in BaseX
by Christian Grün 25 Jun '11

25 Jun '11

Dear all, we'd like to give you a preview on our upcoming EXPath Packaging API implementation, which will be released together with BaseX 6.7 next week: http://docs.basex.org/wiki/Packaging_API If you are curious enough, feel free to try our latest stable snapshot: http://files.basex.org/releases/latest/ Two example archives are found at: http://files.basex.org/xar/ Your feedback is valuable to us. If you come across any unexpected behavior, or if you have problems to get it work (..no matter if it's up to you or us..), please give us a note! Regarding the different versions of the Packaging Spec: we decided to only support the latest version (Candidate Module 11 November 2010, http://expath.org/spec/pkg) which means that out-of-dated xar archives need first to be updated. Have fun, Christian ___________________________ BaseX Team Christian Grün Uni KN, Box 188 78457 Konstanz, Germany http://www.basex.org

2 1

Running XQuery in oxygen or Eclipse with default (opened) collection
by Jan Vlčinský (CAD) 24 Jun '11

24 Jun '11

Hi I am currently trying to learn, how to use Oxygen and/or Eclipse to develop XQueries. For installation I followed just the instructions form wiki http://docs.basex.org/wiki/Integrating_oXygen and http://docs.basex.org/wiki/Integrating_Eclipse With Eclipse it worked for me a bit easier, the only problem I had was to specify in the code, what collection I am going to use as default one. My wish is that my xquery code does not have to refer explicitly to single collecion name (configuring collection name around the xquery is fine, but I want prevent having it in the xquery code itself). I did not find any method how to do this. With using explicit collection name in the xquery code it works well. I know I could employ some external parameter and use it to bear the name of the collection, but this is still not, what I would call "use the currently opened collection" and I do not like the fact, my code has to be modified for this purpose. With oxygen is the situation the same with the complication, it keeps asking for xml input, when I use XQuery Transformation - and that is not, what I need. Any advise? Jan *Ing. Jan Vlčinský* CAD programy Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com http://cz.linkedin.com/in/vlcinsky

2 2