October 2010 - BaseX-Talk - mailman.uni-konstanz.de

Out of memory bug
by Ivan Lagunov 31 Jul '13

31 Jul '13

Hello, I've got 'Out of memory' error and corrupted database again, this time it's for 6.3 version. Steps: 1. Started server, started terminal client, checked there are no text and attribute indexes. 2. Started my Java application that runs several tests on database: querying, adding, removing. 3. Then I forgot to reopen terminal client and used the one that I opened in Step 1. 4. Run optimize command for the collection and got error. 5. Cannot open the collection any more as I'm getting 'Out of memory'. It's clear that I shouldn't have used the terminal client that I opened before execution of my Java application. But sometimes it's hard to remember. And it's completely unacceptable that these actions result in the corrupted database. You really should catch ArrayIndexOutOfBoundsException everywhere in order to save database from being corrupted. Terminal client stacktrace: > optimize Possible bug? Feedback is welcome: basex-talk(a)mailman.uni-konstanz.de BaseX 6.3: java.lang.ArrayIndexOutOfBoundsException: -1 org.basex.data.PathSummary.add(PathSummary.java:68) org.basex.core.cmd.Optimize.stats(Optimize.java:68) org.basex.core.cmd.Optimize.run(Optimize.java:34) org.basex.core.Command.run(Command.java:236) org.basex.core.Command.exec(Command.java:218) org.basex.core.Command.execute(Command.java:66) org.basex.server.ServerProcess.run(ServerProcess.java:161) > open products Out of Main Memory. Best regards, Ivan

4 8

Re: [basex-talk] REST questions
by Christian Grün 31 Oct '10

31 Oct '10

Hi Nigel, thanks for the detailed wish list. First of all, I'd advise to follow the steps described at http://basex.org/rest. This tutorial seems to be well accepted by our users. As you might have seen, it's still based on version 6.1, so it will be updated with the next official release. > 1. Start the REST service. > ========================== > > When I start the BaseX server by doing; > > C:\>java -cp basex-6.3.jar org.basex.BaseXServer -s > > does that also start the REST service? No; the Jetty server and some other external libraries are needed to run the REST service; these aren't included in the default distribution of BaseX (which doesn't rely on any external packages), but found in the basex-api.zip package instead. We'll soon automatize the process of including a runnable REST/JaxRx version in each minor release. > 2. Test the rest service. > ========================= > > How can I test if the REST service is running on localhost? Pointing a > browser at http://localhost:8984/basex/jax-rx/ returns the following error; If you follow the tutorial, you shouldn't encounter any problems; just tell us if you need an updated, precompiled version of the basex-api.zip distribution package. Note, next, that all browsers will give you different results. Some browsers will display the result as XML, whereas others (such as Chrome) doesn't show anything. You might try tools like 'curl' to get all output printed on screen. > Connection refused: connect This indicates that the web server hasn't been started; see 1. > 3. Using REST. > ============== > How do I create a restful interface? Say I want to create a REST API that > simply does the following; > > An HTTP GET to locahost:8984/basex/jax-rx/foo/ returns <hello/> > An HTTP GET to locahost:8984/basex/jax-rx/bar/ returns <world/> Hm, I'm not sure what you want to do here. If your database "foo" includes the document "<hello/>", you'll indeed get this result. Note that the next version of REST/JaxRx, on which we're currently working, will support hierarchical paths; i.e., you'll be able to watch parts of a database by specifying the correspondent collection path. An example: locahost:8984/basex/jax-rx/YourDB/path/to/relevant/documents?query=query/to/be/executed Hope thie helps, Christian

1 0

REST questions
by Nigel Alderton 31 Oct '10

31 Oct '10

I'm hoping to find a step-by-step 'hello world' example for using the BaseX REST interface. To clarify, I want to be able to: 1. Start the REST service. ========================== When I start the BaseX server by doing; C:\>java -cp basex-6.3.jar org.basex.BaseXServer -s does that also start the REST service? 2. Test the rest service. ========================= How can I test if the REST service is running on localhost? Pointing a browser at http://localhost:8984/basex/jax-rx/ returns the following error; error on line 1 at column 1: Document is empty If I 'View Source' for that page in the browser, the source is just; Connection refused: connect 3. Using REST. ============== How do I create a restful interface? Say I want to create a REST API that simply does the following; An HTTP GET to locahost:8984/basex/jax-rx/foo/ returns <hello/> An HTTP GET to locahost:8984/basex/jax-rx/bar/ returns <world/> How do I build that interface? What are the steps? Are 'foo' and 'bar' collections? What's the overall structure? 4. Documentation ================ I know that formal documentation is a bit thin on the ground, but would anyone be willing to send me their own informal personal notes about BaseX REST? I am currently struggling to even get started with the BaseX REST API. Nigel

1 0

Large updating query
by Huib Verweij 31 Oct '10

31 Oct '10

Hi BaseX-ers, I'm running into an out-of-memory error when running an updating query over (almost) every document in the database. Every doc has the form <root> <entry><node3/><node2/>...</entry> <entry><node2/><node1/>...</entry> ... </root> I am reordering the elements in every <entry/> element in every document (e.g. <node1/><node2/><node3/>). The query is easy enough to write, it reorders the elements in every <entry/> and replaces the <entry/> in the document with the reordered <entry/>. It works for just one document, but as I said, the db keels over when applying the query to every document. (I am not using any doc() function by the way, just matching on the top level <root/> node.) Do you have a suggestion how I could make this work? I've looked at the documentation on the BaseX website about XQuery Update, but frankly, I do not really understand the bit about documents not being updated, I just select a node from the database, update it, works fine... I must be missing something... (the point probably ;-)). I am using the basex REST api. Huib Verweij.

2 4

Working with indices
by Jan Vlčinský 29 Oct '10

29 Oct '10

Hi I have a bigger collection DDR-All Database Properties Name: DDR-All Size: 743 MB Nodes: 33340760 Height: 8 Database Creation Source Path: D:/var/TICEReports/JSDI data/bezne Time Stamp: 26.10.2010 02:08:30 Input Size: 542 MB Encoding: utf-8 Documents: 79313 Whitespace Chopping: ON Entity Parsing: OFF Indexes Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: OFF When I run xquery count(//MSG) it shows result 511990, which is correct. However, it takes a lot of time to produce that result. I thought, that with existing indices it shall bring the result in seconds, and not in minutes (like 10 minutes) Note, that I have created the collection before 6.3 was released, so this might be the case and I would have to import it all again. I was trying to run xquery, using indices xquery basex:index('MSG', 'path') xquery basex:index('//MSG', 'path') but I got a feeling, I do not got it properly, as it responds "Unknown index path" What I would like to know is - names of indices available - what is basex:index function usable for and for what it is not - can I create my own indices? Like calculating date from dateTime fileds in XML. I guess it is not supported. - any further reading? Most of the description is related to full text, but this is not my case. Jan PS: Despite of some performance problems (query lasting 25 minutes) I was able to dig out some very valuable information, which would be inefficient and probably much more error prone in any other way. http://janvlcinsky.s3.amazonaws.com/projects/Monitor/websamples/timeplot/tg… -- *Jan Vlčinský* Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com

2 1

Re: [basex-talk] Joins
by Christian Grün 28 Oct '10

28 Oct '10

Hi Patrick, > it is possible to connect two or more xml-databases with one xquery join in BaseX? That's no problem; here's an arbitrary example: for $a in basex:db('db1')//one for $b in basex:db('db2')//two where $a/@id = $b/@id return .... Mails to our mailing list are preferred; as this might be of general interest, I also passed this on to our list. Hope this helps, Christian > I did not find any possibilities to use this feature :( > > Thank you a lot. > > Patrick

1 0

Empty collection reports 1 document
by Jan Vlčinský 27 Oct '10

27 Oct '10

Hi Collection with none and one document inside reports the same count of documents: 1 > create db Test3 Database 'Test3' created in 119.19 ms. > xquery count(collection('Test3')) 1 Query executed in 0.43 ms. > xquery for $doc in collection("Test3") return base-uri($doc) file:/D:/opt/BaseX/Test3 Query executed in 0.74 ms. > xquery / Query executed in 0.33 ms. > add as xml.xml <root/> Path 'xml.xml' added in 3.67 ms. > xquery / <root/> Query executed in 1.1 ms. > xquery count(collection('Test3')) 1 Query executed in 0.54 ms. > With best regards Jan PS: If I should fill it to some bug tracking tool, just let me know where I can find it. Sourceforge? -- *Jan Vlčinský* Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com

2 4

How to output plain text (CSV) from XQuery
by Jan Vlčinský 27 Oct '10

27 Oct '10

Hi According to what I found it might be quite implementation specific. In http://www.stylusstudio.com/xquerytalk/201004/003992.html I found some instructions Using Saxon: declare namespace saxon = "http://saxon.sf.net/"; declare option saxon:output "indent=no"; declare option saxon:output "method=html"; Using eXist: declare namespace exist = "http://exist.sourceforge.net/NS/exist"; declare option exist:serialize "method=html indent=no" Is there any simple trick to do that? Thanks Jan -- *Jan Vlčinský* Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com

2 2

Removing document from collection erases indexes
by Ivan Lagunov 26 Oct '10

26 Oct '10

Hello, Is it the correct behavior that indexes are automatically removed when I remove a document from a collection (version 6.3)? I mean text and attribute indexes, I haven't tried full-text index. Best regards, Ivan Lagunov

3 5

How to get base-uri under my control
by Jan Vlčinský 26 Oct '10

26 Oct '10

Hi I am adding new documents to collection and would like to have base-uri without prefix of BaseX startup location > add as 3ndtest to \special\path <root/> Path '3ndtest' added in 3.88 ms. > xquery for $doc in collection() return base-uri($doc) file:/D:/opt/BaseX/second.xml file:/D:/opt/BaseX/second.xml file:/D:/opt/BaseX/s econd.xml file:/D:/opt/BaseX/second.xml file:/D:/opt/BaseX/third.xml file:/D:/op t/BaseX/third.xml file:/D:/opt/BaseX/third.xml file:/D:/opt/BaseX/test file:/D:/ opt/BaseX/2ndtest file:/D:/opt/BaseX/path/3ndtest file:/D:/opt/BaseX/special/pat h/3ndtest file:/D:/opt/BaseX/special/path/3ndtest file:/D:/opt/BaseX/special/pat h/3ndtest Query executed in 2.68 ms. > I tried many ways how to specify target - without and with slash and backslash - no change. Is there any way to manage that? Jan Vlčinský PS: Be sure this is not a problem if it is not possible. it would be only a bit nicer.

2 2