- BaseX-Talk - mailman.uni-konstanz.de

Bulk Loading XML documents with Add(docPath).execute(CONTEXT)
by pw＠themail.co.uk 22 Aug '10

22 Aug '10

When adding documents from a directory I get things working using a few small documents and Add(docPath).execute(CONTEXT) but when I want to add hundreds of documents (average size 1Mb) I get a memory error. I am using Jython and have -J-Xms1200M and -J-Xmx1200m should I use an iterator and only add one document at a time? [this doesn't seem to me to be the way to use Add() ] TIA Peter

2 1

how do I embed character data in XQuerys?
by Godmar Back 22 Aug '10

22 Aug '10

Hi, following Christian's insinuation we're now abandoning XQJ and are porting our application to use the ClientSession API. I'm performing a textual replacement of the external variables in my XQuery expressions before sending them to BaseX's server. I'm facing difficulties with embedding character data in the XQuery expressions that are sent to the server. Notably: xquery <c-code>if (cond) { // a C comment }</c-code> produces: Stopped at line 1, column 33: [XPST0003] Expecting "}", found "C". whereas: xquery <c-code><![CDATA[ if (cond) { // a C comment } ]]></c-code> produces: <c-code> if (cond) { // a C comment } </c-code> Is this a bug in BaseX? I thought that the use of CDATA is optional and required only if one wishes to avoid escaping <, >, & in PCDATA. If CDATA is required, I would have to convert my XML document in this way. The problem did not occur in the XQJ binding. I haven't consulted the grammar; I'm following my intuition here that XQuery should allow embedded XML fragments. Is this not so? - Godmar

3 7

Student project based on BaseX
by Antonis Thomas 22 Aug '10

22 Aug '10

Hello all, we are two master students at the university of Utrecht. We have a DB Architecture class that includes a small project (15%*7.5 ECTS= 1.125 ECTS). So, BaseX looks like a good chance to work on such a project. Our idea so far is based on some benchmarking experiments and small summary on how it works (maybe its relation with Monet) including any experiments we run (5-6 pages). Do you have anything better for us to do (maybe something you can get advantage of yourselves) or any guidelines/hints/whatever? Take into account that out time limit is approximately a week (full-time) and that we cannot do more work that what deserves 2.25ECTS for the 2 of us. I look forward to an answer (so we can decide and start working -the deadline is 9/7-. Best regards, -- Antonis Thomas Information & Computing Sciences Utrecht University Utrecht, The Netherlands

2 13

how to separate multiple results when using BaseX's client/server interface
by Godmar Back 19 Aug '10

19 Aug '10

Hi, when I use the client/server interface and run a query that returns multiple nodes, BaseX appears to return the results concatenated (and, somewhat oddly, with interspersed newlines). For instance, xquery declare namespace atom='http://www.w3.org/2005/Atom'; doc('myfeed')//atom:title returns: <title xmlns="http://www.w3.org/2005/Atom">First Title</title> <title xmlns="http://www.w3.org/2005/Atom">Second Title</title> <title xmlns="http://www.w3.org/2005/Atom">Third Title</title> .... Does anybody happen to know how to parse those into separate XML documents in Java? (Short of preprocessing the XML to find the matching end tag for each element?) For background, classes such as javax.xml.parsers.DocumentBuilder can only parse one document at a time, and throw an error if the input source contains multiple documents/fragments. Alternatively, would it make sense for BaseX to support returning results one by one (perhaps using a cursor approach) to ease the client's task of parsing the returned XML? - Godmar

3 3

Re: [basex-talk] Error while creating an attribute index with the latest from SVN
by Huib Verweij 19 Aug '10

19 Aug '10

Hi Christian, Op 19 aug 2010, om 13:18 heeft Christian Grün het volgende geschreven: > The stack trace indicates that the merge of several temporary index > instances causes problems. Regarding the SVN history, the current > index builder shouldn't have been changed that much since… 6.1.8? As a > shortcut, it might help to give the JVM different -Xmx values; 1gb > should be completely more than enough for 780 mb of input data. In the > GUI, you should be able to observe how much RAM is actually used > during index construction. That was it. 1Gb didn't work, but 2Gb was enough to get it done in about 50 seconds. > >> I don't get anything else, if there's something I can do to debug it let me know. > > It might help if you could provide us with the original data (which > will of course be handled confidentially). It might take some time, > however, as I'm currently busy writing my thesis… I can't supply you with the original data I'm afraid. That would breach my contract with the MPI. Anyway, thanks for your help again and good luck with the thesis. Huib.

1 0

How is transaction supported in BaseX?
by 汤力丞 19 Aug '10

19 Aug '10

Recently I choosed BaseX as our cybermarketing data store. The most appealing feature for us is ACID transaction support as BaseX claims while eXist supports only invisible transactions limited to crash recovery. But I failed to find any example or command related such as ROLLBACK or COMMIT. So how can I fully utilize this important feature? Thanks! Best regards, Michael Tang

2 1

Re: [basex-talk] BaseX [NOT PROTECTIVELY MARKED]
by Christian Grün 18 Aug '10

18 Aug '10

Hi Peter (cc to the mailing list), the following page might interesting for you, on which we've compiled some statistics on created databases: http://basex.org/documents Reg. number of documents and single doc size: the number depends on various meta data such as the number of XML nodes in the document. As you can see on the quoted page, we've added 10 Mio. documents of the CoPhIR collection in one database, and one document with 421 GB. As soon as one database gets too large, several databases can be created and accessed by a single XQuery. Hope this helps, Christian On Wed, Aug 18, 2010 at 2:59 PM, <Peter.Winstanley(a)scotland.gsi.gov.uk> wrote: > Classification: NOT PROTECTIVELY MARKED > > Hi Christian > > I've just started looking at BaseX and would appreciate some pointers to > performance information. My obvious expectation for comparisons would be > with eXist and MarkLogic. > > I was wondering if you had any information on the maximum numbers of > documents that BaseX can handle, and also the maximum size of a single > document that could be practically handled (here I'm thinking about > multi-dimensional data 'cubes') > > Kind regards > > Peter > > Dr Peter Winstanley > ISIS: Information Management Unit > The Scottish Government > 5 Atlantic Quay > 150 Broomielaw > Glasgow G2 8LU > Scotland > 0300 244 6895

1 0

BaseX as persistence layer
by chas＠munat.com 18 Aug '10

18 Aug '10

I'm trying to use BaseX as a persistence layer in a project (together with XStream). I have a few questions and I'm not having any luck with the documentation. I've gotten pretty far. I have a User object (simplified for this example) with an id (UUID), a username (String), and a password (String). XStream turns this into: <User> <id>98bdb472-97ad-4de1-b1c6-a482d7f06d94</id> <username>Chas</username> <password>secret</password> </User> I've created a DB called "db" (clever, huh?): new CreateDB("cop", "<blog><users/></blog>").execute(context) QUESTION: If I rerun the above, will it overwrite the DB if it already exists, or will it just open the existing DB? Can't find anything in the API about that. Then I save my User like this: new XQuery("insert node" + chas + " into /blog/users").execute(context) where chas is the above XML. This works great! When I need to reinflate the object, I just do: chas = new XQuery(""" for $x in /forum/users/User where $x/username="Chas" return $x """).execute(context) Then I use XStream to reinflate the object. Any suggestions for improvement? So now the problem. I need to update this object in the database. The simplest way is just to update the whole object. So I change my username and use XStream to create the new XML: <User> <id>98bdb472-97ad-4de1-b1c6-a482d7f06d94</id> <username>Charles</username> <password>secret</password> </User> I read that one must use put() to export the changes, but maybe I'm misunderstanding this. I'm have this: for $x in /forum/users/User where $x/username="Chas" return replace node $x with " + chas This and seems to work, but then when I try to delete the node thus: for $x in /forum/users/User where $x/password="secret" return delete nodes $x It does not work. What am I missing? Thanks for any help. If this thing works, I'm happy to open it up so others can work on it, too. Chas.

5 8

Schwerer Bug von Namespace!
by Truong An Nguyen 18 Aug '10

18 Aug '10

Hallo, ich verwende im Moment die BaseX 6.2.4. Wieso sind die Namespaces unterschiedlich behandelt in zwei folgenden Situationen: + Fall 1: Die Datenbank wurden erzeugt von einem ganzen Ordner: Create Database c:\Root declare namespace ns='http://iso.org/OTX'; collection('Root')/ns:otx/ns:procedures/ns:procedure/ns:realisation Die Ergbenisse sind: "<realisation xmlns:hmi="http://iso.org/OTX/StandardInterface/HMI" xmlns:diag="http://iso.org/OTX/StandardInterface/DiagCom" xmlns:env="http://iso.org/OTX/StandardInterface/Environment"> + Fall 2: Die gleiche Datei wurde zu einer leeren Collection hinzugefügt. Create Database Empty Add TO c:\Root c:\Root\file.xml Dann funktioniert die Abfrage im Fall 1 nicht mehr. Wenn ich die Namespace von procedures, procedure und realisation wie folgend wegmache, funktioniert es. declare namespace ns='http://iso.org/OTX'; collection('Root')/ns:otx/procedures/procedure/realisation Aber alle Namespace-Informationen im Ergebnis sind auch weg. Ich bekomme nur "<realisation> ...." Fall 2 ist mir sehr wichtig. Könntet ihr bitte diesen Bug beheben? Vielen Dank Gruß, An

1 0

Q.: safety/concurrent access to BaseX database
by Godmar Back 18 Aug '10

18 Aug '10

Hi, I just wanted to confirm some assumptions about the safety of accessing a single BaseX instance from multiple JVMs, or from multiple org.basex.api.xqj.BXQDataSource class instances loaded by different class loaders in the same VM. Am I correct that it is safe to access the same database via XQJ in such situations? By "safety," I mean that a) the database protects its internal data structures in the face of such concurrency and b) there is a serializable schedule of individual XQJ executions that produces the final database state. Is this true? Please note that I'm not asking for transactions across multiple XQJ executions here. - Godmar

2 6