February 2013 - BaseX-Talk - mailman.uni-konstanz.de

add a 'path' argument to the CREATE command
by Fabrice Etanchaud 25 Feb '13

25 Feb '13

Dear all @ BaseX, Db :create has now a path argument. Could the CREATE command get this argument too ? For the moment, one have to do the following set of commands to create a collection and put the documents in a specific path : - Create... - Rename... - Optimize... - Close So, optimization is done twice, with a real impact on creation's duration. Does somebody have comments on the possibility to directly move data from a collection to another without having to export and add files ? I'm writing Talend components for BaseX, is this interesting other people out there ? Best regards from the snowy French Riviera... Fabrice Etanchaud Questel-Orbit

1 0

On the client protocol level: Can a query be executed multiple times?
by Francis Avila 25 Feb '13

25 Feb '13

I have no official connection to basex, but I have played around with the server protocol with the idea of writing a Python driver for it that is more idiomatic Python and that better manages changes of state on the connection. What I have discovered, at least as of BaseX 7.3, is the following: A query can be executed multiple times, where "execute" means any of the result-producing commands: RESULTS, EXECUTE, FULL A query *cannot* have any parameters rebound. In other words it's not possible to prepare a query, bind an external variable, execute the query, then bind another variable, execute again, and get another answer. Although no error will be reported when a var is rebound the results act as if it were not rebound. So once a query is executed, you must treat the query as an immutable result. If you wish to issue the same query with a different bound value, you must create a new query with QUERY (producing a new query id). Also, since there is no iterative partial execution of queries (like an SQL cursor, for example) and all results are returned at once, your question "how is the previous execution ended?" is not meaningful. The previous execution is ended when you receive results. Closing a query deregisters it. It no longer has any existence, and any attempt to use it will return an error "Unknown Query ID". Also note that query text does not seem to be evaluated when the query ID is created, so any syntax errors will not be caught until you directly (RESULTS, EXECUTE, FULL commands) or indirectly (OPTIONS command, maybe UPDATING as well?) execute the query. I have written a very low level BaseX console in Python which I use to examine the exact semantics of the client-server protocol. Below is a log from one of those demonstrating many of the things I have said. Lines that start with "?" are user-entered commands. A bare integer produces a byte of that value, and a double-quoted string produces a null-terminated utf8 string. Lines starting with ">" are the bytes sent to the server, and "<" are the bytes received from the server. Note that I have no idea what the "INFO" command is supposed to indicate! ? 0 "declare variable external $x; $x to $x+10" > '\x00declare variable external $x; $x to $x+10\x00' < '0\x00\x00' ? 6 0 > '\x06\x00\x00' < '\x00\x01Unknown Query ID: \x00' ? 6 "0" > '\x060\x00' < '1\x00\x00' ? 3 "0" "x" "1" "xs:integer" > '\x030\x00x\x001\x00xs:integer\x00' < '\x00\x00' ? 6 "0" > '\x060\x00' < '\x00\x00' ? 7 "0" > '\x070\x00' < "\x00\x01Stopped at line 1, column 17:\n[XPST0003] Expecting '$', found 'e'.\x00" ? \2 "0" > '\x020\x00' < '\x00\x00' ? 0 "declare variable $x external'; $x to $x+10" > "\x00declare variable $x external'; $x to $x+10\x00" < '2\x00\x00' ? 2 "2" > '\x022\x00' < '\x00\x00' ? 0 "declare variable $x external; $x to $x+10" > '\x00declare variable $x external; $x to $x+10\x00' < '3\x00\x00' ? 6 "3" > '\x063\x00' < '\x00\x00' ? 7 "3" > '\x073\x00' < 'byte-order-mark=no,cdata-section-elements=,doctype-public=,doctype-system=,encoding=UTF-8,escape-uri-attributes=no,format=yes,include-content-type=no,indent=yes,indents=2,media-type=,method=xml,newline=\\n,normalization-form=NFC,omit-xml-declaration=yes,separator= ,standalone=omit,suppress-indentation=,tabulator=no,template=,undeclare-prefixes=no,use-character-maps=,version=,wrap-prefix=,wrap-uri=\x00\x00' ? 3 "3" "x" "0" "xs:integer" > '\x033\x00x\x000\x00xs:integer\x00' < '\x00\x00' ? 5 "3" > '\x053\x00' < '0 1 2 3 4 5 6 7 8 9 10\x00\x00' ? 4 "3" > '\x043\x00' < '40\x0041\x0042\x0043\x0044\x0045\x0046\x0047\x0048\x0049\x00410\x00\x00\x00' ? 31 "3" > '\x1f3\x00' < '40\x0041\x0042\x0043\x0044\x0045\x0046\x0047\x0048\x0049\x00410\x00\x00\x00' ? 3 "3" "x" "5" "xs:integer" > '\x033\x00x\x005\x00xs:integer\x00' < '\x00\x00' ? 4 "3" > '\x043\x00' < '40\x0041\x0042\x0043\x0044\x0045\x0046\x0047\x0048\x0049\x00410\x00\x00\x00' ? 5 "3" > '\x053\x00' < '0 1 2 3 4 5 6 7 8 9 10\x00\x00' ? 2 "3" > '\x023\x00' < '\x00\x00' ? 6 "3" > '\x063\x00' < '\x00\x01Unknown Query ID: 3\x00' ? > From: Hans Hübner <hans.huebner(a)gmail.com> > Subject: [basex-talk] On the client protocol level: Can a query be executed multiple times? > Date: February 23, 2013 2:41:30 PM CST > To: basex-talk(a)mailman.uni-konstanz.de > > > I have a question regarding the client protocol: Can a query that has been prepared using the "QUERY" protocol message be executed only once, or multiple times? If it can be executed multiple times, with different parameter sets, how is the previous execution ended? The documentation of the "CLOSE" message seems to indicate the the query is unregistered when it is closed. > > Is there some documentation or sample code that illustrates the expected sequence of client messages for query execution? > > Thanks, > Hans

1 0

Re: [basex-talk] Xquery over all collections
by pw＠themail.co.uk 24 Feb '13

24 Feb '13

Hi Lukas Sorry to be so long in actually picking this up... but I think that let $r := (collection('A'), collection('B'),collection('C'))//br is a lot faster than let $r := for $d in db:list() return collection($d)//br Any thoughts? Cheers Peter > > > >---- Original Message ---- >From: lukaskircher1(a)gmail.com >To: pw(a)themail.co.uk >Subject: Re: [basex-talk] Xquery over all collections >Date: Wed, 6 Feb 2013 08:40:07 +0100 > >>Hi Peter, >> >>for example, if I look for all *<br>* elements over all >>documents/collections: >> >>*for $d in db:list()* >>*return collection($d)//br* >>* >>* >>Is this what you were looking for? >> >>Cheers, >>Lukas >> >> >>On Wed, Feb 6, 2013 at 1:13 AM, <pw(a)themail.co.uk> wrote: >> >>> let $r := (collection('A'), collection('B'), >>> collection('C')) >>> >>

1 0

BaseX C# Client Slow Remote Performance
by Scott Graves 23 Feb '13

23 Feb '13

I'm testing inserts / updates / queries and ran into a puzzling issue. Testing for the existence of a node where the client and server are running on the same box executes in about 4-7ms; however, testing the same query on a remote server executes in a minimum of 1-1.5s. I've tested on both jre6 and jre7 as well as BaseX installations on 2 different servers. Same issue either way. Although the logs appear to be from 127.0.0.1, I'm actually using an SSH tunnel for testing remotely but there's no real difference. 13:40:51.212 127.0.0.1:32264 admin OK Database 'LibraryDatabase' was opened in 0.84 ms. 1.21 ms 13:40:51.222 127.0.0.1:32264 admin OK QUERY[0] for $d in /db/table[@name='Music Library']/Setting/DataList/Data where $d/@Key='Id' and $d/@Value='0F5349957FB73A4585FBC7A392D4DEBA20F41667869FE6A8EDB14332E80C255C' return $d 0.12 ms 13:40:51.232 127.0.0.1:32264 admin OK ITER[0] 1.56 ms 13:40:51.247 127.0.0.1:32264 admin REQUEST EXIT 13:40:51.247 127.0.0.1:32264 admin OK 0.32 ms 13:40:52.327 127.0.0.1:32265 admin REQUEST CHECK LibraryDatabase 13:40:52.327 127.0.0.1:32265 admin OK Database 'LibraryDatabase' was opened in 0.8 ms. 1.17 ms Any help would be appreciated. I've spent only 1 day experimenting so far. - Scott

2 4

BACKUP/RESTORE - 32 bits limitation / java 7
by Fabrice Etanchaud 22 Feb '13

22 Feb '13

Dear BaseX users, BACKUPing a big database (leading to a bigger than 2 Go archive) is ok, But RESTOREing it fails because archive is corrupted. It seems to be related to the 32 bit zip library limitation, that may be solved with java 7. Did someone already try BACKUP/RESTORing big databases with BaseX running on a java 7 jre ? Best regards, Fabrice Etanchaud Questel-Orbit

2 1

OPTIMIZE ALL : <DBNAME>_224548755 could not be renamed
by Fabrice Etanchaud 22 Feb '13

22 Feb '13

Dear BaseX Team, It seems to be a problem with the OPTIMIZE ALL command. When applied on a <DBNAME> collection, it ends up with the following message : "<DBNAME>_224548755 could not be renamed" I suppose a new optimized collection is being created aside the current one, and cannot be renamed as <DBNAME> at the end ? Best regards, Fabrice Etanchaud Questel-Orbit

2 1

BaseX xpath querying takes more time than SAX parser takes
by elangovan MuthuSwamy 22 Feb '13

22 Feb '13

Hi All, I am trying to use BaseX Client and server with large XML files(13 MB size). I have created a database for the XML and evaluated an xpath in that. When I executed an xpath it takes around 250 milliseconds to fetch the results. In case of SAX parser I could get the response for the same xpath within 5 seconds. I used following BaseXClient code for this. BaseXClient session = new BaseXClient("localhost", 1984, "admin", "admin"); // code to fetch the large XML file size. Read from the file. String xmlString = ....... InputStream bais = new ByteArrayInputStream(xmlString.getBytes()); session.create("database", bais); String expression = "/configuration/protocol[starts-with(name, 'http')]/name"; *session.query(expression).execute();* // this line takes 250 milliseconds to give the result. Where as same xpath can be executed within 5 milliseconds in SAX parser. Am I basically missing anything or something else. Please advice me. Thanks in advance. -- Regards, Elango.

3 2

Not enough space
by Shannon Scott Shiflett 22 Feb '13

22 Feb '13

Hi, whenever I choose to show the tree visualization, after a very long delay, "Not enough space" is displayed in that space--it's a small database (<100MB) and 8GB RAM. Unless you recommend something else, I'd like to try completely uninstalling and replacing with a fresh BaseX--How to do this on OS X, besides deleting the app and data directory? Thanks. -- Shannon Scott Shiflett Sr. Programmer/Analyst The University of Virginia Press http://rotunda.upress.virginia.edu/

4 8

Whitespace handling on ingest
by Wendell Piez 22 Feb '13

22 Feb '13

Hi, I see the 'CHOP' option, turned on by default, for trimming leading and trailing whitespace and eliminating empty text nodes. What about going further? Is there a good way to normalize whitespace entirely, collapsing any runs of tab-LF-space into single spaces in my data? I think I mentioned earlier the idea of specifying an XSLT transformation to filter data on ingest (for a similar requirement, namely removing all comments and PIs). That might be going too far but any hints you can give me (or pointers to docs) about functionality to address this sort of thing in general would be welcome. Thanks! Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

2 3

GUI enhancement to show only query results as text in a tree?
by Amanda Galtman 22 Feb '13

22 Feb '13

Hi, The BaseX GUI is very nice for interactively tweaking a query or exploring my data set. However, one thing I wish I could do with the GUI is see query results as text in a tree with expandable/collapsible nodes. An example of what I mean is in the right pane of this Qizx screen capture: http://www.axyana.com/qizxopen/_distrib/docs/manual/qizxstudio.html#d0e4472 The tree in the screen capture makes it easy to distinguish between adjacent result items, and the collapsibility of result items makes it easier to decide when you want to browse the results at a high level or dig into a specific result. By contrast, the BaseX GUI's text view shows all the text of the query results, but they run together. The folder visualization has expandable/collapsible nodes, but because it shows the query-result nodes in the context of the entire database, you might have to expand a lot of nodes to discover where the query-result nodes are. While it can be useful to see the query-result nodes in context, I often want to see only the results themselves; and if I decide I need more context, I tweak my query to return more context. Would it be possible in a future release of BaseX to get a text tree-view of just the query results? Better yet, is it already available but I don't know about it? Thanks, Amanda

2 2