February 2016 - BaseX-Talk - mailman.uni-konstanz.de

Automated Docker Build: basex/basexhttp
by Jens Erat 19 Feb '16

19 Feb '16

Dear BaseX community, over the last weeks, interest in Docker utilization with BaseX hevily increased, and several images have been proposed. I've already made some experience running a BaseX pet project in BaseX for about a year now, and together with the BaseX core team created an image based on those experiences and current best practices. basex/basexhttp Docker Image ---------------------------- Finally there's the official basex/basexhttp Docker image readily available on the Docker Hub. It's an automated build directly from source, with nightly builds and tagged builds for future BaseX releases (starting with 8.4.1/8.5, whatever will come first). It is derived from the Maven base image (and thus Debian), and is automatically updated when the base images experience updates (like Java security fixes). - https://hub.docker.com/r/basex/basexhttp/ - https://github.com/BaseXdb/basex/blob/master/Dockerfile While the image is named basexhttp and always includes the HTTP server, it can also be used for running the plain basexserver or even basexclient. DBA Application Container ------------------------- As an example for deriving your own application images and also for interfacing BaseX for administrative tasks and ad-hoc queries, the DBA is also made available as a container. - https://hub.docker.com/r/basex/dba/ - https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/dba Documentation ------------- As you're used to, documentation is available in the BaseX wiki. - http://docs.basex.org/wiki/Docker - - - If you've got any feedback, questions or proposals, feel free to get in touch with me or the core BaseX team on the usual ways. Kind regards from Lake Constance, Germany, Jens -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de OpenPGP: 0D69 E11F 12BD BA07 7B37 26AB 4E1F 799A A4FF 2279

2 4

Getting element counts in the visualizations?
by Steinar Bang 19 Feb '16

19 Feb '16

Platform: Intel i7 Windows 7 Enterprise BaseX 8.4 Today I opened an 80MB XML file in the BaseX GUI, and I was amazed at the speed of BaseX, compared to e.g. trying to open, count, and extract stuff from the same file in emacs. What I needed from BaseX was finding the size of the contents of the second level elements (ie. the elements immediately below the root element). The visualizations were great at navigating in the file and the Map visualization showed the sizes I was interested in. I was comparing two files with the same structure, one file with size 60MB, the other with size 80MB, and tried to figure out where the 20MB had gone. On thing I was looking for, but did not find, was the number of child elements of a given element. I was hoping for a tooltip when hovering over an element in the visualizations (map, folder and tree), or a properties dialog when right clicking the elements in the visualization. Is the information present somewhere? (I know the element child count has to be there somewhere, because without it, the map visualization couldn't be rendered...?) How hard is this information to get at? Thanks! - Steinar PS what I ended up doing, was selecting the elements I wanted to find the size of in map and tree, and then saving the contents of the Result view for each element. A bit more clumsy than I would have liked, but much faster than doing it in emacs.

2 6

Re: [basex-talk] Sending Bytes, Not Strings, To BaseX Using the Ruby Client
by Eliot Kimber 19 Feb '16

19 Feb '16

I turned my UTF-8 file into a UTF-16 file and trying to commit it to BaseX via the Ruby client it did not work: BaseXClient.rb:50:in `execute': Resource "/opt/basex/?" not found. (RuntimeError) Where "?" is some kind of "unrecognized character" indicator Cheers, E. ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com On 2/18/16, 10:26 AM, "Eliot Kimber" <basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: >I'm implementing server-side git hooks for use in GitLab under Docker >where Java is not available (at least that I can see). The hooks load or >delete files from databases in BaseX. > >I'm trying to implement the hooks in Ruby (which is much more pleasant >than bash scripting in any case) and I'm using the BaseXClient.rb from >https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby > >I need to create or replace files by sending the bytes--I'd rather not >read the input file into a Ruby string and send that since I don't trust >Ruby to not hose up the data (even when it's UTF-8 I still don't trust it, >but I only started using Ruby yesterday so maybe my mistrust is >misplaced?). > >Using the AddExample.rb as guide, I'm doing this: > >(Earlier code to open or create database, which works). > >file = File.new("../../" + path, "rb") > bytes = file.read > file.close > puts "file=/#{bytes}/" > @basex.add(path, "#{bytes}") > >I also tried: > >@basex.add(path, bytes) > > > >And I get this result (I added some debugging messages to sendCmd()): > >ensureDatabase(): Checking database "_dfst^metadata^temp^master"... >BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53 ms. >Added or modified file: "test-newname.xml" >file=/<test>This is a test 20</test> >/ > >*** sendCmd(): >cmd= >arg=test-newname.xml >input=<test>This is a test 20</test> >BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1): >Premature end of file. (RuntimeError) > > from commit-hooks/git/server-side/BaseXClient.rb:64:in `add' > from commit-hooks/git/server-side/post-receive:80:in `block in update' > from commit-hooks/git/server-side/post-receive:74:in `each' > from commit-hooks/git/server-side/post-receive:74:in `update' > from commit-hooks/git/server-side/post-receive:111:in `block in <main>' > from commit-hooks/git/server-side/post-receive:103:in `each' > from commit-hooks/git/server-side/post-receive:103:in `<main>' >Eliots-MBP:hooks ekimber$ > >A couple of things here: > > >Where is the extra ".xml" in the target filename coming from? > >What is causing the premature end of file? It feels like it's trying >interpret the second argument as a filename rather than the data to be >loaded. > >If I use basex.execute("add to #{path} #{bytes}") it works but of course I >get duplicate files if I run the command twice. > >If I try: > >@basex.execute("replace #{path} #{bytes}") > >Then I get the same failure. > > >So something is not right. > >My Docker container is running 8.4.1 beta. > >What am I missing? > >Thanks, > >Eliot >---- >Eliot Kimber, Owner >Contrext, LLC >http://contrext.com > > > >

2 5

Setting base-dir for xquery:eval
by Marc van Grootel 19 Feb '16

19 Feb '16

Hi, I am having quite some fun with xquery:eval. I am working on a little module for executable documentation (similar to Python doctest). I want to write documentation in asciidoc, parse it and execute source code blocks inside the asciidoc. Parsing works by calling asciidocj and so far I can run simple queries, evaluate them, compare them with an expected outcome and write the result back as evaluated asciidoc. Now I want to tackle more advanced queries that import modules. But when running xquery:eval with a string it will try to resolve paths relative to the code module and not relative to the file the string came from. Ideally I would like to be able to set the base-dir for xquery:eval. The asciidoc file isn't always in the correct path relative to the module that I need to import. Other options I'm considering but are less ideal. - Manipulate the module import paths in the query string using regexps - Writing out to a temp file and then use xquery:parse-uri (but this is tricky as I may not want to clutter existing directories with tempfiles only to have the imports work). Unless there are better options I think I'll opt for string massaging using regexps. But maybe someone has a better idea. --Marc

3 5

Sending Bytes, Not Strings, To BaseX Using the Ruby Client
by Eliot Kimber 18 Feb '16

18 Feb '16

I'm implementing server-side git hooks for use in GitLab under Docker where Java is not available (at least that I can see). The hooks load or delete files from databases in BaseX. I'm trying to implement the hooks in Ruby (which is much more pleasant than bash scripting in any case) and I'm using the BaseXClient.rb from https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby I need to create or replace files by sending the bytes--I'd rather not read the input file into a Ruby string and send that since I don't trust Ruby to not hose up the data (even when it's UTF-8 I still don't trust it, but I only started using Ruby yesterday so maybe my mistrust is misplaced?). Using the AddExample.rb as guide, I'm doing this: (Earlier code to open or create database, which works). file = File.new("../../" + path, "rb") bytes = file.read file.close puts "file=/#{bytes}/" @basex.add(path, "#{bytes}") I also tried: @basex.add(path, bytes) And I get this result (I added some debugging messages to sendCmd()): ensureDatabase(): Checking database "_dfst^metadata^temp^master"... BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53 ms. Added or modified file: "test-newname.xml" file=/<test>This is a test 20</test> / *** sendCmd(): cmd= arg=test-newname.xml input=<test>This is a test 20</test> BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1): Premature end of file. (RuntimeError) from commit-hooks/git/server-side/BaseXClient.rb:64:in `add' from commit-hooks/git/server-side/post-receive:80:in `block in update' from commit-hooks/git/server-side/post-receive:74:in `each' from commit-hooks/git/server-side/post-receive:74:in `update' from commit-hooks/git/server-side/post-receive:111:in `block in <main>' from commit-hooks/git/server-side/post-receive:103:in `each' from commit-hooks/git/server-side/post-receive:103:in `<main>' Eliots-MBP:hooks ekimber$ A couple of things here: Where is the extra ".xml" in the target filename coming from? What is causing the premature end of file? It feels like it's trying interpret the second argument as a filename rather than the data to be loaded. If I use basex.execute("add to #{path} #{bytes}") it works but of course I get duplicate files if I run the command twice. If I try: @basex.execute("replace #{path} #{bytes}") Then I get the same failure. So something is not right. My Docker container is running 8.4.1 beta. What am I missing? Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

1 0

Where Is Documentation On Module Lookup?
by Eliot Kimber 17 Feb '16

17 Feb '16

I'm trying to find the documentation on how BaseX looks up modules and I can't find it in the current documentation. I know I found it in the past. Where should I be looking? Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

2 3

Indenting XML query result from a function
by Hondros, Constantine (ELS-AMS) 16 Feb '16

16 Feb '16

This is probably easy, but I can't find an inbuilt method or function to do this easily. I have a function that will return certain nodes from a DB. The DB was built with CHOP off - plenty of white-space present between nodes. declare function dq:errorGivenName($db as xs:string) as element()* { db:open(concat($db, '_author))/*/*[matches(givenname, "[^\p{L}\p{M}-[\s-'''\.]]")] }; I can't predict where the function will be called so I've got no control over serialization parameters. Instead I'd like to pretty print the XML fragment returned by the function. I could write something to do this, but there must be a simple way ... right? C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 4

Binding external variable in GUI
by buddyonweb-software＠yahoo.com 15 Feb '16

15 Feb '16

At the risk of a) sounding like a real newbie b) being accused of not Googling for an answer (which I did - but apparently overlooked) I am wondering how I can bind an external variable using just XQuery syntax in the BaseX GUI. I have found how to set the variable via command line options, and how to do it in Java, but I merelywanted to run a quick interactive test in the GUI to test an idea of mine (before writing a bunch of Java code).What I have done is in a tab in the GUI, I have included the module, and before running a function, want to set that external variable. Surprisingly I found myself at a loss on how to do what appears to be a simple task. So, thanks in advance for any help....and for the rest of this community for not laughing at me too loudly Buddy

3 2

Spectactularly slow performance with db:open vs. db:text
by Hondros, Constantine (ELS-AMS) 15 Feb '16

15 Feb '16

Hello Basexers, I'm getting such a low performance on a relatively simple join between two databases that I feel there must be something going wrong here. I can provide the sources if necessary, but basically DB1 is 26 MB, about 80,000 small documents; DB2 is 47 MB, about 18,500 small documents. I'm using 8.4, by the way, haven't tested on other releases. Query 1 [returns in 144 minutes] ---------------------------------------- for $a in (db:open('DB1')/item/order-id) return if (db:open('DB2')//order-id[. = $a]) then $a else () Note that the optimized query uses db:open-pre to access DB2. When I re-write the query myself to use the TEXT index then performance is excellent. But why such a difference? Query 2 [returns in 0.3 second] ---------------------------------------- for $a in (db:open('DB1')/item/order-id) return if (db:text('DB2', $a)/parent::order-id) then $a else () ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 3

Serialization to jsonml / Whitespace-Handling
by Günter Dunz-Wolff 15 Feb '16

15 Feb '16

Hi all, hi Christian I want to serialize my xml-documents to jsonml like so: json:serialize(doc('bettelweib.xml')//*:body, map { 'format': 'jsonml' }) Inside the jsonml are lots of strings with unnecessary whitespace like: "Das Bettelweib von\n Locarno." How could I remove that whitespace and also \n during or after the serialization? Thanks for any help. Regards, Günter

2 4