March 2017 - BaseX-Talk - mailman.uni-konstanz.de

Preparing BaseX 8.5.4 (please don't forget Issue #1341)
by Jan Dölle 20 Feb '18

20 Feb '18

Hi, we noticed you prepare release 8.5.4. Please don't forget to fix issue: SQL with BaseX 8.5.3 #1341 Thx Jan -- Jan Dölle E-Mail: basex-list(a)id1.de Telefon +49-69-244502-0 Home: www.id1.de Information Design One AG, Baseler Straße 10, 60329 Frankfurt am Main Registereintrag: Amtsgericht Frankfurt am Main, HRB 52596 Vorstand: Robert Peters, Aufsichtsrat: Christian Hecht (Vorsitz)

4 5

standalone vs GUI character parsing
by Bridger Dyson-Smith 13 Jun '17

13 Jun '17

Hi all - I'm able to create a database with the GUI from an XML document that contains an invalid character (U+0000) -- I guess BaseX does some character scrubbing, which is awesome :). When I try creating a database with the same input XML document from the commandline (using the basex executable), I get an error message. Here are my commands in standalone mode: BaseX 8.5.3 [Standalone] Try 'help' to get more information. > create db test-bad-char /usr/home/bridger/src/another-test.xml "/usr/home/bridger/src/another-test.xml" (Line 4): An invalid XML character (Unicode: 0xb) was found in the element content of the document. > xquery db:create("test-bad-char", "/usr/home/bridger/src/another-test.xml", "/usr/home/bridger/src/", map { "chop": false() }) Stopped at ., 1/10: [FODC0002] "/usr/home/bridger/src/another-test.xml" (Line 4): An invalid XML character (Unicode: 0xb) was found in the element content of the document. > This isn't a surprise (I'm more surprised (pleasantly) by the GUI behavior); is there a way to apply the same "scrubbing" from the GUI in standalone mode? I'm sure there is, but I'm not able to figure out which option to apply. As always, thank you for your time and trouble, and thanks for BaseX. Best, Bridger

3 17

Re: [basex-talk] Basex Docker Container: How to Control Options
by Eliot Kimber 02 Apr '17

02 Apr '17

Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup. Made some progress but something is still not working. I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable. However, if I then add this: COPY basex/dot_basex /srv/.basex As before, on startup I get this: docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was. Cheers, E. -- Eliot Kimber http://contrext.com On 3/26/17, 1:40 PM, "Eliot Kimber" <basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com

1 4

Deadlock with concurrent write and reads in BaseX
by Simon Chatelain 31 Mar '17

31 Mar '17

Hello all, I am facing an issue with my application using BaseX 8.6.2. Basically what I have is one thread (Add thread) receiving data from a socket as XML fragment and storing it in BaseX. On the other hand, my application provides a REST interface to retrieve data from the DB. When retrieving data what happen is, I first execute a simple Xquery in a thread (Primary Read), and for each hit returned by this first query, I execute another query in another thread (Secondary read) to retrieve additional data. The problem is that I end up in a situation where all three threads are stopped waiting for something. The Add thread is park as it cannot acquire a write lock, while the Primary read is running, as it should be. But the Secondary read is also parked, thus blocking the Primary read, and so we end up in a kind of deadlock. The PARALLEL option in .basex is set to 4, and the FAIRLOCK option is set to true. Some technical details: In the primary read I am using session.setOutputStream(pos); session.execute(new XQuery(flwor)); where session is a LocalSession and pos is a PipedOutputStream In Secondary read, I am reading the other side of the piped stream, and for each line read from the stream, I execute another query (well not exactly in the real case scenario, but this simplification seems good enough to exhibit the problem, and is shown in the attached BaseXDeadlock.java example). You can also see the stacks of all three threads in BaseXDeadlockStacks.txt In this example, if I don’t start the Add thread, then Primary and Secondary read are running as expected. On the other hand, if I change my implementation of Primary and Secondary read, and use a blocking queue to transfer data from Primary to Secondary read like shown below: Query query = session.query(flwor) while (query.more()) { queue.put(query.next()); } And then taking from the queue in Secondary read, then it works. This implementation is shown in BaseXNoDeadlock.java. Using the latter is good enough for me, as it does the job, even if there is a small performance penalty compared to the first implementation, in my real case scenario. So my question is: is my first implementation fundamentally broken or is there something wrong on BaseX side? Best regards and many thanks to all the BaseX team for this efficient and nice to use tool. Simon Chatelain

2 1

BaseX hanging on REST/XQ pipelining
by Erik Peterson 31 Mar '17

31 Mar '17

It seems like on some RESTXQ/REST pipelining (restxq calling another restxq api) causes basex to hang. In this scenario a mock client was created to test the system. A test was written to automate mock client requests. A restxq path was created run tests. More specifically... 1) Browse to http://localhost:8984/unit-test/test where last parameter "test" maps the directiory where unit tests are located. 2) #1 is handled by a RESTXQ function where a REST URL is fetched: fetch(http://localhost:8984/rest?command=test+../webapp/mock-client/test) 3) #2 is handled by a unit-test that fetches a mock-client URL fetch(http://localhost:8984/mock-client/do-something) 4) #3 is handled by a the mock-client hitting the system fetch(http://localhost:8984/system/do-something) - #3 and #4 (mock-client) work great - #2 (the unit-test) work great when running tests from command line OR when the test is a simple unit:assert(<canary/>). But basex hangs otherwise Thanks for any insight on why the system calling back to itself may cause a system hang.

2 2

suddenly...java function x is unknown
by Erik Peterson 31 Mar '17

31 Mar '17

I've been integrating basex with 3rd party java jar files by dropping them in the basex/lib directory. This has worked fine until today when I now git "java function x is unknown". I've... - re-pulled everything from our repo to make sure all binaries are as they were when working - restarted osx - updated java - using BaseX VERSION 8.6.1 (February 24, 2017) It's working on other dev machines but not mine. Why would BaseX stop loading the classes from the lib jar files? Is there a better way I should be deploying jars? Thanks.

2 2

Performance: associating one with many, or many with one?
by Jay Straw 29 Mar '17

29 Mar '17

Hi List! I'm building an application in base-x that's sort of like a local music wiki: bands, members, venues, promoters, and albums and songs are some of what I'm representing in XML. And what I'm wondering is that, in a situation where there's one of something (a musician), and a bunch of something else they're associated with (song writting or performance credits), which node should get the reference? The numerous node gets the reference back to the one: <album name="We Might Rock You" by="The Quesadillas"> <track number="1" name="Hot Cross Buns" written-by="Joe Schmoe" /> <track number="2" name="Bicycle" written-by="Joe Schmoe" /> </album> [ etc ] Or the one refers to the many: <person name="Joe Schmoe"> <songs-written> <song name="Hot Cross Buns" /> <song name="Bicycle" /> </songs-written> </person> Example 1, I could use something like: track[@written-by="Joe Schmoe"] Example 2: person[@name="Joe Schmoe"]/songs-written/song My other thought was putting the references in both, but I'll have to do this with many types of objects I'm representing and I don't want things to get out of control (then again, if I go to export an <album/> or <band/> and I'm using the second method, I'll have to do a join) I have a feeling in a large set, the second way wins out. Even where I live, with only around 750,000 people in our entire state (Alaska), we have easily 100 - 200 acts, including bands, solo performers, traditional acts, etc, with thousands of individuals that compose them. But I want bands that are extinct, bands that are on hiatus, etc, and I hope my software is useful to others in larger cities as well. So, maybe a pointless question because my db will be so tiny compared to some folks who post on here, but just curious :-) Also, if this problem has a name I can research myself, that'd be great! Be well, Jay Straw

3 2

Basex Docker Container: How to Control Options
by Eliot Kimber 26 Mar '17

26 Mar '17

I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com

1 0

Possible to Access Environment Variables from XQuery?
by Eliot Kimber 25 Mar '17

25 Mar '17

I’m running BaseX in a Docker container started via docker-compose. I need to communicate some local configuration details to BaseX. Can set these as environment variables (e.g., using .env with docker-compose) but I don’t see anything in the docs that would let me access environment variables from XQuery. Is there a built-in way to do this? Thanks, Eliot -- Eliot Kimber http://contrext.com

3 3

and operator in Multiple Predicates
by Tushar Deshmukh 23 Mar '17

23 Mar '17

Namaste , and operator in predicates seems tobe not working. my xquery is as below and xml is attached herewith. declare namespace UDF="TallyUDF"; let $fromdate:=20170228 let $todate:=20170228 let $pledger_code:=57 let $pledger_status:="P" for $allitems in db:open('10096',"Vouchers")/ENVELOPE/Voucher[./Date/text() >= $fromdate and ./Date/text() <= $todate and ./AllLedgerEntries.LIST/UDF:AsPerListAllocations.LIST/UDF:ColPartyCode.LIST/UDF:ColPartyCode/text()= $pledger_code and ./AllLedgerEntries.LIST/UDF:AsPerListAllocations.LIST/UDF:MaheshCIFlag.LIST/UDF:MaheshCIFlag/text()= $pledger_status] return $allitems/MasterId Expected 2 records/nodes instead 4. Tushar Deshmukh,

2 1