September 2016 - BaseX-Talk - mailman.uni-konstanz.de

8.5.3 and xquery:invoke ...
by Marco Lettere 11 Sep '16

11 Sep '16

Hello again, at the end I managed to reproduce the issue we are currently facing with xquery:invoke in 8.5.3 (and also in latest snapshot). Please consider the three queries [1,2,3]. The store [2] into a subdirectory src of the directory containing [1] and store [3] into a directory src2 of src. You should get the error message [4]. If instead you change the parameter of xquery:invoke to "src/src2/q2.xqm" it works. Seems like the reference for static-base-uri in [2] remains the one of [1] even if it is reported by static-base-uri() to be correctly "/home/user/src/q1.xqm"(I've double-checked this). Sorry it's not clear to me whether this is meant to be the correct behaviour ... Thanks for any help, Marco. [1] q.xqm import module namespace q1="urn:q1" at "src/q1.xqm"; q1:f() [2] q1.xqm module namespace q1 = "urn:q1"; declare function q1:f(){ xquery:invoke("src2/q2.xqm") }; [3] q2.xqm 1 = 1 [4] Stopped at /home/user/src/q1.xqm, 4/16: [FODC0002] Resource 'src2/q2.xqm' does not exist.

3 4

question on starting multiple instance of basex
by Wang, Genneva 11 Sep '16

11 Sep '16

Hi I’m trying to set up to having multiple instance of basex in my local dev environment, presumably to simulate production environment where our application is pointing to two different basex that running a different server. I’ve running basex as client server application in dev environment. Is there a way to do that? I’ve tried to start basex server with 2 different ports A and B, and also starting client with port A and B, but both seemed to be pointing to the same list of the databases that was created. Any tips would be greatly appreciated. Thanks! Regards, -Genneva

2 1

fork-join and recursive function
by Carl Leitner 11 Sep '16

11 Sep '16

Hi All, I have hierarchical information encoded an XML document that looks something like: <organization entityID=“1”> <organization entityID=“2"> <parent entityID=“1”> </organization> <organization entityID=“3"> <parent entityID=“2”> </organization> <organization entityID=“4"> <parent entityID=“1”> </organization> There are around 80,000 entries like this and I need to regular do extractions of sub-hiearchies (see the commented out version of the below function). The commented-out version runs in about a minute using a modest amount of memory. Hoping to take advantage of this <https://github.com/BaseXdb/basex/commit/ac86bbbc3cd1f71461ce94d803cab46f21e…>, I modified the function (the uncommented version) which tries to use xquery:fork-join in 8.5.2beta (the July 12th snapshot). The parallelized one chews up the 3g of available memory, unceremoniously throws exceptions (Exception in thread "qtp198198276-19” ), with the occasional: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1857) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2073) Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp198198276-19" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp198198276-14" java.lang.OutOfMemoryError: GC overhead limit exceeded and runs for tens of minutes (perhaps more - I always kill the process). Any ideas on what I can do to improve the situation? Thanks in advance. Cheers, -carl declare function csd_bl:get_child_orgs($orgs,$org) { let $org_id := $org/@entityID return if (functx:all-whitespace($org_id)) then () else let $c_orgs := $orgs[./parent[@entityID = $org_id]] let $t0 := trace($org_id, "creating func for ") let $t1 := trace(count($c_orgs), " func checks children: ") let $c_org_funcs:= for $c_org in $c_orgs return function() { ( trace($org_id, "executing child func for ") , $c_org, csd_bl:get_child_orgs($orgs,$c_org))} return xquery:fork-join($c_org_funcs) (: let $c_orgs := if (functx:all-whitespace($org_id)) then () else $orgs[./parent[@entityID = $org_id]] return for $c_org in $c_orgs let $t0 := trace($org_id, "processing children for ") return ($c_org,csd_bl:get_child_orgs($orgs,$c_org)) :) };

2 18

Serialize a string to unencode XML entities?
by Hondros, Constantine (ELS-AMS) 11 Sep '16

11 Sep '16

Ok, this must be possible. Can someone point me towards the correct serialization $params such that let $doi := <doi>10.1002/1521-4141(200207)32:7<2021::AID-IMMU2021>3.0.CO;2-J</doi> return serialize($doi, $params) will return (to a calling Java application) the UTF-8 string 10.1002/1521-4141(200207)32:7<2021::AID-IMMU2021>3.0.CO;2-J where the entities < and > have been replaced with the UTF-8 byte representation for U+003C and U+003E respectively? Thanks in advance, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Using the same database in many BaseX instances
by cmarchand＠oxiane.com 11 Sep '16

11 Sep '16

Hello, as discussed here few monthes ago, I do this : Install many instances (8) of BaseX on the same computer Start one instance, here called "main-instance" Create a database "library" , and load a filesystem directory into "library" Stop main-instance For each "other-instance", create a sym-link to main-instance/data/library Start "other-instances" On each instance, create 2 new databases "input" and "output", load a file in "input", process it and store result in "output". The process queries a lot "library", but never modifies "library". Then export "output" and drop databases "input" and "output". I have about 100 files to process; and I distribute them on the various instances I had a first test environment, with only 4 isntances, and everything was working perfectly. Now, I'm working on target environment, a bigger computer with 8 instances. Process fails, at least one time per instance, so between 8 and 12 times on 100 process. The exception is java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:136) at org.basex.data.DiskData.close(DiskData.java:151) at org.basex.core.Datas.unpin(Datas.java:54) at org.basex.core.cmd.Close.close(Close.java:45) at org.basex.query.QueryResources.close(QueryResources.java:110) at org.basex.query.QueryContext.close(QueryContext.java:596) at org.basex.query.QueryProcessor.close(QueryProcessor.java:251) at org.basex.core.cmd.AQuery.query(AQuery.java:124) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.server.ClientListener.run(ClientListener.java:136) If I change the sym-links by a true copy (cp -r main/data/library other/data/), it now works correctly, but the copy takes a long time (database is about 30gb), and it consumes disk space. The computer I use has many available cores, and many RAM to use, so I'd love to add other instance, many 16 more to reach 24 isntances, but if database grow, I'm going to have disk-space troubles... 1 - is it legal to share a database (and files) between many instances as far as database is access read-only ? 2 - is there a way to configure a database to be read-only ? 3 - why symbolic links were perfectly usable with 4 instances, and do not with 8 instances ? (Hum I have to try symlinks with only 4 instances on the new computer...) 4 - are there other specific things to check to do what I need ? Best regards, Christophe

3 2

bug (?) in fn:serialize
by Hans-Juergen Rennau 09 Sep '16

09 Sep '16

Dear BaseX team, is this a bug? serialize(<e> </e>) returns <e/> I need my blanks! Hans-Jürgen

4 6

read grant to anonymous user
by raffaele messuti 08 Sep '16

08 Sep '16

hi, i'm quite a newbie to basex (and generally xquery) a simple question (i'm not finding answers in the wiki): how i can grant a read permission to an anonymous (no password) user? the common use case is to access raw xml files via rest /rest/{db}/{file.xml}. this don't work: > grant read on [db] to * Admin user cannot be modified. as a solution i was thinking to proxy basex through nginx and use "proxy_set_header Authorization" thanks PS/ my first attempt to learn xquery is here, into a jupyter notebook maybe is interesting for someone: https://github.com/atomotic/bni-xquery/blob/master/bni-xquery.ipynb -- raffaele

1 1

Upgrade from 7.9 to 8.3
by Bram Vanroy 07 Sep '16

07 Sep '16

Hi everyone We're in a bit of a pickle. We upgraded to BaseX 8.3 after some hard drives failed, but now the PHP Basex client can't connect to the server any more. Here is the general idea: - basexserver is launched from server A, but the .basex configuration file it uses is stored on server B. This is the unchanged file we used in BaseX 7.9; - the data that the .basex file refers to is on server C, this has not changed either since the hard drive crash; - the PHP basex client is on server D, this file has not been changed either. So I am guessing that either we need to update one of our files, OR we'd need to re-import our data into the new BaseX server. Do you have any idea how to handle this? Kind regards Bram

2 1

Re: [basex-talk] REST API performance, massive disk IO
by Jens Erat 06 Sep '16

06 Sep '16

Hi Benedikt, sorry for not answering for a week. > our storage driver is the default aufs: > > Storage Driver: aufs > Root Dir: /var/lib/docker/aufs > Backing Filesystem: extfs > Dirs: 189 > Dirperm1 Supported: false > > ... based on ext4, also with default option (journaling only on metadata). That should be fine. Copy on write file systems in general require some attention when using databases (and other applications with massive small writes), but you don't use one (so you shouldn't have an issue with the file system here). > However, we have a persistent volume for the BaseX database files pointed to a regular folder on the host (ext4). To my understanding the storage driver isn't used in this case. You're right. > Is there a recommendation for a basex filesystem? The file system is not too important, as long as no copy on write is applied (chattr +t for BTRFS, but read up on that before using it). Ext4, XFS, ... should not require any special attention. I'm not aware anybody every considered different file systems and BaseX performance in a reasonable benchmark yet, but BaseX' access patterns do not differ substantially from other database systems. Regards, Jens -- Jens Erat Universität Konstanz Kommunikations-, Infomations-, Medienzentrum (KIM) Abteilung Basisdienste D-78457 Konstanz Mail: jens.erat(a)uni-konstanz.de

1 0

question on starting multiple instance of basex
by Wang, Genneva 04 Sep '16

04 Sep '16

Hi I’m trying to set up to having multiple instance of basex in my local dev environment, presumably to simulate production environment where our application is pointing to two different basex that running a different server. I’ve running basex as client server application in dev environment. Is there a way to do that? I’ve tried to start basex server with 2 different ports A and B, and also starting client with port A and B, but both seemed to be pointing to the same list of the databases that was created. Any tips would be greatly appreciated. Thanks! Regards, -Genneva

1 0