- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] FYI: Balisage talk involving BaseX
by Christian Grün 13 Sep '16

13 Sep '16

Hi all, I’d like to point you to an interesting talk from Amanda Galtman on RESTXQ and JavaScript… http://balisage.net/Proceedings/vol17/html/Galtman01/BalisageVol17-Galtman0… http://balisage.net/Proceedings/vol17/author-pkg/Galtman01/BalisageVol17-Ga… Have fun, Christian

1 0

csv:parse in the age of XQuery 3.1
by Joe Wicentowski 12 Sep '16

12 Sep '16

Dear BaseX developers, I noticed in example 3 under http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with option { 'format': 'map' } returns a map of maps, with hardcoded row numbers: map { 1: map { "City": "Newton", "Name": "John" }, 2: map { "City": "Oldtown", "Name": "Jack" } } Using maps, which are unordered, to represent something ordered like rows in a CSV, hardcoded row numbers are necessary for reassembling the map in document order. I assume this was a necessary approach when the module was developed in the map-only world of XQuery 3.0. Now that 3.1 supports arrays, might an array of maps be a closer fit for CSV parsing? array { map { "City": "Newton", "Name": "John" }, map { "City": "Oldtown", "Name": "Jack" } } I'm also curious, do you know of any efforts to create an EXPath spec for CSV? Putting spec and CSV in the same sentence is dangerous, since CSV is a notoriously under-specified format: "The CSV file format is not standardized" (see https://en.wikipedia.org/wiki/Comma-separated_values) But perhaps there is a common enough need for CSV parsing that such a spec would benefit the community? I thought I'd start by asking here, since BaseX's seems to be the most developed (or only?) CSV module in XQuery. Then there's the question of how to approach implementations of such a spec. While XQuery is probably capable of parsing and serializing small enough CSV, CSVs do get large and naive processing with XQuery would tend to run into memory issues (as I found with xqjson). This means implementations would tend to write in a lower-level language. eXist, for example, uses Jackson for fn:parse-json(). I see Jackson has a CSV extension too: https://github.com/FasterXML/jackson-dataformat-csv. Any thoughts on the suitability of XQuery for the task? Joe

6 21

8.5.3 and xquery:invoke ...
by Marco Lettere 11 Sep '16

11 Sep '16

Hello again, at the end I managed to reproduce the issue we are currently facing with xquery:invoke in 8.5.3 (and also in latest snapshot). Please consider the three queries [1,2,3]. The store [2] into a subdirectory src of the directory containing [1] and store [3] into a directory src2 of src. You should get the error message [4]. If instead you change the parameter of xquery:invoke to "src/src2/q2.xqm" it works. Seems like the reference for static-base-uri in [2] remains the one of [1] even if it is reported by static-base-uri() to be correctly "/home/user/src/q1.xqm"(I've double-checked this). Sorry it's not clear to me whether this is meant to be the correct behaviour ... Thanks for any help, Marco. [1] q.xqm import module namespace q1="urn:q1" at "src/q1.xqm"; q1:f() [2] q1.xqm module namespace q1 = "urn:q1"; declare function q1:f(){ xquery:invoke("src2/q2.xqm") }; [3] q2.xqm 1 = 1 [4] Stopped at /home/user/src/q1.xqm, 4/16: [FODC0002] Resource 'src2/q2.xqm' does not exist.

3 4

question on starting multiple instance of basex
by Wang, Genneva 11 Sep '16

11 Sep '16

Hi I’m trying to set up to having multiple instance of basex in my local dev environment, presumably to simulate production environment where our application is pointing to two different basex that running a different server. I’ve running basex as client server application in dev environment. Is there a way to do that? I’ve tried to start basex server with 2 different ports A and B, and also starting client with port A and B, but both seemed to be pointing to the same list of the databases that was created. Any tips would be greatly appreciated. Thanks! Regards, -Genneva

2 1

fork-join and recursive function
by Carl Leitner 11 Sep '16

11 Sep '16

Hi All, I have hierarchical information encoded an XML document that looks something like: <organization entityID=“1”> <organization entityID=“2"> <parent entityID=“1”> </organization> <organization entityID=“3"> <parent entityID=“2”> </organization> <organization entityID=“4"> <parent entityID=“1”> </organization> There are around 80,000 entries like this and I need to regular do extractions of sub-hiearchies (see the commented out version of the below function). The commented-out version runs in about a minute using a modest amount of memory. Hoping to take advantage of this <https://github.com/BaseXdb/basex/commit/ac86bbbc3cd1f71461ce94d803cab46f21e…>, I modified the function (the uncommented version) which tries to use xquery:fork-join in 8.5.2beta (the July 12th snapshot). The parallelized one chews up the 3g of available memory, unceremoniously throws exceptions (Exception in thread "qtp198198276-19” ), with the occasional: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1857) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2073) Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp198198276-19" java.lang.OutOfMemoryError: GC overhead limit exceeded Exception in thread "qtp198198276-14" java.lang.OutOfMemoryError: GC overhead limit exceeded and runs for tens of minutes (perhaps more - I always kill the process). Any ideas on what I can do to improve the situation? Thanks in advance. Cheers, -carl declare function csd_bl:get_child_orgs($orgs,$org) { let $org_id := $org/@entityID return if (functx:all-whitespace($org_id)) then () else let $c_orgs := $orgs[./parent[@entityID = $org_id]] let $t0 := trace($org_id, "creating func for ") let $t1 := trace(count($c_orgs), " func checks children: ") let $c_org_funcs:= for $c_org in $c_orgs return function() { ( trace($org_id, "executing child func for ") , $c_org, csd_bl:get_child_orgs($orgs,$c_org))} return xquery:fork-join($c_org_funcs) (: let $c_orgs := if (functx:all-whitespace($org_id)) then () else $orgs[./parent[@entityID = $org_id]] return for $c_org in $c_orgs let $t0 := trace($org_id, "processing children for ") return ($c_org,csd_bl:get_child_orgs($orgs,$c_org)) :) };

2 18

Serialize a string to unencode XML entities?
by Hondros, Constantine (ELS-AMS) 11 Sep '16

11 Sep '16

Ok, this must be possible. Can someone point me towards the correct serialization $params such that let $doi := <doi>10.1002/1521-4141(200207)32:7<2021::AID-IMMU2021>3.0.CO;2-J</doi> return serialize($doi, $params) will return (to a calling Java application) the UTF-8 string 10.1002/1521-4141(200207)32:7<2021::AID-IMMU2021>3.0.CO;2-J where the entities < and > have been replaced with the UTF-8 byte representation for U+003C and U+003E respectively? Thanks in advance, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Using the same database in many BaseX instances
by cmarchand＠oxiane.com 11 Sep '16

11 Sep '16

Hello, as discussed here few monthes ago, I do this : Install many instances (8) of BaseX on the same computer Start one instance, here called "main-instance" Create a database "library" , and load a filesystem directory into "library" Stop main-instance For each "other-instance", create a sym-link to main-instance/data/library Start "other-instances" On each instance, create 2 new databases "input" and "output", load a file in "input", process it and store result in "output". The process queries a lot "library", but never modifies "library". Then export "output" and drop databases "input" and "output". I have about 100 files to process; and I distribute them on the various instances I had a first test environment, with only 4 isntances, and everything was working perfectly. Now, I'm working on target environment, a bigger computer with 8 instances. Process fails, at least one time per instance, so between 8 and 12 times on 100 process. The exception is java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:136) at org.basex.data.DiskData.close(DiskData.java:151) at org.basex.core.Datas.unpin(Datas.java:54) at org.basex.core.cmd.Close.close(Close.java:45) at org.basex.query.QueryResources.close(QueryResources.java:110) at org.basex.query.QueryContext.close(QueryContext.java:596) at org.basex.query.QueryProcessor.close(QueryProcessor.java:251) at org.basex.core.cmd.AQuery.query(AQuery.java:124) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.server.ClientListener.run(ClientListener.java:136) If I change the sym-links by a true copy (cp -r main/data/library other/data/), it now works correctly, but the copy takes a long time (database is about 30gb), and it consumes disk space. The computer I use has many available cores, and many RAM to use, so I'd love to add other instance, many 16 more to reach 24 isntances, but if database grow, I'm going to have disk-space troubles... 1 - is it legal to share a database (and files) between many instances as far as database is access read-only ? 2 - is there a way to configure a database to be read-only ? 3 - why symbolic links were perfectly usable with 4 instances, and do not with 8 instances ? (Hum I have to try symlinks with only 4 instances on the new computer...) 4 - are there other specific things to check to do what I need ? Best regards, Christophe

3 2

bug (?) in fn:serialize
by Hans-Juergen Rennau 09 Sep '16

09 Sep '16

Dear BaseX team, is this a bug? serialize(<e> </e>) returns <e/> I need my blanks! Hans-Jürgen

4 6

read grant to anonymous user
by raffaele messuti 08 Sep '16

08 Sep '16

hi, i'm quite a newbie to basex (and generally xquery) a simple question (i'm not finding answers in the wiki): how i can grant a read permission to an anonymous (no password) user? the common use case is to access raw xml files via rest /rest/{db}/{file.xml}. this don't work: > grant read on [db] to * Admin user cannot be modified. as a solution i was thinking to proxy basex through nginx and use "proxy_set_header Authorization" thanks PS/ my first attempt to learn xquery is here, into a jupyter notebook maybe is interesting for someone: https://github.com/atomotic/bni-xquery/blob/master/bni-xquery.ipynb -- raffaele

1 1

Upgrade from 7.9 to 8.3
by Bram Vanroy 07 Sep '16

07 Sep '16

Hi everyone We're in a bit of a pickle. We upgraded to BaseX 8.3 after some hard drives failed, but now the PHP Basex client can't connect to the server any more. Here is the general idea: - basexserver is launched from server A, but the .basex configuration file it uses is stored on server B. This is the unchanged file we used in BaseX 7.9; - the data that the .basex file refers to is on server C, this has not changed either since the hard drive crash; - the PHP basex client is on server D, this file has not been changed either. So I am guessing that either we need to update one of our files, OR we'd need to re-import our data into the new BaseX server. Do you have any idea how to handle this? Kind regards Bram

2 1