March 2016 - BaseX-Talk - mailman.uni-konstanz.de

slow GUI-editor when using package-manager eXPath
by Rob Stapper 10 Apr '16

10 Apr '16

Hi Christian, New tools, new challenges. I've rearranged my source-code in packages/libraries conform the eXPath-specs in BaseX now. But since than the GUI-editor is becoming very slow, even unworkably slow. What is causing this and can you give me any tips how speed up the GUI-editor? I've got 7 libraries now. The one with the largest interface counts 11 entries in file:expath-pkg.xml, all public entries, the smallest library counts one entry. TIA, Rob

3 12

BaseX optimizer performance on REx-generated parser
by Gunther Rademacher 05 Apr '16

05 Apr '16

Hi, in a recent update of REx parser generator (v5.37), some type specifications were removed from generated XQuery and XSLT functions. This affects tail-recursive functions, and it was done because it turned out that MarkLogic fails to optimize tail calls, presumably due to an explicit type check caused by the type specification (see http://markmail.org/message/gxi26da4crk2v5ge ). Now it was reported that this change causes performance problems with BaseX (see https://twitter.com/apb1704/status/714219874441146368 ). I reproduced the problem as follows: - download the XQuery 3.1 grammar from http://bottlecaps.de/rex/CR-xquery-31-20151217.ebnf - generate a parser from it on http://bottlecaps.de/rex/ using command line options: -xquery -tree -main - run the generated parser, using this command line basex -binput={42} CR-xquery-31-20151217.xquery This works OK, but it takes about 80 seconds. Some analysis showed that the time can be influenced by putting back the type specification to function p:map2, i.e. declaring 'as xs:integer' as the result type: declare function p:map2($c as xs:integer, $lo as xs:integer, $hi as xs:integer) as xs:integer This variant completes in less than 3 seconds. But even when declaring a return type of as xs:integer? which might be the type that can be inferred statically, it completes fast. Is this possibly a problem with the optimizer? Which variant of generated code would be preferable for BaseX - typed or untyped? Thanks, Gunther

3 21

Async Module
by Christian Grün 03 Apr '16

03 Apr '16

Hi everyone, As some of you know, James Wright has presented his excellent work on the Promise Module in Prague [1,2]. The promise:fork-join function is now part of BaseX, along with some additional functions for asynchronous query processing [3,4]. Your feedback is welcome as usual, Christian [1] https://github.com/james-jw/xq-promise/ [2] http://archive.xmlprague.cz/2016/files/xmlprague-2016-proceedings.pdf [3] http://docs.basex.org/wiki/Async_Module [4] http://files.basex.org/releases/latest/

3 3

[ANN] Openshift BaseX quick start v0.6
by Andy Bunce 03 Apr '16

03 Apr '16

Hi, A new version is available at https://github.com/Quodatum/openshift-basex-quick-start. New features: - Enable REST and WebDAV - support for setting initial users - support for setting initial data - Use Java 8 Thanks to Michael Sperberg-McQueen and Günter Dunz-Wolff for helpful comments on the previous version. /Andy

3 6

Internal optimization
by Gabriel Tessarolli 30 Mar '16

30 Mar '16

Hi, First of all, thank you for the excellent software you produce and maintain! Keep up the good work. I've been using BaseX for some academic experiments on XQuery processing, and I got this situation that you guys can probably explain. Here is some context: - I am using version 8.2.3. - Database 'expdb' was created with default options of that version, using 'auction.xml' document generated from Xmark benchmark [1]. - BaseX is running with default options too. - What the query does is irrelevant. When I execute this query: for $pe in doc('expdb/auction.xml')/site/people/person for $cat in doc('expdb/auction.xml')/site/categories/category[position() >= 1 and position() < 101] where count($pe/profile/interest) > 3 and $pe/profile/interest/@category = $cat/@id return <match> <person>{$pe/name}</person> <category>{$cat/name}</category> </match> the resulting optimized query (in 'Query Info' window, on GUI) is this: for $pe_0 in *db:open-pre*("expdb",0)/*:site/*:people/*:person[3.0 < count(*:profile/*:interest)] for $cat_1 in *db:open-pre*("expdb",0)/*:site/*:categories/*:category[position() = 1 to 100][(@*:id = $pe_0/*:profile/*:interest/@*:category)] return element match { (element person { ($pe_0/*:name) }, element category { ($cat_1/*:name) }) } If I change the original query to (note that I am only switching the position of 'for' clauses): for $cat in doc('expdb/auction.xml')/site/categories/category[position() >= 1 and position() < 101] for $pe in doc('expdb/auction.xml')/site/people/person where count($pe/profile/interest) > 3 and $pe/profile/interest/@category = $cat/@id return <match> <person>{$pe/name}</person> <category>{$cat/name}</category> </match> the optimized query changes to: for $cat_0 in *db:open-pre*("expdb",0)/*:site/*:categories/*:category[position() = 1 to 100] for $pe_1 in *db:attribute*("expdb", $cat_0/@*:id)/self::*:category/parent::*:interest/parent::*:profile/parent::*:person[3.0 < count(*:profile/*:interest)] return element match { (element person { ($pe_1/*:name) }, element category { ($cat_0/*:name) }) } You can see that BaseX was able to use attribute index to optimize last query, reducing its execution time (by a lot!). My question is: what is the explanation for this behavior? Thank you in advance! Gabriel Tessarolli --- [1] http://www.xml-benchmark.org/

2 2

detection of external variables
by Andy Bunce 30 Mar '16

30 Mar '16

Hi, I am looking for a way to determine for a module what external variables are defined. Neither inspect:module [1] or inspect:xqdoc seem to return this information. I wonder if adding an *external *attribute to the inspect:module o/p for variables (and functions?) would be a solution. e.g. <variable name="samples:test-string" uri="http://basex.org/modules/samples" type="xs:string" *external="true"*> <description>This is a sample string.</description> </variable> /Andy [1] http://docs.basex.org/wiki/Inspection_Module#inspect:module

2 4

Out of memory getting document from Client API
by buddyonweb-software＠yahoo.com 29 Mar '16

29 Mar '16

Using the client API I am trying to retrieve a 600MB file from BaseX. Version I am using is BaseX 8.3. I have successfully stored this document using the client API without issue. As an aside, I have update the basexserver script to include 4GB of memory as Java command line option. Here is the snippet from that script:# Options for virtual machine (can be extended by global options) BASEX_JVM="-Xmx4g $BASEX_JVM" In my Java client side application I have memory set to 3 GB of memory (script snippet below):exec "$JAVACMD" -Xms1g -Xmx3g .....rest of args..... Here is a snippet of my Java code. Note that this works just fine when operation on smaller documents, but I wanted to stress test the code. String query = "doc('theCollectionName/extract.xml')";ByteArrayOutputStream baos = new ByteArrayOutputStream(); ClientSession session = new ClientSession(host, port, user, pwd);session.setOutputStream(baos);ClientQuery cq = session.query(query); cq.execute(); However, I keep getting out of memory with the below stack trace. I would think that the amount of space I am providing in both thebasexserver script (4GB) and my client side app (3GB) would be plenty to read in a 600MB file from BaseX. Is there something I am overlookingor something I am not doing correctly? Any insights/feedback is appreciated. Exception in thread "queue" java.lang.OutOfMemoryError: Java heap space at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149) at java.lang.StringCoding.decode(StringCoding.java:193) at java.lang.StringCoding.decode(StringCoding.java:254) at java.lang.String.<init>(String.java:534) at java.io.ByteArrayOutputStream.toString(ByteArrayOutputStream.java:221) at org.basex.api.client.ClientSession.exec(ClientSession.java:261) at org.basex.api.client.ClientQuery.execute(ClientQuery.java:102)

1 0

REST-Interface /Openshift/ Security
by Günter Dunz-Wolff 29 Mar '16

29 Mar '16

Hi all, I'm in the beginning of a BaseX basexhttp installation on OpenShift. I'm using Openshift quick start for BaseX from Andy Bunce. For my tests, I'm working with a simple factbook-db. Everything is working, but I can't figure out, how to protect the database to be deleted via the REST-Interface: With a simple curl -i -X DELETE "http://xxx-xxx.rhcloud.com/rest/factbook" the Database is dropped. In the database, there is only the user admin. In web.xml the REST-servlet is configured with <servlet> <servlet-name>REST</servlet-name> <servlet-class>org.basex.http.rest.RESTServlet</servlet-class> <init-param> <param-name>org.basex.user</param-name> <param-value>admin</param-value> </init-param> <init-param> <param-name>org.basex.password</param-name> <param-value>xxx</param-value> </init-param> </servlet> <servlet-mapping> <servlet-name>REST</servlet-name> <url-pattern>/rest/*</url-pattern> </servlet-mapping> What part of the security-management am I missing? Thanks a lot for any help. Best regards, Günter Dunz-Wolff

2 2

async module vs xq-promise
by Maximilian Gärber 28 Mar '16

28 Mar '16

Hi all, I just started to use the async module and I guess I am out of sync about the relation to xq-promise: Basex now has asyn:fork-join which exectues in parallel and comes back with all results xq-promise has p:fork-join which seems to be the previous implementation Could I swap one for the other? Side note: async:eval which works like xquery:eval is great, but I have to do something like this for functions: ``` let $ids := async:eval("declare variable $f external; declare variable $name external; $f($name)", map { 'f': '_:get-meta', 'name' : $name }) return $ids ``` With get-meta being a local function that accepts a single parameter. Is there an easier way to do this? Regards, Max

2 1

Re: [basex-talk] using Xerces2 with schema-validation
by Christian Grün 22 Mar '16

22 Mar '16

Hi Rob (cc to the list), You may be better off using the ZIP distribution. Usually, it’s mostly the basex.jar and basex-api.jar files that need to be updated. Hope this helps, Christian On Tue, Mar 22, 2016 at 3:04 PM, Rob Stapper <r.stapper(a)lijbrandt.nl> wrote: > HI Christian, > > Oops, not too clever. But to my defense, in my original code I use an xsd-file starting with the xml-processor for defining the version: > <?xml version="1.1" encoding="UTF-8"?> > > It turns out that my ignorance of java's classpath and BaseX's installation procedure are the cause of this trouble: > - Environment-variable classpath is ignored when java is started with the -cp option > - Manually copying the Xerces-file to the BaseX-library and changes to the basexgui.bat-file[1] are getting undone when installing a new BaseX-version. > > My solution for the moment is to keep an edited basexgui.bat-file in my document directory which I copy to BaseX-bin directory every time I install a new Basex-version. > > What is the best strategy for preserving for example classpath-extensions when installing a new version of BaseX? > Is it an idea preserving all non BaseX batch-file in BaseX/bin-directory, in my case: "myBasexgui.bat", when executing the BaseX-installation-procedure? > > [1] line added to basexgui.bat: "for %%a in (L:\xerces-2_11_0-xml-schema-1.1-beta\*.jar) do set CP=!CP!;%%a" > > Take care, > Rob > > PS, calling Xerces2's validation works now as does XML-schema's assert. Thanx. > > -----Oorspronkelijk bericht----- > Van: Christian Grün [mailto:christian.gruen@gmail.com] > Verzonden: maandag 21 maart 2016 18:11 > Aan: Rob Stapper > CC: BaseX > Onderwerp: Re: [basex-talk] using Xerces2 with schema-validation > > Hi Rob, > > it seems you forgot the version argument? > > Cheers, > Christian > > > On Mon, Mar 21, 2016 at 12:06 PM, Rob Stapper <r.stapper(a)lijbrandt.nl> wrote: >> Hi, >> >> >> >> I would like to use xerces2 for schema-validation. See attached file, >> but I can’t get it activated from within BaseX. >> >> I’ve read the chapter on validation [1] and followed the instructions : >> copying the 4 files to the library. >> >> I can see that these 4 file are part of the –cp option when calling java. >> >> But the use of assert-element while validating gives an error: >> >> >> >> Stopped at L:/Program Files/BaseX/demo-xsd-validation.xq, 1/14: >> >> [bxerr:BXVA0001] Validation failed: >> file:/C:/Users/Rob/AppData/Local/Temp/BaseX-375999151462208291.tmp, 6:33: >> s4s-elt-invalid-content.1: The content of '#AnonType_assert-demo' is >> invalid. Element 'assert' is invalid, misplaced, or occurs too often. >> >> >> >> [1] http://docs.basex.org/wiki/Validation_Module#XML_Schema_Validation >> >> >> >> What am I doing wrong? >> >> >> >> TIA for your response, >> >> Rob >> >> >> >> PS I’m using the latest or one of the latest snapshot and java 1.8 >> runtime on a windows-Vista machine >

2 1