January 2018 - BaseX-Talk - mailman.uni-konstanz.de

java.lang.NegativeArraySizeException
by Stefanie Nageldiek 15 Jan '18

15 Jan '18

Hi! I'm trying to move a XSLT Transformation into XQuery. First step is Creating a db and trying to run the XSLT with XQuery. I have a small input file (50 MB) and a large input file (2GB) for testing. So I set up the db with the small input file, ran a parametrizised query calling the XSLT Transformation and everything went well. To be sure everything will work on the bigger db too, I tried to do the same, which fails - I had to allow for more memory which I did. (To be sure the stylesheet works with the big input file for the db I checked it (it takes 20 Minutes to produce a ~1 GB File). This is the error I get: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.6.7 Java: Oracle Corporation, 1.8.0_151 OS: Windows 10, amd64 Stack Trace: java.lang.NegativeArraySizeException at java.util.Arrays.copyOf(Unknown Source) at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:25) at org.basex.io.out.PrintOutput.print(PrintOutput.java:76) at org.basex.io.out.NewlineOutput.print(NewlineOutput.java:33) at org.basex.io.serial.MarkupSerializer.printChar(MarkupSerializer.java:254) at org.basex.io.serial.MarkupSerializer.attribute(MarkupSerializer.java:131) at org.basex.io.serial.Serializer.node(Serializer.java:416) at org.basex.io.serial.Serializer.node(Serializer.java:158) at org.basex.io.serial.StandardSerializer.node(StandardSerializer.java:105) at org.basex.io.serial.AdaptiveSerializer.node(AdaptiveSerializer.java:75) at org.basex.io.serial.Serializer.serialize(Serializer.java:109) at org.basex.io.serial.AdaptiveSerializer.serialize(AdaptiveSerializer.java:66) at org.basex.query.value.Value.serialize(Value.java:222) at org.basex.query.value.Value.serialize(Value.java:205) at org.basex.query.func.xslt.XsltTransform.read(XsltTransform.java:75) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:47) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:33) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:65) at org.basex.query.QueryContext.value(QueryContext.java:406) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:185) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:93) at org.basex.query.scope.MainModule$1.next(MainModule.java:125) at org.basex.query.QueryContext.cache(QueryContext.java:618) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:112) at org.basex.core.cmd.AQuery.query(AQuery.java:86) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.exec(GUI.java:474) at org.basex.gui.GUI.access$4(GUI.java:428) at org.basex.gui.GUI$6.run(GUI.java:416) Compiling: - pre-evaluate root() to document-node() - pre-evaluate map { "param1":"default", "param2":"products" } to map - pre-evaluate map { "method":"xml", "indent":"no", "omit-xml-declaration":"no" } to map - inline $path_2 - pre-evaluate concat("c:/work/xslt_to_xquery/", "xslt_post_processing_bmwp-2526.xsl") to xs:string - inline $style_3 Optimized Query: let $document_4 := xslt:transform(db:open-pre("big-xml",0)/*, "c:/work/xslt_to_xquery/i-know-which.xsl", map { "param1": "default", "param2": "products", ... }) return file:write(concat("c:/work/xslt_to_xquery/", db:name(.), "_out_", "default", "products", ".xml"), $document_4, map { "omit-xml-declaration": "no", "method": "xml", ... }) Query: declare variable $param1 external := "default"; declare variable $param2 external := "products"; let $path := 'c:/work/xslt_to_xquery/' let $style := concat($path, 'i-know-which.xsl') let $document := xslt:transform(/*, $style, map { "param1": $param1, "param2": $param2 }) return file:write(concat($path, db:name(.), '_out_', $param1, $param2, '.xml'), $document, map { "method": "xml", "indent": "no", "omit-xml-declaration":"no" }) Query plan: <QueryPlan compiled="true"> <GFLWOR> <Let> <Var name="$document" id="4"/> <XsltTransform name="transform(input,stylesheet[,params[,options]])"> <IterPath> <DBNode name="big-xml" pre="0"/> <IterStep axis="child" test="*"/> </IterPath> <Str value="c:/work/xslt_to_xquery/i-know-which.xsl" type="xs:string"/> <Map size="2"> <Str value="param1" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="param2" type="xs:string"/> <Str value="products" type="xs:string"/> </Map> </XsltTransform> </Let> <FileWrite name="write(path,data[,params])"> <FnConcat name="concat(atom1,atom2[,...])"> <Str value="c:/work/xslt_to_xquery/" type="xs:string"/> <DbName name="name(node)"> <ContextValue/> </DbName> <Str value="_out_" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="products" type="xs:string"/> <Str value=".xml" type="xs:string"/> </FnConcat> <VarRef> <Var name="$document" id="4"/> </VarRef> <Map size="3"> <Str value="omit-xml-declaration" type="xs:string"/> <Str value="no" type="xs:string"/> <Str value="method" type="xs:string"/> <Str value="xml" type="xs:string"/> <Str value="indent" type="xs:string"/> <Str value="no" type="xs:string"/> </Map> </FileWrite> </GFLWOR> </QueryPlan> Please let me know which information you need. Many thanks Steffie

2 1

update Java
by Giuseppe Celano 15 Jan '18

15 Jan '18

Hi, I write to ask whether it is now advisable to update to Java 9 (while using the BaseX 8.6.x). Thanks. Best, Giuseppe

2 1

Isilex on GIT
by xavierlaurent.salvador 08 Jan '18

08 Jan '18

I was told to post again this little news because of holydays, so sorry ;) if you wish to collaborate... it would be cool Hi List, just a small message to draw your attention on our new Git Repo dedicated to Isilex. We put all our stuff into : https://github.com/Isilex/download and all the web documentation at: https://isilex.github.io/easy-xml/ and https://isilex.github.io/easy-xml-fr/ Cheers, Envoyé depuis mon smartphone Samsung Galaxy.

1 0

How to set DBPath in BaseX version 8.6.7 as in version 7.3.1?
by Bang Pham Huu 05 Jan '18

05 Jan '18

Hello, I'm using BaseX version 7.3.1 and I can set dbpath where the collections are created with this code: new Set("dbpath", "/home/xxxx/basex-database").execute(context); However, when changing to BaseX version 8.6.7, this statement will throw exception "org.basex.core.BaseXException: Global option 'DBPATH' cannot be set." and instead it tries to create new collections to a default folder: /home/xxxx/BaseXData How can I set BaseX to work on existing dbpath folder (/home/xxxx/basex-database) as in version 7.3.1? Because I have existing data here and not the newly created one. Thanks,

2 1

BaseX Sharding with ActiveMQ & Docker
by James Sears 05 Jan '18

05 Jan '18

In the past I hit a scalability limit with BaseX - a billion+ nodes kind of a made querying it a bit slower than I liked. I thought I'd try and address this, so I’ve written some code and placed it in GitHub: https://github.com/jameshnsears/xqa-documentation What I've done is proof of concept, that's all - no way "finished". I'm emailing the list in the hope that what I've done so far might generate some constructive criticism. Maybe my approach has potential, maybe it doesn't? There are only four components so far, the first three are Docker containers: * an ActiveMQ instance * a load balancer * a shard * a command line client exists to load the XML, from file, into an ActiveMQ queue. The software requires close to zero configuration. For example, each shard you start will automatically receive XML from the load balancer. And the load balancer distributes XML so that each shard holds the same # of documents. There's a Travis project associated with the above link - it shows how easy it is to run the software end to end. So far my effort is all about ingesting the XML, before I move further I thought I'd canvass some feedback - so if anyone has any then please give it :-) Thanks.

2 1

Hundreds of thousands of java.io.IOException: Stream Closed in BaseX 7.3.1 from only catalina.out of Tomcat's log?
by Bang Pham Huu 05 Jan '18

05 Jan '18

Hello, I've been using BaseX (version 7.3.1 with Tomcat 8) to store an XML database from XML files and using XQuery to query the database. It works great before with simple Java servlet as I recall. After that, I updated to use Spring Boot (version 1.5.2) for this web application (only update from Java servlet to Spring Controller, the core to query BaseX doesn't change), and it seems this weird exception java.io.IOException: Stream Closed appears randomly by 100000 times in Tomcat's log (catalina.out). I tried to catch this exception in my web application but it is not able to catch. My web application's log (secore.log) also couldn't catch this error, only Tomcat's log (catalina.out) logs this error but with 100000 times (the log file is huge). Any idea to solve this problem will help me a lot? Here is the stack trace in catalina.out for 1 exception: https://pastebin.com/XGu7Vr2f And here is the configuration for log4j to log to both console and file (secore.log) and only console (catalina.out) contains this exception: https://pastebin.com/09ma3eyw Thanks,

2 1

Isilex on GitHub
by Xavier-Laurent SALVADOR 02 Jan '18

02 Jan '18

Hi List, just a small message to draw your attention on our new Git Repo dedicated to Isilex. We put all our stuff into : https://github.com/Isilex/download and all the web documentation at: https://isilex.github.io/easy-xml/ <https://isilex.github.io/easy-xml/> and https://isilex.github.io/easy-xml-fr/ Cheers, -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*.

2 1