BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

DFDL module
by Kristian Kankainen 15 Jan '18

15 Jan '18

Hello, I started writing a small wrapper module for using DFDL with XQuery in BaseX. For this I simply use system calls. This is not an elegant way of doing it, so I wonder, has anyone else had any experience with using DFDL or even integrated it in a more sustainable way? My project will evolve here: https://github.com/keeleleek/basex-daffodil I would like to add a sentence on how cool DFDL is, but I wont. Best regards, Kristian K

2 2

Encoding problem with proc:system
by Kristian Kankainen 15 Jan '18

15 Jan '18

Hello all, I am running a system command with proc:system. The command should return xml, but because of non-ascii letters, the returned string gets scrambled. (I am aware that proc:system receives a string, proc:execute returns an element.) Running the system command in a shell works fine, I get this (snippet) <element>aapõ</element> When running the command through proc:system, I get this instead <element>aap....lement> Where the õ (= C3 B5 in hex representation) has been replaced with F5 BC AF B0. I get the same result specifying the encoding option of proc:system with "utf-8" or just leaving blank. My shell uses utf-8. How could I find out what encoding I need to use in the proc:system function? Best regards, Kristian K

2 3

java.lang.NegativeArraySizeException
by Stefanie Nageldiek 15 Jan '18

15 Jan '18

Hi! I'm trying to move a XSLT Transformation into XQuery. First step is Creating a db and trying to run the XSLT with XQuery. I have a small input file (50 MB) and a large input file (2GB) for testing. So I set up the db with the small input file, ran a parametrizised query calling the XSLT Transformation and everything went well. To be sure everything will work on the bigger db too, I tried to do the same, which fails - I had to allow for more memory which I did. (To be sure the stylesheet works with the big input file for the db I checked it (it takes 20 Minutes to produce a ~1 GB File). This is the error I get: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.6.7 Java: Oracle Corporation, 1.8.0_151 OS: Windows 10, amd64 Stack Trace: java.lang.NegativeArraySizeException at java.util.Arrays.copyOf(Unknown Source) at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:25) at org.basex.io.out.PrintOutput.print(PrintOutput.java:76) at org.basex.io.out.NewlineOutput.print(NewlineOutput.java:33) at org.basex.io.serial.MarkupSerializer.printChar(MarkupSerializer.java:254) at org.basex.io.serial.MarkupSerializer.attribute(MarkupSerializer.java:131) at org.basex.io.serial.Serializer.node(Serializer.java:416) at org.basex.io.serial.Serializer.node(Serializer.java:158) at org.basex.io.serial.StandardSerializer.node(StandardSerializer.java:105) at org.basex.io.serial.AdaptiveSerializer.node(AdaptiveSerializer.java:75) at org.basex.io.serial.Serializer.serialize(Serializer.java:109) at org.basex.io.serial.AdaptiveSerializer.serialize(AdaptiveSerializer.java:66) at org.basex.query.value.Value.serialize(Value.java:222) at org.basex.query.value.Value.serialize(Value.java:205) at org.basex.query.func.xslt.XsltTransform.read(XsltTransform.java:75) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:47) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:33) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:65) at org.basex.query.QueryContext.value(QueryContext.java:406) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:185) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:93) at org.basex.query.scope.MainModule$1.next(MainModule.java:125) at org.basex.query.QueryContext.cache(QueryContext.java:618) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:112) at org.basex.core.cmd.AQuery.query(AQuery.java:86) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.exec(GUI.java:474) at org.basex.gui.GUI.access$4(GUI.java:428) at org.basex.gui.GUI$6.run(GUI.java:416) Compiling: - pre-evaluate root() to document-node() - pre-evaluate map { "param1":"default", "param2":"products" } to map - pre-evaluate map { "method":"xml", "indent":"no", "omit-xml-declaration":"no" } to map - inline $path_2 - pre-evaluate concat("c:/work/xslt_to_xquery/", "xslt_post_processing_bmwp-2526.xsl") to xs:string - inline $style_3 Optimized Query: let $document_4 := xslt:transform(db:open-pre("big-xml",0)/*, "c:/work/xslt_to_xquery/i-know-which.xsl", map { "param1": "default", "param2": "products", ... }) return file:write(concat("c:/work/xslt_to_xquery/", db:name(.), "_out_", "default", "products", ".xml"), $document_4, map { "omit-xml-declaration": "no", "method": "xml", ... }) Query: declare variable $param1 external := "default"; declare variable $param2 external := "products"; let $path := 'c:/work/xslt_to_xquery/' let $style := concat($path, 'i-know-which.xsl') let $document := xslt:transform(/*, $style, map { "param1": $param1, "param2": $param2 }) return file:write(concat($path, db:name(.), '_out_', $param1, $param2, '.xml'), $document, map { "method": "xml", "indent": "no", "omit-xml-declaration":"no" }) Query plan: <QueryPlan compiled="true"> <GFLWOR> <Let> <Var name="$document" id="4"/> <XsltTransform name="transform(input,stylesheet[,params[,options]])"> <IterPath> <DBNode name="big-xml" pre="0"/> <IterStep axis="child" test="*"/> </IterPath> <Str value="c:/work/xslt_to_xquery/i-know-which.xsl" type="xs:string"/> <Map size="2"> <Str value="param1" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="param2" type="xs:string"/> <Str value="products" type="xs:string"/> </Map> </XsltTransform> </Let> <FileWrite name="write(path,data[,params])"> <FnConcat name="concat(atom1,atom2[,...])"> <Str value="c:/work/xslt_to_xquery/" type="xs:string"/> <DbName name="name(node)"> <ContextValue/> </DbName> <Str value="_out_" type="xs:string"/> <Str value="default" type="xs:string"/> <Str value="products" type="xs:string"/> <Str value=".xml" type="xs:string"/> </FnConcat> <VarRef> <Var name="$document" id="4"/> </VarRef> <Map size="3"> <Str value="omit-xml-declaration" type="xs:string"/> <Str value="no" type="xs:string"/> <Str value="method" type="xs:string"/> <Str value="xml" type="xs:string"/> <Str value="indent" type="xs:string"/> <Str value="no" type="xs:string"/> </Map> </FileWrite> </GFLWOR> </QueryPlan> Please let me know which information you need. Many thanks Steffie

2 1

update Java
by Giuseppe Celano 15 Jan '18

15 Jan '18

Hi, I write to ask whether it is now advisable to update to Java 9 (while using the BaseX 8.6.x). Thanks. Best, Giuseppe

2 1

Isilex on GIT
by xavierlaurent.salvador 08 Jan '18

08 Jan '18

I was told to post again this little news because of holydays, so sorry ;) if you wish to collaborate... it would be cool Hi List, just a small message to draw your attention on our new Git Repo dedicated to Isilex. We put all our stuff into : https://github.com/Isilex/download and all the web documentation at: https://isilex.github.io/easy-xml/ and https://isilex.github.io/easy-xml-fr/ Cheers, Envoyé depuis mon smartphone Samsung Galaxy.

1 0

How to set DBPath in BaseX version 8.6.7 as in version 7.3.1?
by Bang Pham Huu 05 Jan '18

05 Jan '18

Hello, I'm using BaseX version 7.3.1 and I can set dbpath where the collections are created with this code: new Set("dbpath", "/home/xxxx/basex-database").execute(context); However, when changing to BaseX version 8.6.7, this statement will throw exception "org.basex.core.BaseXException: Global option 'DBPATH' cannot be set." and instead it tries to create new collections to a default folder: /home/xxxx/BaseXData How can I set BaseX to work on existing dbpath folder (/home/xxxx/basex-database) as in version 7.3.1? Because I have existing data here and not the newly created one. Thanks,

2 1

BaseX Sharding with ActiveMQ & Docker
by James Sears 05 Jan '18

05 Jan '18

In the past I hit a scalability limit with BaseX - a billion+ nodes kind of a made querying it a bit slower than I liked. I thought I'd try and address this, so I’ve written some code and placed it in GitHub: https://github.com/jameshnsears/xqa-documentation What I've done is proof of concept, that's all - no way "finished". I'm emailing the list in the hope that what I've done so far might generate some constructive criticism. Maybe my approach has potential, maybe it doesn't? There are only four components so far, the first three are Docker containers: * an ActiveMQ instance * a load balancer * a shard * a command line client exists to load the XML, from file, into an ActiveMQ queue. The software requires close to zero configuration. For example, each shard you start will automatically receive XML from the load balancer. And the load balancer distributes XML so that each shard holds the same # of documents. There's a Travis project associated with the above link - it shows how easy it is to run the software end to end. So far my effort is all about ingesting the XML, before I move further I thought I'd canvass some feedback - so if anyone has any then please give it :-) Thanks.

2 1

Hundreds of thousands of java.io.IOException: Stream Closed in BaseX 7.3.1 from only catalina.out of Tomcat's log?
by Bang Pham Huu 05 Jan '18

05 Jan '18

Hello, I've been using BaseX (version 7.3.1 with Tomcat 8) to store an XML database from XML files and using XQuery to query the database. It works great before with simple Java servlet as I recall. After that, I updated to use Spring Boot (version 1.5.2) for this web application (only update from Java servlet to Spring Controller, the core to query BaseX doesn't change), and it seems this weird exception java.io.IOException: Stream Closed appears randomly by 100000 times in Tomcat's log (catalina.out). I tried to catch this exception in my web application but it is not able to catch. My web application's log (secore.log) also couldn't catch this error, only Tomcat's log (catalina.out) logs this error but with 100000 times (the log file is huge). Any idea to solve this problem will help me a lot? Here is the stack trace in catalina.out for 1 exception: https://pastebin.com/XGu7Vr2f And here is the configuration for log4j to log to both console and file (secore.log) and only console (catalina.out) contains this exception: https://pastebin.com/09ma3eyw Thanks,

2 1

Isilex on GitHub
by Xavier-Laurent SALVADOR 02 Jan '18

02 Jan '18

Hi List, just a small message to draw your attention on our new Git Repo dedicated to Isilex. We put all our stuff into : https://github.com/Isilex/download and all the web documentation at: https://isilex.github.io/easy-xml/ <https://isilex.github.io/easy-xml/> and https://isilex.github.io/easy-xml-fr/ Cheers, -- Ce message peut contenir des informations réservées exclusivement à son destinataire. Toute diffusion sans autorisation est interdite. Si vous n'en êtes pas le destinataire, merci de prendre contact avec l'expéditeur et de détruire ce message. *This email may contain material for the sole use of the intended recipient. Any forwarding without express permission is prohibited. If you are not the intended recipient, please contact the sender and delete all copies*.

2 1

Basex API: repeated execution of same query over different input
by Hondros, Constantine (ELS-AMS) 27 Dec '17

27 Dec '17

I want to be able to execute the same XQuery hundreds of thousands (even millions) of times over different XML input. Is there a way to avoid re-compiling the query itself every time? It looks to me like using either XQuery.execute(ctx) or the QueryProcessor class will end up re-parsing the query itself. I know that Basex is fast in this, but I will end up with a performance hit over time. Just wondering whether things have evolved since this thread: https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg03064.ht… Thanks, Constantine Hondros Senior Manager Content Analytics & Content Engineering Digital Transformation Elsevier BV Radarweg 29 | 1043 NX Amsterdam | the Netherlands Tel: +31 20 4852772 | Mob: +31610735630 E-mail: c.hondros(a)elsevier.com<mailto:m.nauta@elsevier.com> [cid:image001.jpg@01CBEEC6.C17A8AC0] Amsterdam | Chennai | Exeter | Oxford | Shannon | St. Louis ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk