- BaseX-Talk - mailman.uni-konstanz.de

Hitting the Tail Recursion wall
by Andreas Mixich 02 Apr '19

02 Apr '19

Hi, I am trying to implement the algorithm outlined at https://tools.ietf.org/html/rfc3986#section-5.2.4 (Two other implementations are at [1] and [2]) The following function is my work in progress. It seems, all goes fine, till the last `else`, which is commented with `(: E :)`. I hit the tail recursion barrier and I do not see why. The function may not (yet) make sense, since I am still experimenting. `local:remove-dot-segments("/a/b/c/./../../g",[])` ``` (:~ Remove dot segments from a URI path component according to RFC 3986 Section 5.2.4 @param $path the path component of an URL @param $out an empty (!) array @return the path component with the relative part (dot segments) removed and resolved to the absolute path. @see https://tools.ietf.org/html/rfc3986#section-5.2.4 :) declare function local:remove-dot-segments( $path as xs:string, $out as array(*)) as item()* { concat("in: ", $path, " out: ", string-join(array:flatten($out))), if (not(contains($path, "."))) then $path else if (string-length($path) > 0) then (: A :) if (substring($path,1,3) = "../") then local:remove-dot-segments(substring($path, 4), $out) else if (substring($path, 1, 2) = "./") then local:remove-dot-segments(substring($path, 3), $out) else (: B :) if (substring($path,1,3) = "/./") then local:remove-dot-segments(concat("/", substring($path, 4)), $out) else if (substring($path, 1, 2) = "/.") then local:remove-dot-segments(concat("/", substring($path, 3)), $out) else (: C :) if (substring($path, 1, 4) = "/../") then let $ret := array:remove($out, array:size($out)) return local:remove-dot-segments(concat("/", substring($path, 5)), $ret) else if (substring($path, 1, 3) = "/..") then let $ret := array:remove($out, array:size($out)) return local:remove-dot-segments(concat("/", substring($path, 4)), $ret) else (: D :) if ($path = ".." or $path = ".") then local:remove-dot-segments(concat(substring((), 4), ""), $out) else (: E :) let $ret := if (starts-with($path, "/")) then array:append($out, concat("/", substring-before(substring($path,2), "/"))) else array:append($out, substring-before(substring($path,1), "/")) return local:remove-dot-segments($path, $ret) else $path }; ``` Sorry for throwing such a big function at you, but I do not find the place, where the stack overflow gets triggered. It does not happen, if I return an empty sequence at `else (: E :)`. If I replace that part (after `else (: E :)`) with `local:remove-dot-segments($path, $out)` I get the stack overflow as well, which is, what confuses me the most. [1]https://github.com/ariutta/remove-dot-segments/blob/master/index.js [2] https://gist.github.com/rdlowrey/5f56cc540099de9d5006 -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 4

Using local xqj
by Gardner, Arthur 01 Apr '19

01 Apr '19

Hi! I've downloaded BaseX 9.2 (Windows distro). One thing I want to try is the xqj implementation, running locally on the server. I found the class net.xqj.basex.local.BaseXXQDataSource But when I try getConnection() on an instance of this, I get the following: java.lang.NoClassDefFoundError: org/basex/core/Proc I've searched for this class, but without success. Any ideas out there? Arthur Gardner | Infosys Ltd MetLife | Clarks Summit, PA Office 570-587-6898 | Mobile 262-442-7472 The information contained in this message may be CONFIDENTIAL and is for the intended addressee only. Any unauthorized use, dissemination of the information, or copying of this message is prohibited. If you are not the intended addressee, please notify the sender immediately and delete this message.

2 2

Quick Windows issue: Invalid reply on port 1984, works fine on port 8984
by Jim Salmons 01 Apr '19

01 Apr '19

Hello BaseX folk, I have just installed version 9.1.2 and am starting to learn BaseX in support of my post-cancer #PayItForward Bonus Round #CitizenScientist research in #DigitalHumanities and #MachineLearning. My goal is to "cut out the middleman" of writing intermediate JSON and XML fragment files in order to update the #MAGAZINEgts ground-truth storage file that I am developing for a digital collection of Softalk Apple magazine at the Internet Archive. With BaseX and its Python interface, I want to write new and updated document structure and content depiction metadata directly to a local copy of this ground-truth master file which will then be easily popped out of the database as an XML file for uploading to the Softalk Apple collection at the Internet Archive. I hit a bump, reported to the Github Issue queue, where my Windows 10 installation can not open the BaseX GUI exe launcher. The work-around, as noted in my issue report, is to use the Getting Started advice to execute the jar directly. By creating a desktop icon that launches the execuable jar in the installation folder, the GUI launches as expected. With that minor issue behind me, I am wondering why my attempts to connect to the server via localhost:1984 port does not work while everything is fine and dandy when hitting port 8984. Port 1984 returns an invalid HTTP response. (Note: Other ports that are not assigned to anything else return a connection refused, so this invalid response is not the same as the server not being there.) If I keep a Powershell window open with the server running, every once in a while (after several minutes or more) the server throws this error: java.lang.NullPointerException at org.basex.core.Context.perm(Context.java:252) at org.basex.core.Command.run(Command.java:250) at org.basex.core.Command.run(Command.java:143) at org.basex.core.cmd.Close.close(Close.java:54) at org.basex.server.ClientListener.close(ClientListener.java:232) at org.basex.BaseXServer$1.run(BaseXServer.java:157) at java.base/java.util.TimerThread.mainLoop(Timer.java:556) at java.base/java.util.TimerThread.run(Timer.java:506) which may be related to port 1984 not responding, a firewall issue perhaps? Honestly, I don't see anything that looks like a BaseX process or service running in Task Manager (other than a generic 'Java(TM) Platform SE binary' process, and there is nothing that looks related in the Windows Firewall entries. I'm still investigating this here, but if anyone knows what is akimbo, I would greatly appreciate your advice. Thank you... and looking forward to being a new BaseX community member. :-) Happy-Healthy Vibes, -: Jim Salmons :- Jim Salmons Twitter: @Jim_Salmons https://www.researchgate.net/profile/Jim_Salmons (my #CitizenScientist profile) www.medium.com/@Jim_Salmons/ (my mostly #DigitalHumanities writing)

3 4

Access variable database without locking all databases
by Leah Chatkeonopadol 29 Mar '19

29 Mar '19

Hi, I'm using BaseX as a database for a web application with RESTXQ API endpoints. Each endpoint has a database name specified as a path param, for example: declare %rest:GET %rest:path('data/{$database}/read') function readData($database as xs:string) { let $doc := fn:doc($database || '/data.xml') return $doc }; All of the queries inside the function are only run against a single database, but as described in the transaction management <http://docs.basex.org/wiki/Transaction_Management#Limitations> documentation, using fn:doc with a variable will lock all the databases. Is there any way around this? Any suggestions would be much appreciated. Thanks!

1 0

Higher order functions in XSLT
by nikos dimitrakas 28 Mar '19

28 Mar '19

Hi! Is there something in BaseX that prohibits using higher order functions in XSLT (called with xslt:transform)? for-each($somenodes, function($q) { string($q) }) and for-each($somenodes, string#1) both work in XQuery in BaseX, but when I use the same call in an XSLT that I call with xslt:transform, I get the following message XPST0003: Inline functions require support for higher-order-functions, which needs Saxon-PE or higher. I am using Saxon EE 9.9.1.2 (also tried PE 9.9.1.2) The XSLT works fine if I run it with Saxon directly without using BaseX and the function xslt:transform. Is there something I can do to get this to work, or is this out of my control? Regards /nikos

3 3

Compare List Membership in XQuery
by Chris Yocum 26 Mar '19

26 Mar '19

Hello, I have a question that is more XQuery based than BaseX specifically but I thought I would pose it here to see if any one knows. The basic problem statement is: given a list of tags that have an id attribute and a list of target attributes which should correspond to those ids, are there any target attrbutes which reference ids which do not exist? Basically, I am trying to link up a tag called biblFull or biblStruct which have an id attribute with a tag named title which has a target attribute which should match up with one of the two tags (biblFull or biblStruct) above. I have formulated the query below which I believe *should* give me all the title tags which do not exist either in the biblFull or biblStruct id lists. I am not getting the results that I am expecting. let $biblFull := distinct-values(collection('edil_target/Prologue Merged 2013.xml')//biblFull/@id) let $biblStruct := distinct-values(collection('edil_target/Prologue Merged 2013.xml')//biblStruct/@id) for $title in collection('edil_target/eDIL-A.xml')//entry//title where $title/@target != $biblStruct or $title/@target != $biblFull return $title I am basically getting all the title tags back which is not what I am expecting at all. Can anyone shed any light on this? Thank you very much in advance! All the best, Chris

6 13

fn:serialize#2 serialization different in BaseX and Saxon
by Andreas Mixich 25 Mar '19

25 Mar '19

Hello, This code: let $asString := '<html><head><meta name="foo" content="bar"/></head></html>' let $asElement := <html><head><meta name="foo" content="bar"/></head></html> return ( "method: html " ,"AS STRING:" ,serialize($asString,map{"method":"html"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"html"}) ,"method: xhtml " ,"AS STRING:" ,serialize($asString,map{"method":"xhtml"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"xhtml"}) ,"method: text: " ,"AS STRING:" ,serialize($asString,map{"method":"text"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"text"}) ) when executed with Saxon, in oXygen, gives me: <?xml version="1.0" encoding="UTF-8"?>method: html AS STRING: &lt;html&gt;&lt;head&gt;&lt;meta name="foo" content="bar"/&gt;&lt;/head&gt;&lt;/html&gt; AS ELEMENT: <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta name="foo" content="bar"> </head> </html> method: xhtml AS STRING: &lt;html&gt;&lt;head&gt;&lt;meta name="foo" content="bar"/&gt;&lt;/head&gt;&lt;/html&gt; AS ELEMENT: <html> <head> <meta name="foo" content="bar"></meta> </head> </html> method: text: AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: With BaseX, however, I get: method: html AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: <html> <head> <meta name="foo" content="bar"> </head> </html> method: xhtml AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: <html> <head> <meta name="foo" content="bar"></meta> </head> </html> method: text: AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: which has me baffled. * why is the result to the very last function empty? * how can I convert an XML node to an entity escaped string, _without_ passing it in as string, but as element() or node()? The background is, that I want to write a function, that takes an XML element()* and results in the node(s) escaped to xs:string*, so, basically, what Saxon does (though I still wonder, how the Saxon people then get an unescaped string of XML, as BaseX does). Thanks. -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 1

Way to escape XML?
by Andreas Mixich 24 Mar '19

24 Mar '19

Hi, I must be missing the obvious, but is there a way to serialize element() or node() into xs:string with the XML getting entity escaped in BaseX? Something like: let $xml as element() := <xml>Hello World</xml> return serialize($xml, map{"method":"entity-escaped-string"} would result in <xml>Hello World</xml> -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 2

IllegalStateException from XsltTransformer
by Johannes Bauer 22 Mar '19

22 Mar '19

Hello, we have a BaseX web application with RESTXQ services. I was doing some load tests to see how it behaves in high load scenarios. Some tests did throw IllegalStateExceptions somewhere in the Saxon XSLT code (see stacktrace below). We use a XSLT stylesheet to transform the XML response of our services to an intermediate format that allows the transformation to JSON. (:~ : Transform XML to Json XML Schema (https://www.w3.org/TR/xslt-30/#schema-for-json) : @param $xml XML node : @return transformed XML :) declare function json:to-json-xml($xmlas element()) { xslt:transform($xml,'xml-to-json.xslt',map {},map {'cache' : true()}) }; Is there some concurrency problem with the xslt module? Best regards Johannes Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.1.2 Java: Oracle Corporation, 1.8.0_191 OS: Linux, amd64 Stack Trace: java.lang.IllegalStateException: The Controller is being used recursively or concurrently. This is not permitted. at net.sf.saxon.Controller.checkReadiness(Controller.java:1287) at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:685) at net.sf.saxon.s9api.AbstractXsltTransformer.applyTemplatesToSource(AbstractXsltTransformer.java:347) at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:338) at net.sf.saxon.jaxp.TransformerImpl.transform(TransformerImpl.java:71) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:109) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:59) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:37) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) at org.basex.query.expr.List.value(List.java:155) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:73) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:73) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:181) at org.basex.query.func.FuncCall.invoke(FuncCall.java:89) at org.basex.query.func.FuncCall.value(FuncCall.java:62) at org.basex.query.func.FuncCall.iter(FuncCall.java:67) at org.basex.query.scope.MainModule.iter(MainModule.java:118) at org.basex.query.QueryContext.iter(QueryContext.java:335) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:56) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:54) at org.basex.http.BaseXServlet.service(BaseXServlet.java:59) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.content42.services.filter.LoggingFilter.doFilter(LoggingFilter.java:44) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

1 0

Known issue? DB file names with capital letters get corrupted on OS switch.
by France Baril 17 Mar '19

17 Mar '19

Hi, I recently moved from Mac to Linux and noticed that when using the webDAV connection, all files that have capital letters in their name become corrupted when open in Linux. Luckily, out of the thousands of files that we have, less than 2 dozens used capitalization. I still think this could become a large issue for someone someday, so I felt I should report it. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 2