March 2019 - BaseX-Talk - mailman.uni-konstanz.de

fn:serialize#2 serialization different in BaseX and Saxon
by Andreas Mixich 25 Mar '19

25 Mar '19

Hello, This code: let $asString := '<html><head><meta name="foo" content="bar"/></head></html>' let $asElement := <html><head><meta name="foo" content="bar"/></head></html> return ( "method: html " ,"AS STRING:" ,serialize($asString,map{"method":"html"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"html"}) ,"method: xhtml " ,"AS STRING:" ,serialize($asString,map{"method":"xhtml"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"xhtml"}) ,"method: text: " ,"AS STRING:" ,serialize($asString,map{"method":"text"}) ,"AS ELEMENT:" ,serialize($asElement,map{"method":"text"}) ) when executed with Saxon, in oXygen, gives me: <?xml version="1.0" encoding="UTF-8"?>method: html AS STRING: &lt;html&gt;&lt;head&gt;&lt;meta name="foo" content="bar"/&gt;&lt;/head&gt;&lt;/html&gt; AS ELEMENT: <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta name="foo" content="bar"> </head> </html> method: xhtml AS STRING: &lt;html&gt;&lt;head&gt;&lt;meta name="foo" content="bar"/&gt;&lt;/head&gt;&lt;/html&gt; AS ELEMENT: <html> <head> <meta name="foo" content="bar"></meta> </head> </html> method: text: AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: With BaseX, however, I get: method: html AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: <html> <head> <meta name="foo" content="bar"> </head> </html> method: xhtml AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: <html> <head> <meta name="foo" content="bar"></meta> </head> </html> method: text: AS STRING: <html><head><meta name="foo" content="bar"/></head></html> AS ELEMENT: which has me baffled. * why is the result to the very last function empty? * how can I convert an XML node to an entity escaped string, _without_ passing it in as string, but as element() or node()? The background is, that I want to write a function, that takes an XML element()* and results in the node(s) escaped to xs:string*, so, basically, what Saxon does (though I still wonder, how the Saxon people then get an unescaped string of XML, as BaseX does). Thanks. -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 1

Way to escape XML?
by Andreas Mixich 24 Mar '19

24 Mar '19

Hi, I must be missing the obvious, but is there a way to serialize element() or node() into xs:string with the XML getting entity escaped in BaseX? Something like: let $xml as element() := <xml>Hello World</xml> return serialize($xml, map{"method":"entity-escaped-string"} would result in <xml>Hello World</xml> -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 2

IllegalStateException from XsltTransformer
by Johannes Bauer 22 Mar '19

22 Mar '19

Hello, we have a BaseX web application with RESTXQ services. I was doing some load tests to see how it behaves in high load scenarios. Some tests did throw IllegalStateExceptions somewhere in the Saxon XSLT code (see stacktrace below). We use a XSLT stylesheet to transform the XML response of our services to an intermediate format that allows the transformation to JSON. (:~ : Transform XML to Json XML Schema (https://www.w3.org/TR/xslt-30/#schema-for-json) : @param $xml XML node : @return transformed XML :) declare function json:to-json-xml($xmlas element()) { xslt:transform($xml,'xml-to-json.xslt',map {},map {'cache' : true()}) }; Is there some concurrency problem with the xslt module? Best regards Johannes Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.1.2 Java: Oracle Corporation, 1.8.0_191 OS: Linux, amd64 Stack Trace: java.lang.IllegalStateException: The Controller is being used recursively or concurrently. This is not permitted. at net.sf.saxon.Controller.checkReadiness(Controller.java:1287) at net.sf.saxon.trans.XsltController.applyTemplates(XsltController.java:685) at net.sf.saxon.s9api.AbstractXsltTransformer.applyTemplatesToSource(AbstractXsltTransformer.java:347) at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:338) at net.sf.saxon.jaxp.TransformerImpl.transform(TransformerImpl.java:71) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:109) at org.basex.query.func.xslt.XsltTransform.transform(XsltTransform.java:59) at org.basex.query.func.xslt.XsltTransform.item(XsltTransform.java:37) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) at org.basex.query.expr.List.value(List.java:155) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:73) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:73) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.expr.If.value(If.java:131) at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:181) at org.basex.query.func.FuncCall.invoke(FuncCall.java:89) at org.basex.query.func.FuncCall.value(FuncCall.java:62) at org.basex.query.func.FuncCall.iter(FuncCall.java:67) at org.basex.query.scope.MainModule.iter(MainModule.java:118) at org.basex.query.QueryContext.iter(QueryContext.java:335) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:56) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:54) at org.basex.http.BaseXServlet.service(BaseXServlet.java:59) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.content42.services.filter.LoggingFilter.doFilter(LoggingFilter.java:44) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

1 0

Known issue? DB file names with capital letters get corrupted on OS switch.
by France Baril 17 Mar '19

17 Mar '19

Hi, I recently moved from Mac to Linux and noticed that when using the webDAV connection, all files that have capital letters in their name become corrupted when open in Linux. Luckily, out of the thousands of files that we have, less than 2 dozens used capitalization. I still think this could become a large issue for someone someday, so I felt I should report it. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 2

supporting XML Catalog files in xslt:transform() (patch)
by Liam R. E. Quin 14 Mar '19

14 Mar '19

The enclosed patch -- would you prefer a github pull request? -- makes xslt:transform() aware of XML catalog files, as other XML parsing already is. The same CATFILE preference is used (via the query context). I refactored CatalogWrapper slightly so it could be reused. I also took away the line that sets verbosity to 0, as without it you can control verbosity via a system property, or using the CatalogManager.properties file. I tested this with xml-commons-resolver-1.2/resolver.jar and with the built-in resolver and both seem to work. I have not added tests. In addition, it'd be worth adding something to the documentation, especially about the xml.catalog.verbosity property (just verbosity in the .properties file). Possible breaking change: i also removed the line that sets prefer=public. I spent ages trying to get catalogs working before i dicovered this, as i was using a system identifier! The code could check to see if the corresponding system property is set (users can't override the API with system properties, frustratingly), but since catalogs already say prefer=public or prefer=system in them, and it'd have needed to have been the same to work, i don't think this change breaks anythign in practice. It may make some catalogs start to work that had not been working, so maybe it's worth a line in the release notes. Liam -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Web slave for vintage clipart http://www.fromoldbooks.org/

4 13

Regex j-flag
by Andreas Hartmann 13 Mar '19

13 Mar '19

Hi The "j" flag for regular expressions introduced in version 9.1 doesn't seem to work in 9.1.2 anymore. For example, matches("Fenster", "\bfenst", "ij") produces [FORX0001] invalid regular flag 'j'. The same expression works in 9.1.1. Best Greetings, Andreas

2 1

Write functions in sequence
by Giuseppe G. A. Celano 12 Mar '19

12 Mar '19

Hi I wrote a single script which should do: write a file -> open this file -> write another different file. I put the write expressions in the right sequence, but it seems that the second one cannot happen because the file created by the first write function has not yet been created at the time the second function is invoked. Does anyone have a suggestion about this? Thanks. Best, Giuseppe

3 7

Serialization issue with HTTP response
by Marco Lettere 08 Mar '19

08 Mar '19

Hi all, we have to deal with a third-party REST service which in case of error conditions returns this mime type: /*application/problem+json; charset=utf-8*/. I wrote this RestXQ [1] to mock it. Just copy it into restxq.xqm... When I call it like [2] I get /*[XPTY0004] Cannot convert xs:base64Binary to xs:string: "IntcImFcIjpcImJcIn0i"*/. This is because the response is seen as a binary and this can be double checked calling with [3]. I felt (maybe wrongly) that specializations of the mimetype had been faced in BaseX implementation of http:request somewhere back in time but I couldn't find anything in the mail archives. The strange thing however is that I get the same error when calling like [4] with override-mime-type set to text/json or application/json which instead I was expecting to work. Is the behaviour in this scenario expected or does this hide some inconsistency? Thanks, as usual, for the support. Regards, Marco. [1] declare%rest:path("mc/dummy") %rest:GET %output:method("json") function page:dummy() { <rest:response> <http:response status="400"> <http:header name="Content-Type" value="application/problem+json; charset=utf-8"/> </http:response> </rest:response>, '{"a":"b"}' }; [2] json:parse( http:send-request(<http:request method="get"/>,"http://localhost:8984/mc/dummy")[2] ) [3] json:parse( convert:binary-to-string( http:send-request(<http:request method="get"/>,"http://localhost:8984/mc/dummy")[2] ) ) [4] json:parse( http:send-request(<http:request method="get" override-media-type="application/json"/>,"http://localhost:8984/mc/dummy")[2] )

2 5

Using HTTP Digest Authentication in a RESTXQ services
by Omar Siam 04 Mar '19

04 Mar '19

Hi list! I just experimented a bit to see if there is an obvious way to use HTTP digest authentication for some RESTXQ service I plan to write. I would prefer not to reimplement the HTTP digest logic in XQuery. I thought maybe there is a way to use the built in digest logic like for the REST endpoint. I don't seem to get that to work. Any suggestions? Best regards Omar Siam

2 5

RESTXQ and regexp
by Marco Lettere 04 Mar '19

04 Mar '19

Hello all, according to the docs this should be possible: (: Matches all paths with "app" as first, a number as second, and "order" as third segment :) declare %rest:path("app/{$code=[0-9]+}/order") function page:order($full-path) { ... }; (: Matches all other all paths starting with "app/" :) declare %rest:path("app/{$path=.+}") function page:others($path) { ... }; And it actually is... We are currently in the very unfortunate situation where we need the following: (: Matches anything followed by /input :) declare %rest:path("app/{$path=.+}/input") function page:inputs($path) { ... }; (: Matches all other all paths starting with "app/" :) declare %rest:path("app/{$path=.+}") function page:others($path) { ... }; Unfortunately this makes all the calls to be redirected to the page:others function ... is there a motivation behind it or could it be possible to work around that? Thanks, Marco.

3 3