January 2019 - BaseX-Talk - mailman.uni-konstanz.de

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

BaseX/GUI v9.1.2 memory use
by Rick Graham 18 Apr '19

18 Apr '19

Hello, Thanks again, as always, for a great product. I just installed BaseX v9.1.2 (upgrading from a previous v9.1.2 snapshot), launched the GUI and then got interrupted. When I returned, almost all of the JVM's memory was being used. I hit "GC" several times but it didn't seem to help. I had no database loaded/open. Seems like some memory isn't getting freed properly. Here's my "INFO" General Information: > Version: 9.1.2 > Used Memory: 1593 MB > Global options: > AUTHMETHOD: Basic > CACHETIMEOUT: 3600 > DBPATH: /usr/local/src/basex/data > DEBUG: false > FAIRLOCK: false > HOST: localhost > HTTPLOCAL: false > IGNORECERT: false > IGNOREHOSTNAME: false > KEEPALIVE: 600 > LANG: English > LANGKEYS: false > LOG: true > LOGMSGMAXLEN: 1000 > LOGPATH: .logs > NONPROXYHOSTS: > PARALLEL: 8 > PARSERESTXQ: 3 > PASSWORD: > PORT: 1984 > PROXYHOST: > PROXYPORT: 0 > REPOPATH: /usr/local/src/basex/repo > RESTPATH: > RESTXQPATH: > SERVERHOST: > SERVERPORT: 1984 > STOPPORT: 8985 > TIMEOUT: 30 > USER: > WEBPATH: /usr/local/src/basex/webapp > Local options > ADDARCHIVES: true > ADDCACHE: false > ADDRAW: false > ARCHIVENAME: false > ATTRINCLUDE: > ATTRINDEX: true > AUTOFLUSH: true > AUTOOPTIMIZE: false > BINDINGS: > CASESENS: false > CATFILE: > CHECKSTRINGS: true > CHOP: true > COMPPLAN: true > COPYNODE: true > CREATEFILTER: *.xml > CREATEONLY: false > CSVPARSER: > DEFAULTDB: false > DIACRITICS: false > DOTCOMPACT: false > DOTPLAN: false > DTD: false > ENFORCEINDEX: false > EXPORTER: > FORCECREATE: false > FTINCLUDE: > FTINDEX: false > HTMLPARSER: > INLINELIMIT: 100 > INTPARSE: false > JSONPARSER: > LANGUAGE: en > LSERROR: 0 > MAINMEM: false > MAXCATS: 100 > MAXLEN: 96 > MAXSTAT: 30 > MIXUPDATES: false > PARSER: xml > QUERYINFO: true > RUNQUERY: true > RUNS: 1 > SERIALIZE: true > SERIALIZER: > SKIPCORRUPT: false > SPLITSIZE: 0 > STEMMING: false > STOPWORDS: > STRIPNS: false > TAILCALLS: 256 > TEXTINCLUDE: > TEXTINDEX: true > TEXTPARSER: > TOKENINCLUDE: > TOKENINDEX: false > UPDINDEX: false > WRITEBACK: false > XINCLUDE: true > XMLPLAN: true Best regards, Richard

6 15

Serialization issue with HTTP response
by Marco Lettere 08 Mar '19

08 Mar '19

Hi all, we have to deal with a third-party REST service which in case of error conditions returns this mime type: /*application/problem+json; charset=utf-8*/. I wrote this RestXQ [1] to mock it. Just copy it into restxq.xqm... When I call it like [2] I get /*[XPTY0004] Cannot convert xs:base64Binary to xs:string: "IntcImFcIjpcImJcIn0i"*/. This is because the response is seen as a binary and this can be double checked calling with [3]. I felt (maybe wrongly) that specializations of the mimetype had been faced in BaseX implementation of http:request somewhere back in time but I couldn't find anything in the mail archives. The strange thing however is that I get the same error when calling like [4] with override-mime-type set to text/json or application/json which instead I was expecting to work. Is the behaviour in this scenario expected or does this hide some inconsistency? Thanks, as usual, for the support. Regards, Marco. [1] declare%rest:path("mc/dummy") %rest:GET %output:method("json") function page:dummy() { <rest:response> <http:response status="400"> <http:header name="Content-Type" value="application/problem+json; charset=utf-8"/> </http:response> </rest:response>, '{"a":"b"}' }; [2] json:parse( http:send-request(<http:request method="get"/>,"http://localhost:8984/mc/dummy")[2] ) [3] json:parse( convert:binary-to-string( http:send-request(<http:request method="get"/>,"http://localhost:8984/mc/dummy")[2] ) ) [4] json:parse( http:send-request(<http:request method="get" override-media-type="application/json"/>,"http://localhost:8984/mc/dummy")[2] )

2 5

Schema 1.1 validation and Xerces cta-full-xpath-checking?
by Zimmel, Daniel 15 Feb '19

15 Feb '19

Hi, is there any chance BaseX will support the cta-full-xpath-checking option (see https://xerces.apache.org/xerces2-j/features.html)? Without it, schema 1.1 validation is limited. I could not find any switches to configure Xerces features, but did find someone mentioning it from two years ago: https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg08446.ht… Or have I missed something in the documentation? Regards, Daniel

3 5

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

WWW-Authenticate header
by Johannes Bauer 05 Feb '19

05 Feb '19

Hello BaseX Team, I'm trying to implement a RESTXQ service that uses JWT tokens for authorization and authentication. For this purpose I use a permission check annotated function that validates the token and returns a 401 response if the token is invalid or missing. declare %perm:check('/admin','{$perm}')function security:check-admin($perm) { let $token := $perm?authorization where empty($token) return <rest:response> <http:response status='401'> <http:header name='WWW-Authenticate' value='Bearer realm="c42"'/> </http:response> </rest:response> }; When I call the endpoint without token I get the expected response but the WWW-Authenticate header is different to the one that I have defined: WWW-Authenticate Basic realm="BaseX" Is there a way to override the BaseX provided header? Thanks for your input. Best regards Johannes

2 2

%perm:allow annotation
by Johannes Bauer 04 Feb '19

04 Feb '19

Hi again, I've another question about the %perm:allow annotation. Is it possible to use multiple instances of this annotation? My expectation would be that any of the listed roles is allowed to call the function. For example: declare %rest:GET %rest:path("/multi") %perm:allow("role1") %perm:allow("role2") function test:multi()as item()* { <response>multi</response> }; declare %rest:GET %rest:path("/single") %perm:allow("role1") function test:single()as item()* { <response>single</response> }; declare %perm:check('/admin','{$perm}')function test:check($perm) { () }; When I call /multi the first time I get a response. But all further calls to /multi will result in a NullPointerException. Calls to /single do not have this problem. My error stacktrace is: java.lang.NullPointerException at org.basex.util.list.ObjectList.finish(ObjectList.java:235) at org.basex.query.value.seq.StrSeq.get(StrSeq.java:64) at org.basex.http.restxq.RestXqPerm.map(RestXqPerm.java:43) at org.basex.http.restxq.RestXqFunction.bind(RestXqFunction.java:263) at org.basex.http.restxq.RestXqResponse.bind(RestXqResponse.java:61) at org.basex.http.web.WebResponse.create(WebResponse.java:53) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:50) at org.basex.http.BaseXServlet.service(BaseXServlet.java:59) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) I think the error is at the location where he parses the perm:allow annotations. Best regards Johannes

2 1

BaseX-specific serialization parameters
by Joe Wicentowski 31 Jan '19

31 Jan '19

Hi all, As I read the serialization spec ( https://www.w3.org/TR/xslt-xquery-serialization-31/#serparam) additional values for the "method" parameter and additional serialization parameters should use a non-null namespace URI (see quotes below). I believe BaseX's "csv" method or "tabulator" parameter ( http://docs.basex.org/wiki/Serialization) would fall into this category. Without such a non-null namespace URI, users may mistake BaseX-specific serialization features for those supported in the spec. The solution that the spec seems to suggest would be to use "basex:csv" or "basex:tabulator". I bet such a change would be unwelcome to those with code that uses the existing supported values, but aside from a disruptive cycle of deprecating existing values, etc., an alternative would be for the BaseX documentation to note this non-compliance with the spec. I realize this is nit-picky, but I thought it might be worth raising given the potential for confusion mentioned above, and should this discussion be of interest to anyone. Best, Joe Regarding values for the "method" parameter: > The value of the method parameter is an expanded QName. If the value has a null namespace URI, then the local name identifies a method specified in this document and MUST be one of xml, html, xhtml, text, json , or adaptive; in this case, the output method specified MUST be used for serializing. If the namespace URI is non-null, then it identifies an implementation-defined output method; the behavior in this case is not specified by this document. Regarding additional serialization parameters: > Implementations MAY define additional serialization parameters, and MAY allow users to do so. For this purpose, the name of a serialization parameter is considered to be a QName; the parameters listed above are QNames whose expanded-QName has a null namespace URI, while any additional serialization parameters that are either implementation-defined or defined by the host language MUST have names that are namespace-qualified.

2 2

ubuntu 18.04 (bionic)
by Ветошкин Владимир 31 Jan '19

31 Jan '19

2 2

fn:serialize() behaviour
by George Sofianos 30 Jan '19

30 Jan '19

Hi, This is probably a non issue, but I thought I should report it anyway. I was playing around with serialization options today and I noticed that: let $head := <head></head> let $body := <body></body> return serialize(($head, $body), map { "method": "html", "version": "5.0"}) will return <!DOCTYPE html> <head></head> <!DOCTYPE html> <body></body> I don't think fn:serialize() is defined in the xquery spec so it's implementation specific so I guess it also could be correct :) Also I have a question, I remember in the past a discussion about need for extra testing (XQuery spec wise) in BaseX? Is this still an issue? Hopefully I can find some time and help outwith that. Regards, George

2 1