- BaseX-Talk - mailman.uni-konstanz.de

copy/modify/return and the base URI
by Hans-Juergen Rennau 28 Aug '23

28 Aug '23

Dear BaseX team, I seem to remember that in earlier versions of BaseX a copy/modify/return did not change the base URI of the processed document. However, now it is lost, or more precisely, replaced with the static base URI of the processing XQuery module. Am I mistaken? At any rate it would be very desirable for copy/modify/return to preserve the base URI of the processed document or fragment. Kind regards,Hans-Jürgen

2 5

XQuery-Java-Binding: Nullpointer "nsMap" is null (on returning a w3c.dom.Node)
by Andreas Hengsbach | nexoma 28 Aug '23

28 Aug '23

Hi all, in a Java module I create a "org.w3c.dom.Node" (with java.xml.parsers.DocumentBuilder) and return it to XQuery/BaseX. Since 10.7 I get this exception: # schnipp Exception in thread "main" java.lang.NullPointerException: Cannot invoke "org.basex.util.hash.TokenMap.put(byte[], byte[])" because "nsMap" is null at org.basex.query.value.node.FElem.build(FElem.java:99) at org.basex.query.value.type.NodeType$3.cast(NodeType.java:60) at org.basex.query.value.type.NodeType$3.cast(NodeType.java:55) at org.basex.query.func.java.JavaCall.toValue(JavaCall.java:262) at org.basex.query.func.java.DynJavaFunc.eval(DynJavaFunc.java:90) at org.basex.query.func.java.JavaCall.value(JavaCall.java:81) at org.basex.query.expr.List.value(List.java:230) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:92) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:92) at org.basex.query.expr.Try.value(Try.java:93) at org.basex.query.expr.List.value(List.java:226) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:92) at org.basex.query.expr.List.value(List.java:230) at org.basex.query.expr.If.value(If.java:158) at org.basex.query.expr.List.value(List.java:230) at org.basex.query.expr.Try.value(Try.java:93) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:92) at org.basex.query.expr.If.value(If.java:158) at org.basex.query.expr.List.value(List.java:226) at org.basex.query.expr.Try.value(Try.java:93) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:48) at org.basex.query.QueryContext.lambda$iter$4(QueryContext.java:357) at org.basex.query.QueryContext.run(QueryContext.java:766) at org.basex.query.QueryContext.iter(QueryContext.java:357) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:95) # schnapp Do I have to create the Node differently now, or is this a bug? nsMap. a map of namespace I guess. In my case the w3c.dom.Node has no namespace. Thanks a lot Andreas

2 4

"Expressions must be all updating" puzzle
by Eliot Kimber 21 Aug '23

21 Aug '23

In BaseX 10: In an updating function I have a call to another updating function that uses update:output() to log messages. If that function call is included, BaseX says that expressions must be all updating, but if I remove the call, BaseX is happy. The calling function is: declare %updating %rest:path('/now/rest/analytics/store-latest-acrolinx-users' ) function now:storeAcrolinxUsers() { let $acrolinxMap as map(*)? := try { acrolinx:getUserDataMap() } catch * { util:logToConsole( 'storeAcrolinxUsers' ,``[`{$err:code}` - `{$err:description}`]`` ,'warn' ) } return if (exists($acrolinxMap)) then let $doc as element() := acrolinx:serializeAcrolinxUserMap($acrolinxMap) let $fileName as xs:string := 'acrolinx-users_' || replace($doc/@timestamp, ':', '~') || '.xml' return db:put(analyticsmgmt:getAnalyticsDatabase(), $doc, relpath:newFile('acrolinx/users', $fileName)) }; The offending function is util:logToConsole(), which is declared as: declare %updating function util:logToConsole( $functionName as xs:string, $message as xs:string, $logLevel as xs:string ) { let $logMessage as xs:string := '[' || upper-case($logLevel) || '] ' || $functionName || '(): ' || $message return ( update:output(prof:dump($logMessage)) , update:output($logMessage) ) }; This is declared as an updating function and returns “updating” functions. If I comment out the call to util:logToConsole() in the first function, then BaseX is happy. If I use the non-updating version of logToConsole(), util:logToLog(), then BaseX is also happy. So my question is: why is BaseX complaining about the use of this updating function in an updating function? There must be some subtlety that I’m missing but I have not yet figured out what it is. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 4

Problem with uploading a file larger than 512 MB .......
by Dmitry Gorskiy 21 Aug '23

21 Aug '23

Dear Sirs, What could be the problem? Big file upload problem. Below is the listing from the console. My computer: ------------------- CPU: 4 cores RAM: 8G OS: Linux BaseX: 107 and BaseX108-20230816.163639 Java: Adoptium idk-17.0.8+7 I'm creating files: dd if=/dev/random of=file_512M bs=1M count=512 …………………………file_530M …………………………file_1G …………………………file_2G and etc. Launching “basexhttp" ( default ...-Xmx2g … ) Let's go in … http://192.168.1.120:8080/dba/files <http://192.168.1.120:8080/dba/files> Downloading files - no problem. Uploading: file 512MB size - no problem. Uploading file 530MB and more - PROBLEM ( looking console output below ) OK, relaunching basexhttp whith -Xmx4G ….. Uploading file 512MB - ok Uploading file 1GB - ok More size - crashes What could be the problem? Thank you very much. Dmitriy Gorskiy. HTTP ERROR 500 java.lang.OutOfMemoryError: Java heap space URI: /dba/file-upload STATUS: 500 MESSAGE: java.lang.OutOfMemoryError: Java heap space SERVLET: RESTXQ CAUSED BY: java.lang.OutOfMemoryError: Java heap space Caused by: java.lang.OutOfMemoryError: Java heap space at java.base/java.util.Arrays.copyOf(Arrays.java:3537) at org.basex.util.list.ByteList.add(ByteList.java:43) at org.basex.io.in.BufferInput.content(BufferInput.java:216) at org.basex.http.RequestContext.body(RequestContext.java:119) at org.basex.http.RequestContext.addMultipart(RequestContext.java:137) at org.basex.http.RequestContext.formValues(RequestContext.java:103) at org.basex.http.restxq.RestXqFunction.bind(RestXqFunction.java:223) at org.basex.http.restxq.RestXqResponse.init(RestXqResponse.java:60) at org.basex.http.web.WebResponse.create(WebResponse.java:54) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel$$Lambda$140/0x00007f5974196000.dispatch(Unknown Source) Powered by Jetty:// 9.4.51.v20230217 <https://eclipse.org/jetty>

2 1

Use Bash to construct multi-file form submission
by Eliot Kimber 18 Aug '23

18 Aug '23

This is not strictly a BaseX question but it is in the context of communicating with a BaseX-implemented REST service and I’m hoping someone in this community can answer my question as I’ve not found a solution the interwebs. I’ve implemented a POST method that expects a multi-part form request with multiple files that then loads those files into a database. The files are all related such that I want something close to an atomic commit for the set of files. Thus the multi-file form approach. The REST method works and I can use from the command line using curl, i.e: curl --location http://localhost:9984/now/rest/validation/reports?debug=false \ --form 'files=@"/Users/eliot.kimber/git-basex/product-content-analytics/validation-reports/vancouver/bundle-strategic-portfolio-mgmt_errors_vancouver_2023-08-14T22~11~28Z_1ae692f8_1692054936.xml"' \ --form 'files=@"/Users/eliot.kimber/git-basex/product-content-analytics/validation-reports/vancouver/bundle-tech-technology-industry_errors_vancouver_2023-08-14T22~11~28Z_1ae692f8_1692054971.xml"' What I haven’t been able to figure out is how to construct the equivalent curl command in a bash script, where the script gets a list of flies to load and then constructs the –form parameters. Everything I’ve tried results in a “Warning: Trailing data after quoted form parameter” message or some more utter failure. It must be some subtlety of how to escape the quotes—my googling around produces lots of stack overflow discussions of how to manage the escapes, but nothing I’ve tried has so far worked. My bash approach is simply: curlcmd=”curl --location …” for file in ${files[@]}; do curlcmd+=” --form ‘files=@\”${file}\”’ done And then execute the resulting string: bash -c $curlcmd I have a separate script that successfully constructs a curl call to a companion end point that takes one file: curl \ "--location" "${serverUrl}:${serverPort}/now/rest/validation/report/${fileName}?debug=${debug}" \ "--header" "Content-Type: text/xml" \ "--data-binary" "@${filePath}" And I’m currently just calling this for each file, which is OK. But I’d really like to be able to send a set of files as a single REST call. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 2

Re: [basex-talk] mapping functions
by Imsieke, Gerrit, le-tex 17 Aug '23

17 Aug '23

Hi Graydon, Replying to the list for posterity. I hope you don’t mind. When you dynamically evaluate an expression (xquery:eval(), job:eval(), …), you start with the default values of the static context [1]. Nothing of the XQuery module from which you evaluate the expression is known (unless supplied by binding, but ordinary function declarations are not amenable to binding), therefore no risk of circularity. As stated in the table in [1], you can use module import [2] to augment this clean slate. In particular the module’s function declarations/signatures will be available then. If you don’t want the dynamically evaluated expression to import the whole module, you can store functions into variables and supply these variables to the evaluation through bindings. Haven’t tried it yet. Gerrit [1] https://www.w3.org/TR/xquery-31/#id-xq-static-context-components [2] https://www.w3.org/TR/xquery-31/#id-module-import On 16.08.2023 14:02, Graydon Saunders wrote: > Thank you for the specific example! > > I'm croggled that works without giving me a circular reference error of some kind, but it does work. > > Much appreciated! > > Graydon > > On Tue, Aug 15, 2023 at 11:34 PM <gerrit.imsieke(a)le-tex.de <mailto:gerrit.imsieke@le-tex.de>> wrote: > > You can import the library module in the evaluated expression. Example: https://github.com/transpect/control-backend/blob/main/webapp/control-backe… <https://github.com/transpect/control-backend/blob/main/webapp/control-backe…> > > Gerrit > > > Sent from MailDroid <https://goo.gl/ODgwBb> > > -----Original Message----- > From: Graydon Saunders <graydonish(a)gmail.com <mailto:graydonish@gmail.com>> > To: BaseX <BaseX-Talk(a)mailman.uni-konstanz.de <mailto:BaseX-Talk@mailman.uni-konstanz.de>> > Sent: Wed, 16 Aug 2023 4:42 > Subject: [basex-talk] mapping functions > > Hello -- > > I have various databases that will be run through the same code after their contents is regularized. So the only different function is the one used for regularization. > > This can be handled by writing a wrapper function and passing in a label to say which function to use, but I was wondering if there's a way to do it in the configuration map directly. The configuration map exists as a text file and gets run through xquery:eval() at the start of the process. > > declare namespace abc='http://some/uri <http://some/uri>'; > 'regularizationFunction': map { > 'typeOne': abc:regularizationOne#1, > 'typeTwo': abc:regularizationTwo#1 > } > > doesn't work, because the abc functions aren't known at evaluation time. These functions are defined in the library module that's evaluating the configuration file but xquery:eval() doesn't know that. I don't see a way to use bindings to get around this. > > Is there a way to map a locally-defined function reference so xquery:eval() will recognize it? > > thanks! > Graydon > > -- > Graydon Saunders | graydonish(a)fastmail.com <mailto:graydonish@fastmail.com> > Þæs oferéode, ðisses swá mæg. > -- Deor ("That passed, so may this.") > -- Gerrit Imsieke Geschäftsführer / Managing Director le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 110, Fax +49 341 355356 510 gerrit.imsieke(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer / Managing Directors: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt ---------------------------------------------- Besuchen Sie uns auf der Frankfurter Buchmesse in Halle 4.0, G94.

2 3

Accessing BaseX server on remote machine
by Furst, Thomas 17 Aug '23

17 Aug '23

Hello, I am attempting to use BaseX from a central location on our network so that all team members can access it. I am using the Python Client to achieve this. Everything works fine when the host is entered as 'localhost'. Once I change it to the remote host name, I am getting a time out error reported back: TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond On the remote machine (Windows 10 Pro for Workstations) I have opened port 1984 in the firewall on both the inbound and outbound. Can someone help me understand what I may be doing incorrectly? Or what I may have overlooked doing that is also required? Thank you, Tom Furst

2 4

mapping functions
by Graydon Saunders 16 Aug '23

16 Aug '23

Hello -- I have various databases that will be run through the same code after their contents is regularized. So the only different function is the one used for regularization. This can be handled by writing a wrapper function and passing in a label to say which function to use, but I was wondering if there's a way to do it in the configuration map directly. The configuration map exists as a text file and gets run through xquery:eval() at the start of the process. declare namespace abc='http://some/uri'; 'regularizationFunction': map { 'typeOne': abc:regularizationOne#1, 'typeTwo': abc:regularizationTwo#1 } doesn't work, because the abc functions aren't known at evaluation time. These functions are defined in the library module that's evaluating the configuration file but xquery:eval() doesn't know that. I don't see a way to use bindings to get around this. Is there a way to map a locally-defined function reference so xquery:eval() will recognize it? thanks! Graydon -- Graydon Saunders | graydonish(a)fastmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")

2 1

passing complex parameters to modules
by Graydon Saunders 13 Aug '23

13 Aug '23

Hello -- I've got a module full of functions. I've got two (and expect more) queries which import the function module. The queries have slightly different requirements; for example, in one case, it's necessary to use normalize-unicode() during a data regularization step, and in the other case it isn't. These differences are generally not at the top level of function that the query itself calls; they're usually a couple levels down in the function module. It doesn't look like the function module can have its own external variables; if it can, that'd be ideal. Otherwise, it seems like the thing to do is to have each query load a parameter map, but then I have to include that map as a parameter in all the functions calls in the function module, which feels inelegant. And I definitely don't want to start introducing a haze of individual values. Is there a better way to do this? It seems like I'm missing something. Thanks! Graydon -- Graydon Saunders | graydonish(a)fastmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")

3 2

How best to cache an intermediate result in the context of a larger query?
by Eliot Kimber 11 Aug '23

11 Aug '23

In the context of a RESTXQ handler that is processing a stored document to generate HTML From it, I need to do some expensive processing and then cache the result for the next time the same document is rendered. It doesn’t make sense to preprocess all the documents to cache the data as part of my overall ingestion process because only a fraction of documents will ever be rendered by this page. I think the answer is to turn on MIXUPDATES so I can invoke an updating function in the process of the larger RESTXQ handling query, but I’m wondering if there’s some less drastic or better-architected approach? For example, should be I be invoking a separate REST end point to do the calculation, blocking on the request, and then fetch the cached result? I read through the update module and the various pages on XQuery update and feel like I still don’t quite understand what my updating options are. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

3 5