BaseX-Talk October 2018

basex-talk@mailman.uni-konstanz.de

25 participants
33 discussions

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

many distinct namespaces
by Сергей Чесноков 24 Jan '19

24 Jan '19

Hi all, Is it possible to bypass the following restriction (I cannot change "ep_ins_med_q.xsd" (Central Bank xbrl scheme))?: BaseX 9.1 Command: CREATE DB bfo D:/portal/xbrl/CBR/final_1_3_1/Taxonomy_1_3_1/www.cbr.ru/xbrl/bfo/ Error: "D:/portal/xbrl/CBR/final_1_3_1/Taxonomy_1_3_1/www.cbr.ru/xbrl/bfo/rep/2018-03-31/ep/ep_ins_med_q.xsd""ep_ins_med_q.xsd" (Line 21): Too many distinct namespaces (limit: 256). Best regards, Sergei.

7 16

Programmatically extracting function signatures
by Bridger Dyson-Smith 07 Nov '18

07 Nov '18

Hi all - I was curious if there was a way to extract function signatures from BaseX. I'm currently reading through the (excellent) wiki but was curious if there was a more automated process I might be able to use. I've been trying to think of something clever with function-lookup, but I can't seem to make it dynamic based on module namespace; e.g. ``` for $fun in function-lookup(xs:QName("archive:delete"), 1) return $fun ``` I may just grab the wiki and parse it :), but I'm also interested in changes between versions of the processor. Any thoughts or suggestions would be greatly appreciated! Best, Bridger

4 7

BaseX 9.0: The Spring Edition!
by Christian Grün 01 Nov '18

01 Nov '18

Dear BaseX aficionados, We are very happy to announce the release of BaseX 9.0! The new version of our XML database system and XQuery 3.1 processor includes some great new features and a vast number of minor improvements and optimizations. It’s both the usage of BaseX in productive environments as well as the valuable feedback of our open source users that make BaseX better and better, and that allow and motivate us to keep going. Thanks to all of you! Along with the new release, we invite you to visit our relaunched homepage: http://basex.org/. Java 8 is now required to run BaseX. The most prominent features of Version 9.0 are: XQUERY - Comprehensive extensions in the internal XQuery optimizer framework - Dynamic hash joins rewritings of general comparisons - Register query jobs as persistent services - Process very large CSV files via the new CSV 'xquery' format - Update Module: higher-order functions for performing updates - Unified static typing, including maps, arrays and function items - COPYNODE: Lightweight copying of XML fragments - ENFORCEINDEX: Enforce rewriting for index access STORAGE - Improved database compression (short strings, whitespaces) WEB APPLICATIONS - Permissions layer: Unified definition of RESTXQ access patterns - RESTXQ: Full support for client- and server-side quality factors - REST: Run BaseX command scripts - Web server upgrade to Jetty 9 REPOSITORIES - Combined packaging mechanism (XQuery and Java) - Java Packaging: Bundling of additional libraries DISTRIBUTIONS - Revised detection and configuration of BaseX home directory - Windows executable: better detection of Java libraries DBA: BASEX DATABASE ADMINISTRATION - Improved editing and evaluation of XQuery modules - Revised file, session and job management More details can be found on our homepage. We are looking forward to your feedback. Enjoy and have fun, Christian, BaseX Team

7 8

ArrayIndexOutOfBoundsException: Maximum array size reached
by Han (US), Paul P 30 Oct '18

30 Oct '18

Hello, We encountered the below error when using this API: archive:extract-to($tempDirPath, file:read-binary($zipFilePath)). The zip file size is 51396593 and the inflated file size is 2796146027. Please let me know if you need additional information. We expect our data files to be at least 10 times bigger than these numbers. POST failed with response 400: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Version: BaseX 9.0.2 Java: Oracle Corporation, 1.8.0_121 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Maximum array size reached. at org.basex.util.Array.newSize(Array.java:238) at org.basex.util.Array.newSize(Array.java:227) at org.basex.io.out.ArrayOutput.write(ArrayOutput.java:32) at java.io.OutputStream.write(OutputStream.java:116) at org.basex.query.func.archive.ArchiveIn.read(ArchiveIn.java:77) at org.basex.query.func.archive.ArchiveExtractTo.item(ArchiveExtractTo.java:40) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:48) at org.basex.query.scope.MainModule.iter(MainModule.java:118) at org.basex.query.QueryContext.iter(QueryContext.java:330) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.core.cmd.AQuery.query(AQuery.java:100) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105) at org.basex.http.rest.RESTQuery.query(RESTQuery.java:69) at org.basex.http.rest.RESTQuery.run0(RESTQuery.java:37) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.core.Command.execute(Command.java:116) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:27) at org.basex.http.BaseXServlet.service(BaseXServlet.java:59) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:419) at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:169) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:474) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Regards, Paul

2 2

slow DB when updating
by radim-havlicek＠post.cz 30 Oct '18

30 Oct '18

Hello, when one client is uploading file to a database (let's say Database1), upd. basex file is created and exists until the file is uploaded to the database Database1. Problem is that meanwhile the upload is running, all other clients reading data from the Database1 read the data very, very slowly. Also clients trying to read data from Database2 or Database3 reads the data very, very slowly while the upload to Database1 is running. Also the client trying to upload data to Database2 is not successful until the upload to Database1 is done. Database1, Database2 and Database3 are databases in the same BaseX database. Can I speed up the loading data from various databases at the same time or the parallel uploading? I've read about the PARALLEL option in .basex option file, but are there any best practices what the value should be regarding to the database size or the number of databases? And are there similar best practices to CACHETIMEOUT, TIMEOUT and KEEPALIVE options? Are there more information than what is at http://docs.basex.org/wiki/Options ? Thanks for your reply in advance. Best regards, Radim Havlicek

2 1

How to pass an XPath expression as parameter?
by Andreas Mixich 29 Oct '18

29 Oct '18

Is there a way to do something like this? declare function local:insert( $doc as node(), $at-location as xpath(), (: I know, there is no 'xpath' type, this is just for demo :) $what as node()* ) as node() { copy $c := $doc modify insert node $what into $at-location return $c }; let $xml := <xml><body><p>This is a paragraph.</p></body></xml> let $nd := <this><is><a>Test</a></is></this> return local:insert($xml, xpath("/body"), $nd) As I see it, it would need a dynamic xs:string => XPath expression conversion or a defer XPath expression, which both do not exist, for $at-location. I managed to do this, however: declare function local:insert($doc, $at-location, $what) { let $update := ``[ declare variable $doc external; declare variable $what external; declare variable $at-location external; copy $c := $doc modify insert node $what into `{$at-location}` return $c ]`` return xquery:eval($update, map{ 'doc':$doc, 'at-location':$at-location, '$what':$what }) }; let $xml := <xml><body><p>This is a paragraph.</p></body></xml> let $node := <this><is><a>Test</a></is></this> return local:insert($xml, "$c/body", $node) Ideally (for me), there would be a 'defer' XPath expression, that starts its life as a string (when passed) and then unfolds upon call. Or a type (or function) fn:xpath(), that behaves that way. Or am I just blind and don't see the obvious, and there *is* a simpler solution? -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 2

BaseX-standard-installation-with-embedded-jetty-- how to deploy other webapp
by Dieter Zanzinger 27 Oct '18

27 Oct '18

Dear BaseX-fellows, the websocket-feature in 9.1 is great. Now- to use ist, I need the BaseX-Version with embedded Jetty. My problem is, that I need another web app, too. Up to now, I used Tomcat as container for BaseX and the other webapp. Is it possible, to deploy another app in the embedded Jetty ? Any hints, how to do that? In Tomcat and - as far as I know in a standard Jetty installation - I just copy the .war into the webapps-Folder. That doesn´t work in the embedded Jetty. I would be glad for hints. Sincerely Dieter Zanzinger

2 1

fn:format-number() for german style numbers
by Wiemer, Sebastian 27 Oct '18

27 Oct '18

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk October 2018