February 2023 - BaseX-Talk - mailman.uni-konstanz.de

BaseX 9.6: The Summer Edition
by Christian Grün 28 Nov '24

28 Nov '24

Dear all, We provide you with a new and fresh version of BaseX, our open source XML framework, database system and XQuery 3.1 processor: https://basex.org/ Apart from our main focus (query rewritings and optimizations), we have added the following enhancements: XQUERY: MODULES, FEATURES - Archive Module, archive:write: stream large archives to file - SQL Module: support for more SQL types - Full-Text Module, ft:thesaurus: perform Thesaurus queries - Fulltext, fuzzy search: specify Levenshtein limit - UNROLLLIMIT option: control limit for unrolling loops XQUERY: JAVA BINDINGS - Java objects of unknown type are wrapped into function items - results of constructor calls are returned as function items - the standard package "java.lang." has become optional - array arguments can be specified with the middle dot notation - conversion can be controlled with the WRAPJAVA option - better support for XQuery arrays and maps WEB APPLICATIONS - RESTXQ: Server-Timing HTTP headers are attached to the response For a more comprehensive list of added and updated features, look into our documentation (docs.basex.org) and check out the GitHub issues (github.com/BaseXdb/basex/issues) Have fun, Your BaseX Team

11 43

recursively used variables
by Rob Stapper 12 Aug '24

12 Aug '24

Hi, The code[1] below and send as attachment generates a error message: “Static variable depends on itself: $Q{http://www.w3.org/2005/xquery-local-functions}test”. I use these variables to refer to my private functions in my modules so I can easyly refer to them in a inheritance situation. It’s not a big problem for me but I was wondering if the error-triggering is justified or that it should work. [1]=========================================== declare variable $local:test := local:test#1 ; declare %private function local:test( $i) { if ( $i > 0) then $local:test( $i - 1) } ; $local:test( 10) =========================================== Kind regards, Rob Stapper Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

4 7

xslt:transform-report result
by Andy Bunce 11 Mar '24

11 Mar '24

Hi, Using BaseX 9.7.1 and saxon9he-9.9.1.jar The documentation suggests the ?result from xslt:transform-report should be *a* document-node where possible [1] This seems not quite to be the case when there are processing instructions or comments at the top level. In these cases a sequence of document-nodes is returned. /Andy [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report let $xslt:=<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="3.0"> <xsl:mode on-no-match="shallow-copy"/> </xsl:stylesheet> let $xml:=document{ <?something type="compact"?>, <foo/>} return xslt:transform-report($xml,$xslt) Returns map { "messages": (), "result": (<?something type="compact"?>, <foo/>) }

3 7

PHP Client Session Object: infinite loop in read method when bpos is null
by Tamara Marnell 13 Jul '23

13 Jul '23

Hello everyone, Once in a while, my server running BaseX will suddenly experience a big spike in write operations, the hard disk space will fill up to 100%, and the whole server will lock up. I can't say what initiates this, because the logs and metrics disprove every theory I can think of. (There's nothing unusual in the access logs or BaseX data logs, memory usage stays consistently at 45% and CPU usage at 5%, no user attempted anything like a huge bulk upload, etc.) However, one thing that always happens during these incidents is that the Apache error log blows up to a crazy size like 1 GB, filled with the message: PHP Notice: Trying to access array offset on value of type null in /[path]/client/session.php on line 164 This line is in the read method of the Session class from GitHub, and it indicates that the byte position passed to the buffer property is null. My error log probably fills up because read() is called by readString() in a while loop that never terminates. There might be extraordinary circumstances in which during construction, socket_connect returns "true" but the response is not valid or expected. -Tamara -- Tamara Marnell Program Manager, Systems Orbis Cascade Alliance (orbiscascade.org <https://www.orbiscascade.org/>) Pronouns: she/her/hers

2 5

Formatting attributes as an indented list?
by Patrick Durusau 07 Mar '23

07 Mar '23

Greetings! I've been tasked with using BaseX to produce: ***** <wg class="cl" rule="S-IO" cltype="VerbElided"> <wg rule="NpaNp" role="s"> <wg type="group" appositioncontainer="true" rule="Np-Appos"> <w ref="PHM 1:1!1" after=" " class="noun" gbiType="proper" xml:id="n57001001001" lemma="Παῦλος" normalized="Παῦλος" strong="3972" number="singular" gender="masculine" case="nominative" gloss="Paul" domain="093001" ln="93.294a" morph="N-NSM" unicode="Παῦλος">Παῦλος</w> ***** The indenting is easy enough and I can even make it deeper if required but is there a command for serialization that will properly format the attributes? My personal suspicion is that inserting \n when each attribute is serialized (and not on the last one) is the easier route but I promised to investigate the command line. Have I overlooked something in the very fine manual? Hope everyone is having a great week! Patrick -- Patrick Durusau patrick(a)durusau.net Technical Advisory Board, OASIS (TAB) Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau

4 10

Unable to set admin password
by Marco Lettere 20 Feb '23

20 Feb '23

Dear all, I'm trying to setup a new instance of BaseX 10 in order to start migrating some applications of ours. Anyway I'm unable to set the admin password. Followin https://docs.basex.org/wiki/BaseX_10#Migrating_Applications, I'm currently trying with [1] and [2] which both ask me to enter a new password. But afterwards I keep getting [3].... and I'm unable to access dba too. I cannot understand what I'm doing wrongly ... Any help? Thanks! [1] java -cp BaseX.jar org.basex.BaseX -c PASSWORD Password: newpassword [2] bin/basex -c PASSWORD Password: newpassword [3] bin/basexclient Username: admin Password: newpassword Access denied: admin

1 2

Examples and instructions
by kjkatinas＠gmail.com 19 Feb '23

19 Feb '23

Labas, Christian, Dar neturiu Lietuviska font tai rasisiu Angliskai. What I am looking for is documentation on how to perform simple UNION's and/or JOIN's. I have two DB's with data. DB1 has a small sample data set. DB2 is a master data repository. I need to find DB1 data that does not exist in the master repository. Sound simple enough, but I need examples with the proper syntax. Thanks in advance for your time and help. Linkejimai, KAS Kestutis J Katinas

2 1

reg: Stack Overflow: Try tail recursion?
by chandra sekhar n 17 Feb '23

17 Feb '23

Dear team, While using the below query on collection getting this try tail recursion issue. db:exists("3860_Master_Voice_JULY-21_Unbilled","SSBS_9038343_01131130904_02JUL21.xml") I am attaching a query plan and database info. Regards, Chandrasekhar

2 4

Help Diagnosing Memory Leak in SAXWrapper or Associated Code
by Eliot Kimber 13 Feb '23

13 Feb '23

I’ve worked out how to add a Xerces grammar cache to the XML parser. For current code from GitHub, I did this in SAXParser: public void parse() throws IOException { final InputSource is = inputSource(); final SAXSource saxs = new SAXSource(is); XMLReader reader = null; try { reader = saxs.getXMLReader(); if(reader == null) { reader = XmlParser.reader(options.get(MainOptions.DTD), options.get(MainOptions.XINCLUDE)); } final EntityResolver er = Resolver.entities(options); if(er != null) reader.setEntityResolver(er); saxh = new SAXHandler(builder, options.get(MainOptions.STRIPWS), options.get(MainOptions.STRIPNS)); reader.setDTDHandler(saxh); reader.setContentHandler(saxh); reader.setProperty(http://xml.org/sax/properties/lexical-handler, saxh); reader.setErrorHandler(saxh); if (true && options.get(MainOptions.DTD)) { XMLGrammarPool pool = getGrammarPool(); try { reader.setProperty(FEATURE_GRAMMAR_POOL, pool); } catch (final NoClassDefFoundError e) { } catch (final SAXNotRecognizedException | SAXNotSupportedException e) { } } reader.parse(is); ... Where getGrammarPool() is simply: static private XMLGrammarPool getGrammarPool() { XMLGrammarPool pool = grammarPool; if (pool == null) { pool = new XMLGrammarPoolImpl(); grammarPool = pool; //.set(pool); } return pool; } When I use the grammar cache to parse DITA docs (setting parse DTDs to true and specifying an XML catalog from DITA Open Toolkit) I see the expected speedup. For example, on a small set of about 2400 maps and topics, no-DTD parsing takes 7 seconds, grammar-cache parsing takes 20 seconds, and no-grammar-cache DTD parsing takes about 2.5 minutes. So roughly an 8x improvement (which is what I’ve measured using the same grammar cache with Saxon, for example). However, using the grammar cache causes some kind of extreme memory leak and I have no idea what it is. Without the grammar cache, parsing these topics requires only a few 100 metabytes of memory DTD or no, but with the grammar cache, memory usage starts at about 1GB and goes up from there. Parsing my full set of 40K maps and topics, memory grows by 1GB every 30 seconds or so until it eventually exceeds even the 14GB I allocated in my last test. The 1GB could be explained by the cache itself, which holds the parsed grammars. Using the debugger, I can see that the grammar cache itself is static once it’s populated with grammars (for my set it ends up loading 10 parsed grammars), so the grammar cache itself doesn’t seem to be the problem. I’m trying to use VisualVM to profile the memory but this is not something I have done before and I’m not sure what classes I should be focusing on. So my questions: 1. Any idea why the simple addition of the grammar cache would cause this kind of memory leak? 2. Any guidance on what classes I should focus on to find the culprit? My reason for using the grammar cache is to have all the default attributes populated in the database without requiring two hours to load my content (which is what I’ve measured in the past for DTD-aware parsing of my 40K content set). Another solution, specific to DITA, would be to use a custom SAX parser that injects the default attributes based on static configuration (for a given set of DITA grammars we know what the defaults will be for every element type and can easily generate the configuration a SAX parser would use). But the current code doesn’t seem to provide an easy way to swap in a custom SAX handler and I’m not really in a position to try to add that level of sophistication to the code. Modulo this memory issue, the grammar cache is a nice simple solution to the DTD parsing requirement that is general to any content set that has consistent DTD or XSDs across the set of documents to be parsed. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 2

XQuery Prolog with serialization settings - basex "Unknown option 'basex'?
by Patrick Durusau 13 Feb '23

13 Feb '23

Greetings! I have the following in the prolog of an XQuery: declare namespace output = 'http://www.w3.org/2010/xslt-xquery-serialization'; declare option output:method 'basex'; declare option output:basex 'indent=yes, newline=\n'; I'm following the example at: https://docs.basex.org/wiki/Serialization for CSV, reasoning the same form should work for basex. Sadly, BaseX 10.4 on Debian reports: [XQST0109] Unknown option 'basex Is it because basex isn't in the namespace of 'http://www.w3.org/2010/xslt-xquery-serialization'; ? OK so I change the namespace to http://tm.durusau.net and it runs! Sadly not applying the requested formatting. Some guidance on serialization please? Thanks! Patrick -- Patrick Durusau patrick(a)durusau.net Technical Advisory Board, OASIS (TAB) Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau

2 2