BaseX-Talk April 2025

basex-talk@mailman.uni-konstanz.de

7 participants
9 discussions

Recovering database in bad state
by Eliot Kimber 01 May '25

01 May '25

I have a database that shows having the expected 2 documents in it in the DBA databases list but when I go to the database in the DBA it shows 0 items. I think this is because the database was being written to and something happened that left it in a bad state. Is there any possibility of recovering this database so it is usable? The practical problem is that it takes several hours to populate this database, so recovering it rather than attempting another multi-hour load would be nice. Here’s what I see for files in the database’s directory on the server: -rw-r--r-- 1 eliot.kimber.adm domain users 73039559 Apr 18 14:50 atv.basex -rw-r--r-- 1 eliot.kimber.adm domain users 7544382 Apr 18 14:52 atvl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 2730980 Apr 18 14:52 atvr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 236 Apr 18 04:17 ftxx.basex -rw-r--r-- 1 eliot.kimber.adm domain users 218072 Apr 18 04:17 ftxy.basex -rw-r--r-- 1 eliot.kimber.adm domain users 1603678 Apr 18 04:17 ftxz.basex -rw-r--r-- 1 eliot.kimber.adm domain users 15 Apr 18 04:17 idp.basex -rw-r--r-- 1 eliot.kimber.adm domain users 15193 Apr 18 04:17 inf.basex -rw-r--r-- 1 eliot.kimber.adm domain users 67 Apr 18 04:17 pth.basex -rw-r--r-- 1 eliot.kimber.adm domain users 28 Apr 18 04:17 swl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 83701760 Apr 18 14:52 tbl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 9 Apr 18 04:17 tbli.basex -rw-r--r-- 1 eliot.kimber.adm domain users 112380335 Apr 18 14:52 tokl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 2829520 Apr 18 14:52 tokr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 1718692 Apr 18 04:17 txt.basex -rw-r--r-- 1 eliot.kimber.adm domain users 351326 Apr 18 04:17 txtl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 285610 Apr 18 02:38 txtr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 0 Apr 18 04:18 upd.basex In the Databases list I see: _temp_lrk_yokohama_link_records<http://mirabel.corp.service-now.com:8984/dba/database?name=_temp_lrk_yokoha…> 2 274 MB 2025-04-18, 14:52:42 But then when I go to the database itself it just say “0 entries”. Thanks, Eliot _____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368 servicenow servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | X<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Instagram<https://www.instagram.com/servicenow>

3 11

inspect:functions
by Rob Stapper 01 May '25

01 May '25

Hi, I want to make use the 'inspect:functions'-function but whatever I do I always get this error: ' [inspect:parse] Stopped at C:/Users/Rob/Projecten/SemanticWorldModel/repo/SWM.main/content/IMPLEMENTATION-DOMAIN/reverseArrayIterator.xqm, 4/27: [XQST0049] Duplicate declaration of static variable .........' It looks like the function traverse all through the module structure following the module imports. And when it encouters a cyclic situation, which is not abnormal within a module structure, the function trigers this error. I think it should just stop on that branche because obviously the model has already been processed. Maybe even, the function should not iterate through the module structure at all, just returning the functions of the requested module. Maybe I'm just doing things wrong here. Can you please shed some light on how to prevent this error when using 'inspect:functions'? Some background for this madness: I want to annotate my FunctionDataStructure, dataStructures as functioncallparameterset, with the name of the module that contains the functions for this type/class of data (class and methods in OO). Thanks in advance for your responce. mvgr. Rob Stapper

2 6

BaseX 11.8: Lightning fast maps, new XQuery 4 features
by Christian Grün 30 Apr '25

30 Apr '25

We are delighted to announce BaseX 11.8, one more important step to the next major version. Here are the most significant changes: - [MOD] Lightning fast construction of ordered maps of all types - [MOD] fn:distinct-values, fn:duplicate-values: faster comparisons - [MOD] Faster map comparison (ordered and unordered) - [ADD] New function util:values-except - [ADD] XQuery 4.0 functionality (functions, %method annotation) - [MOD] XQuery fold functions: more early exits - [MOD] XQuery, fn:parse-xml: options added - [MOD] prof:variables: Better capture of in-scope variables - [FIX] Stricter permission checks when calling built-in functions For a preview of the upcoming 12 features, please visit the following page… https://docs.basex.org/12 …and have a look at our homepage to download the latest version: https://basex.org Have fun, Your BaseX Team

2 3

Problem in Java extension module
by Nico Verwer (Rakensi) 24 Apr '25

24 Apr '25

Greetings, BaseX creators and users! I am trying to make a custom XQuery function in Java. This works fine in most cases, but it seems to lose namespaces on attributes. The Java code in the function converts the BaseX representation of an element into a `org.w3c.dom.Element` DOM; transforms the DOM, creating a new DOM; sends the resulting DOM element back to BaseX. The problem is quite subtle and hard to explain in a few sentences. Therefore I have made a project demonstrating it, which can be seen on [Github](https://github.com/nverwer/possible-bug-in-BaseX). The project includes Java code and a simple XQuery that demonstrates the problem. I am at the point where I think the problem could be a bug in BaseX, which is why I thought it would be a good idea to post this to the basex-talk mailing list. If someone with more knowledge of the inner workings of BaseX could help me, I would much appreciate. Best regards, Nico Verwer

3 2

Way to record node IDs that does not depend on database name?
by Eliot Kimber 22 Apr '25

22 Apr '25

In my long-running data load process that appears to fail, I’ve found the issue but I don’t see an obvious way to correct it. My process creates a temporary content database that contains the latest content version of content previously loaded. This temp database is then the source for a process that creates a set of where-used index records in another database that point to the nodes in the temp content database by node ID. The node-recording elements look like this: <noderef node-id="43617" database="pce-test-data" tagname="mapref" baseuri="/pce-test-data/encryption-support.ditamap" href="cloud-encryption.ditamap" /> Note the “database” attribute: it’s the name of the database the node ID is from. After the process as completed constructing all the where-used records and is ready to swap these new databases into production, I have an XSLT transform that updates the values of the @database attributes to replace the temporary database name with the production name (i.e., remove leading “_temp_” from the database name. I then swap the temp databases in place of the old databases, putting the new data into production. This works fine at small scales, but when I attempt it with my 200K-link database, the XSLT transform either simply never completes or fails in the backgroujnd or would take so long to complete that it would be impractical. In any case, this approach does not work for my full-scale case ☹ So my question is: How can avoid this need to update my node reference elements to reflect the new database name? One solution that comes to mind is simply not recording the database name on the <noderef> element but somewhere else, say in the root element of the document that contains the <nodere>, but that requires that all the <noderef> elements in that context target the same database, which will be true in this case but might not be true in the future (I had designed <noderef> to enable mixing references to nodes in different databases). I could also have the code that’s creating these where-used records manage the prod-to-temp database name dynamically (and that may be my best solution the more I think about it) but starts to look like magic and I try to avoid magic code. So a solution that is less fragile would be ideal. Changing the value requires an update of some sort, whether it’s via XSLT or XQuery update, it’s going to be problematic at this scale. Is there any solution I’ve overlooked? Thanks, Eliot _____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368 servicenow servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | X<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Instagram<https://www.instagram.com/servicenow>

2 2

Runtime error from DBA log viewer
by Eliot Kimber 19 Apr '25

19 Apr '25

Trying to view today’s log, I get this runtime failure: Stopped at /data1/basex/webapp/dba/logs/logs.xqm, 220/25: [FORG0001] Cannot convert to xs:decimal: 3.4345182E+8. Stack Trace: - /data1/basex/webapp/dba/logs/logs.xqm, 157/10 266.98 ms I can provide the log file separately. This is with BaseX 11.6 Cheers, Eliot _____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368 servicenow servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | X<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Instagram<https://www.instagram.com/servicenow>

1 0

Parallelizing writes to a database
by Eliot Kimber 18 Apr '25

18 Apr '25

In my Mirabel system, I create a link database that records all the links made in a set of documents. This becomes a “where used” index over the content. We have on the order if 200K links for one content set, so at 0.1 second per link it takes about 7 hours to build this index. I’m currently doing this in one process that builds the whole index and then stores it in a database. This is failing in hard-to-diagnose ways, for example, because a database has a write lock on it when I go to rename it from it’s temp name to it’s production name (to replace the current production version). The data is such that I could parallelize the processing but I’m not sure how I would do that in BaseX so that I can safely write to a single database from multiple threads. The fork-join() docs clearly say “non-updating” functions, so that doesn’t seem to be an option. I have multiple BaseX HTTP servers running so I could farm processing across them, but I think I would then run into write lock issues. I could create separate databases for each thread of operation and then combine those at the end—that seems like it might be the best option. Have I missed anything? Thanks, Eliot _____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368 servicenow servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | X<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Instagram<https://www.instagram.com/servicenow>

1 0

Bug report - argument = context item, param value several items
by Hans-Juergen Rennau 10 Apr '25

10 Apr '25

Dear BaseX people, I have encountered a bug (?). Please consider this code: = = = = = = = = = begin = = = declare namespace f="http://weatherreport.org/heavyweather"; declare function f:relatedNames($node, $relationship) as xs:string* { let $fnName := function($node) { let $_DEBUG := trace(count($node), '#node: ') return name($node)} let $items := switch($relationship) case 'child' return $node/* default return error() return $items/$fnName(.) => distinct-values() => sort() => string-join(', ')}; let $doc := <doc><a/><b/><c/></doc>return f:relatedNames($doc, 'child') = = = = = = = = = end = = = = = = = = = = = = output = = =#node: 3Stopped at C:/projects/foxpath-examples/pre-balisage/problem-basex2.xq, 15/25:[XPTY0004] Item expected, sequence found: (<a/>, <b/>, <c/>).= = = = = = = = = = = = = = = = The call $fnName(.) receives a single item, but three items arrive at the function item! With kind regards,Hans-Jürgen PS: The funny switch() seems to play a part in the game, when I removed it, the issue vanished.PPS: BaseX 11.8 [Standalone]

2 3

Execution of let clauses versus optimization
by Hans-Juergen Rennau 06 Apr '25

06 Apr '25

Dear BaseX people, I observe a behaviour which appears to me not desirable, although it is, strictly speaking, correct. Please consider this code: let $fnCopy := function($term) {trace('yes', 'within fnCopy: ')} let $_DEBUG := trace('before call fnCopy') let $result := $fnCopy('yogi') (: let $_DEBUG := if ($result eq 'NEVER') then error() else () :) let $_DEBUG := trace('after call fnCopy') return () As long as the commented out clause is not activated, the function is *not* executed. It is strictly speaking correct behaviour: the processor may optimize and skip the call of $fnCopy, as it does not contribute to the value of the containing FLWOR expression. However, when using functions with side-effects (as file:write()), it is of course necessary to be sure that the call is executed. Contrary to other processors, according to my experience BaseX has never skipped let clauses for reasons of optimization, and in many years of usage I have again and again highly appreciated this behaviour. The skipping behaviour is new to me. Is it so that BaseX has changed its policy concerning let clauses not contributing to the FLWOR result? It would be very important to know, because the execution of functions with side effects is everyday practice. Kind regards,Hans-Jürgen

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk April 2025