BaseX-Talk February 2024

basex-talk@mailman.uni-konstanz.de

19 participants
20 discussions

Found problems with map:for-each
by Vincenzo Cestone 20 Feb '24

20 Feb '24

Hi all, with the basex 10.7 version (but even in 9.6 version I have the same issue) I found that the following code wont work as expected: (: It wont work :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := map:for-each($files, function($filename, $content) { file:write-binary($filename, $content, 0) }) return () that is, it will not write the two files hello1.txt and hello2.txt in the basex_home/bin folder. But, if you return $w instead: (: It works :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := map:for-each($files, function($filename, $content) { file:write-binary($filename, $content, 0) }) return $w With a similar implementation, with the classic FLWOR, the issue does not arise, even if I return the empty sequence: (: It works :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := for $filename in map:keys($files) return file:write-binary($filename, map:get($files, $filename), 0) return () that is it will write two files hello1.txt and hello2.txt in basex_home/bin folder. Note that the files in the examples above are for demonstration purposes, however in my code they are actually binary files. My java version is a Oracle JDK 17 This problem also occur to others? Thanks, Vincenzo

2 2

Help with usage of Regex while deleting resources
by Deepak Dinakara 20 Feb '24

20 Feb '24

Hi, I wanted to know if it's possible to give a regex while deleting a resource. I have documents stored in a hierarchy of collections like {year}{month}/doc.xml. Eg: 202301/abc.xml, 202302/def.xml. If I want to delete a resource "abc.xml", Is it possible to issue commands like "*db:delete("db-name", '/*/abc.xml')*" ? Right now, I can do a XQuery with db:list and endsWith and get the complete path of "abc.xml". But regex would have been very handy. Similarly I also want to execute queries against a list of collections using regex. Something like "*for $document in collection('db-name/20230*')*" (First 9 months of 2023) Right now, I am doing something like "for $i in ('01', '02', '03', '04', ... '09') for $document in collection('test-collection/2023' || $i)" But if there are better ways, kindly let me know. Thank you, Deepak

7 10

Slow full-text querying
by Murray, Gregory 18 Feb '24

18 Feb '24

Hello, I have a database with several thousand XML documents, although I have tens of thousands I’d like to add. Each XML document contains a book — both the bibliographic metadata such as title, author, etc. (each in its own element) and the complete OCR text of all pages of the book. Each page of text from each book is in a <page> element with a single text node containing all words from that page in the book, resulting in large blocks of text. I’ve added a full-text index and optimized it. I am finding that full-text searching is very slow. The query shown below consistently takes about 20 seconds to run, even though there are only about 7400 documents. Obviously that’s far too slow to use the query in a web application, where the user expects a quick response. My first thought is whether the query is actually using the full-text index. Is there a way for me to determine that? I’m also wondering if my query is crude or is missing something. I don’t need the text nodes containing the search words; I only need to know which documents contain the words. let $keywords := "apple" for $doc in collection("theocom") let score $score := $doc contains text {$keywords} order by $score descending where $score > 0 return concat($score, " ", base-uri($doc)) As you can see, I’m searching all text in the entirety of each book. Is there a way to rewrite such a query for faster performance? Also, I’m wondering if the structure of the XML documents is such that the documents themselves need to have smaller blocks of text. For example, if the OCR text were contained in <line> elements, each containing only a single line of text, as printed in the original physical book, would full-text searching be noticeably faster, since each text node is much smaller? Thanks, Greg

2 2

XSLT version
by Martin Dowling 08 Feb '24

08 Feb '24

Hi I'm trying to run a query in BaseX (version 8.6.7) which calls an XSLT 2.0 stylesheet using xslt:transform(). I'm getting errors that clearly imply the system does not recognise XSLT 2.0 functions, and is expecting an XSLT 1.0 stylesheet. However, when I run xslt:version(), it returns "2.0" - which implies the system is set up for XSLT 2.0 stylesheets. Is there some other setting somewhere that might need to be updated, to enable the system to run an XSLT 2.0 transform? Thank you, Martin ________________________________ This email and its contents are confidential and may also be privileged and protected by copyright. If you are not the named recipient please notify the sender immediately, delete the email and all attachments completely from your system and do not use, print, copy, disclose or distribute any part of its contents. Bloomsbury Publishing Plc and its subsidiaries may monitor email traffic. Any views expressed in this email are those of the author, and do not necessarily represent those of Bloomsbury. This email does not conclude a binding agreement. Bloomsbury Publishing Plc 50 Bedford Square, London WC1B 3DP. Company registered in England and Wales company no. 1984336. Bloomsbury Publishing Inc. 1385 Broadway, Fifth floor, New York, 10018. Bloomsbury Publishing Pty Ltd, Level 4, 387 George St, Sydney, NSW 2000. Bloomsbury Publishing India Pvt. Ltd. DDA Complex, LSC , Building No.4, Second Floor, Pocket C-6&7, Vasant Kunj, New Delhi 110070. ABC-CLIO | Bloomsbury, ABC-CLIO is an imprint of Bloomsbury, 147 Castilian Drive, Santa Barbara, CA 93117.

4 4

basexhttp out of memory where basexgui suceeds
by Jack Steyn 08 Feb '24

08 Feb '24

Hi, I have a database about 200 MB in size made up of approximately 150 000 documents of similar size and structure as children of the root node. When I run the following script in basexgui a significant amount of memory is consumed (over 1 GB if I'm reading the display correctly), but I do get a result: for $doc in db:get('docs') return $doc update { delete node .//*[local-name() = ('A', 'B', 'C', 'D')] } When I run it over basexhttp I get a java.lang.OutOfMemoryError: Java heap space. I have increased the memory available to the JVM to 4 GB but this has not affected the failure of the script. How can I resolve this? Is there some rewriting of the script that would help, or is it more specific to basexhttp? Many thanks, Jack

2 4

Query regarding delete documents
by Deepak Dinakara 05 Feb '24

05 Feb '24

Hi, I wanted to understand more on how deleting documents impact the storage layout. I did go through https://docs.basex.org/wiki/Storage_Layout but didn't find all the required information. I observed that size of files like "txt.basex" and "atv.basex" doesn't shrink on deleting documents. Is that expected? PS: Thanks for the great product. Thank you, Deepak

2 3

A question about (non standard) BaseX stopping
by Pierre-Yves Jallud 05 Feb '24

05 Feb '24

Hi all, after having configuring a BaseX hub (|cluster|farm|...?), I have a last problem (I hope 🤞😅). When I want to stop a BaseX instance with basexhttpstop (or basexhttp stop), I have different results, but none stop the instance. The context: * I'm using the following java version: *OpenJDK 64-Bit Server VM (Red_Hat-11.0.21.0.9-1.el7_9) (build 11.0.21+9-LTS, mixed mode, sharing)* * in the .bash_profile of the basexusr1 (and for each (system LINUX) BaseX user), I precise the BASEX_JVM variable (*export BASEX_JVM="-Dorg.basex.path=/basex-ferme/sites/basex1"*) to customize the webapp and data repository * */basex-ferme/sites/basex1/basex* is a symbolic link to the BaseX reference (every site has this symbolic link) located here: */basex-ferme/basex* (hum... this is also a symbolic link pointing to the last version of basex: */basex-ferme/basex-10.7*) * the ports are different for each instance and are (only) precised when I launch the BaseX process * I launch each process with this command (adapting basex1, basexusr1 and ports for each BaseX): *su - -s /bin/bash basexusr1 -c "/basex-ferme/sites/basex1/basex/bin/basexhttp -S -h11204 -p11205 -s11206" * * for each command below, the code returned is ($?) 0... but the process is still running 🤔 (as the web site). I only test the access of the DBA service (logging as admin) and I've got no database(at this time). The result of the stop commands: 1. > su - -s /bin/bash basexusr1 -c "/basex-ferme/sites/basex1/basex/bin/basexhttpstop -h11204 -p11205 -s11206" BaseX 10.7 [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @1523ms to org.eclipse.jetty.util.log.Slf4jLog HTTP Server was stopped (port: 11204). Server was stopped (port: 11205). 2. > su - -s /bin/bash basexusr1-c "/basex-ferme/sites/basex1/basex/bin/basexhttp stop -h11204 -p11205 -s11206" BaseX 10.7 [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @1441ms to org.eclipse.jetty.util.log.Slf4jLog HTTP Server was stopped (port: 11204). Server was stopped (port: 11205). 3. > su - -s /bin/bash basexusr1-c "/basex-ferme/sites/basex1/basex/bin/basexhttp stop -s11206" BaseX 10.7 [HTTP Server]: 11205). [main] INFO org.eclipse.jetty.util.log - Logging initialized @1819ms to org.eclipse.jetty.util.log.Slf4jLog HTTP Server was stopped (port: 8080). Server was stopped (port: 1984). 4. > su - -s /bin/bash basexusr1-c "/basex-ferme/sites/basex1/basex/bin/basexhttp stop -h11204 -s11206" BaseX 10.7 [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @1366ms to org.eclipse.jetty.util.log.Slf4jLog HTTP Server was stopped (port: 11204). Server was stopped (port: 1984). 5. > su - -s /bin/bash basexusr1-c "/basex-ferme/sites/basex1/basex/bin/basexhttp stop -p11205 -s11206" BaseX 10.7 [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @1006ms to org.eclipse.jetty.util.log.Slf4jLog HTTP Server was stopped (port: 8080). Server was stopped (port: 11205). ... and there is no track of the stop request in the log file (but there are the tracks of the admin access). The only solution I have found is to kill the process. That's my plan B, but I hope there is another cleaner solution 🤞😎 Has anyone a suggestion to resolve my problem?... and sorry for the complexity of the context 😓 Greetings Pierre-Yves ------------------------------------------------------------------------ *Pierre-Yves Jallud* /Pôle Humanité Numériques / Ingénieur en ingénierie logicielle/ ------------------------------------------------------------------------ IHRIM - UMR 5317 <http://ihrim.ens-lyon.fr> CNRS <https://www.cnrs.fr> ENS de Lyon <http://www.ens-lyon.fr> 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07 +33 (0)4 37 37 63 83 - pierre-yves.jallud(a)ens-lyon.fr ------------------------------------------------------------------------

2 1

hof:until qt4cg drops?
by Andy Bunce 03 Feb '24

03 Feb '24

I wanted to look up the hof:until[1] syntax in the wiki. It tells me that it has been replaced with fn:iterate-while. Well, I hope, when the dust settles, there is something similar to this in the spec, but it looks like it may not be iterate-while . Today a user of the current BaseX release(10.7) is going to struggle to find what hof:until does and wont find any fn:iterate-while . I think deprecating functions and moving forward is great. I do like to see what 4.0 might bring with the BaseX 11 betas but if I catch myself writing anything substantial I stop. The ground is unstable. I think the wiki should center on the functionality of the current release, with just teasers about the future. Christian, I think you hinted that the future may hold some documentation API built into the product, or did I imagine that? /Andy [1] https://docs.basex.org/wiki/Higher-Order_Functions_Module,

2 3

How to configure org.basex.path
by Pierre-Yves Jallud 03 Feb '24

03 Feb '24

Hi all, I would like to precise the home directory of BaseX using org.basex.path (cf. https://docs.basex.org/wiki/Configuration#Home_Directory). Is it possible to configure this when I launch the basexhttp? For exemple, something like (befor launching basexhttp): export org.basex.path=/path/to/my/basex Or maybe: basexhttp -S org.basex.path=/path/to/my/basex ... or what else? Many thanks in advance to enlighten me 😇 Pierre-Yves PS: as you can see, I didn't use Java for a long time 😅 ------------------------------------------------------------------------ *Pierre-Yves Jallud* /Pôle Humanité Numériques / Ingénieur en ingénierie logicielle/ ------------------------------------------------------------------------ IHRIM - UMR 5317 <http://ihrim.ens-lyon.fr> CNRS <https://www.cnrs.fr> ENS de Lyon <http://www.ens-lyon.fr> 15 Parvis René Descartes - BP7000 - 69342 Lyon CEDEX 07 +33 (0)4 37 37 63 83 - pierre-yves.jallud(a)ens-lyon.fr ------------------------------------------------------------------------

3 5

RESTXQ knocked out!
by Hans-Juergen Rennau 01 Feb '24

01 Feb '24

Dear BaseX people, suddenly RESTXQ does not work any more on my machine. (Here, "suddenly" means when using RESTXQ again after a pause of three months and having installed a new BaseX version in the meantime, 11.0 beta.) Any attempt to access a RESTXQ-defined URL produces a mysterious error message: HTTP ERROR 500 java.lang.NoClassDefFoundError: org/basex/build/text/TextOptions URI: /dbaSTATUS: 500MESSAGE: java.lang.NoClassDefFoundError: org/basex/build/text/TextOptionsSERVLET: RESTXQCAUSED BY: java.lang.NoClassDefFoundError: org/basex/build/text/TextOptionsCAUSED BY: java.lang.ClassNotFoundException: org.basex.build.text.TextOptionsCaused by:java.lang.NoClassDefFoundError: org/basex/build/text/TextOptions at org.basex.http.web.WebModule.parse(WebModule.java:63) at org.basex.http.web.WebModules.parse(WebModules.java:376) at org.basex.http.web.WebModules.parse(WebModules.java:367) at org.basex.http.web.WebModules.cache(WebModules.java:336) at org.basex.http.web.WebModules.find(WebModules.java:149) at org.basex.http.web.WebModules.restxq(WebModules.java:116) Emptying the lib/custom folder did not help. Neither did a reinstallation of BaseX (see PS). With kind regards -Hans-Jürgen PS: After observing the error, I tried to install the official release (10.7). To my amazement, the version did not change, it remained the 11.0 beta snapshot, even after restarting the machine. And, of course, the error remained. (After this, I felt knocked out myself. Is there anything that I understand?)

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk February 2024