BaseX-Talk

basex-talk@mailman.uni-konstanz.de

8 participants
5255 discussions

How Best to Determine What BaseX Is Doing When It Pegs the Processor?
by Eliot Kimber 03 Mar '22

03 Mar '22

I have a RESTXQ page that makes a query call to construct a report and then displays it. When I run this page something is triggering BaseX to peg the processor and it basically never returns from whatever it is doing. Using proc:dump() I can see that the call to construct the report runs and returns the expected value—this takes about 3 seconds, and then the page never returns and top shows that the java process for the BaseX HTTP server is pegged at 100%. The behavior is consistent so I know it must be something I’m doing or the effect of some configuration setting I have but no idea what might be going on. I get the behavior both on my development machine and on my production server, so it must be something in my code or some reproducible issue with the databases I’m managing. I’ve tried connecting the BaseX HTTP server to the Eclipse debugger, which works, but I’m not sure where I should be focusing my attention in the code. Turning on the -d flag isn’t that helpful either as the log is flooded with exceptions from BaseX trying to load Java classes for my XQuery modules, which swamps any more detailed debugging log messages and what messages I see appear to be entirely about the HTTP interaction, not what’s happening in the core BaseX server. Is there a particular place in the code I can focus on in the interactive debugger that would be a likely source of the behavior or another debugging technique I haven’t considered? Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 4

Re: [basex-talk] Declaring a variable in bxs that carries over to called xquery scripts
by ETANCHAUD Fabrice 02 Mar '22

02 Mar '22

Hi Patrick, I don't think bxs scripts have something like variables (but I may have missed the point with all the work BaseX team achieves). Bindings are way to parametrize XQuery scripts, and you can indeed set external variables' value in bxs, or on command line. Best regards, Fabrice Here are a few examples : https://docs.basex.org/wiki/Options#BINDINGS Options - BaseX Documentation<https://docs.basex.org/wiki/Options#BINDINGS> This page is linked from the Getting Started Section.. The options listed on this page influence the way how database commands are executed and XQuery expressions are evaluated. Two kinds of options exist: Global Options are valid for all BaseX instances in the same JVM. This is particularly relevant if you are working with the client/server architecture. docs.basex.org https://docs.basex.org/wiki/Command-Line_Options Command-Line Options - BaseX Documentation<https://docs.basex.org/wiki/Command-Line_Options> Standalone []. The following options are available for the standalone Command-Line Client: $ basex -h BaseX [Standalone] Usage: basex [-bcdiIoqrRstuvVwxz] [input] [input] XQuery or command file, or query string -b<args> Bind external query variables -c<input> Execute commands from file or string -d Toggle debugging output -i<input> Bind file or database to context -I<input> Bind input string ... docs.basex.org ________________________________ De: BaseX-Talk de la part de Patrick Durusau Envoyé: Mercredi 02 mars 2022 01:38 À: basex-talk(a)mailman.uni-konstanz.de Objet: [basex-talk] Declaring a variable in bxs that carries over to called xquery scripts Greetings! What I thought would be a simple task has turned out to not be so simple. I wrote: SET CHOP false SET EXPORTER indent=no,omit-xml-declaration=no SET BINDINGS $ndb=Ruth CREATE DB $ndb ./sources/Ruth.xml In BaseX 9.6.4, and when testing it I got this result: CHOP: false EXPORTER: indent=no,omit-xml-declaration=no BINDINGS: $ndb=Ruth Database '$ndb' created in 544.5 ms. What? Why isn't the mapping of Ruth to the variable $ndb effective in the bxs file? Moreover, there are query files that follow in the bxs file that I want to have resolve $ndb to Ruth (the desired name of the database). 1) How should I declare a variable in a bxs file? 2) In such a way that it carries over to following xquery files? BTW, I've seen this posting by Christian: https://stackoverflow.com/questions/28018354/how-to-run-a-xquery-with-xml-f… but it doesn't seem to be working for me. What am I missing? Thanks! Patrick <https://twitter.com/servicenow>-- Patrick Durusau patrick(a)durusau.net Technical Advisory Board, OASIS (TAB) Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau

1 0

CREATEFILTER Setting Not Being Applied
by Eliot Kimber 01 Mar '22

01 Mar '22

In my basex/.basex file I have this local option: # Local Options CREATEFILTER=*.xml,*.dita,*.ditamap I have three BaseX instances running. And in the DBA panel for an instance not running on the default port (i.e., on port 10894) I see this in the Local Options column: CREATEFILTER *.xml,*.dita,*.ditamap SKIPCORRUPT ✓ However, if I do this from the Query console of that same server: db:create(‘mydatabase’, ‘/path/to/docs’) The resulting database only reflects .xml files, not .dita or .ditamap files. I’m wondering if this is a bug in how CREATEFILTER is being applied or my misunderstanding of the implications for CREATEFILTER? This is running 9.6.4 on Centos. If I specify createfilter directly on the db:create() call then of course it works as expected. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

3 3

Identify Unparseable XML Files in File System
by Eliot Kimber 26 Feb '22

26 Feb '22

My validation server loads all the XML documents it can from the file system. I have “skipcorrupt” set to true so that non-well-formed documents don’t fail the database creation or update attempt. However, as part of my validation services, I need to be able to report those documents that are not well formed and therefore didn’t make it into the database. I’m wondering what the easiest/most efficient way to do that would be within BaseX? I’m working with on the order of 36K files. The files as stored in BaseX have the same path and filename as the files on the file system relative to the directory I import from, so the correlation between files and docs in BaseX is direct and simple. One easy solution would be to simply get the disjoint of the list of files on the file system and the docs in the database and then attempt to load each file to verify that is in fact unparseable and not just not-yet-imported. But maybe there’s a more direct way that I’ve overlooked? Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 2

Integration with LDAP for Authentication?
by Eliot Kimber 26 Feb '22

26 Feb '22

For my validation application (now christened “Project Mirabel” after the Encanto character Mirabel Madrigal), I want to add user validation and authentication via connection to our internal LDAP server I think the solution is to build a custom version of BaseX server that replaces the build-in base User implementation with one that uses JNDI to talk to the LDAP server for user validation and authentication. Looking at the Java code I’m not quite sure what the best way to go about this is but I think it’s to provide a library that provides my custom LDAP-using User classes and then replace the built-in user module implementation with one that uses my User classes rather than the base User classes. Am I on the right track? I didn’t see anything that looked like a way to do a run-time configuration of the classes used for authentication. I kind of expected something like a UserValidator interface that is then used by the UserCheck implementation, which would allow for different UserValidator implementations to be set at runtime but the code is much simpler than that, which is obviously sufficient for the majority of BaseX uses. Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

1 0

Multithreading
by Philip Oliver 26 Feb '22

26 Feb '22

There are some very old messages regarding the subject of multithreading in the BaseX server but nothing recent that I can find. With the proliferation of CPUs with large core counts, it seems logical to revisit the topic so that a BaseX server can make efficient use of that resource. Has this been considered lately? BaseX is a fantastic system but I'm afraid that it receives less attention than it should because competitors, particularly Elasticsearch/OpenSearch, are architected to take advantage of multithreading and clustering. In my case, the vast majority of the time, the engine is used on a static database, for full text searching. Even a constrained case of permitted multithreading on a (temporarily) read-only database would be very useful.

1 0

Develop a module, HOWTO
by Ben Engbers 25 Feb '22

25 Feb '22

Hi, I know that it is possible to create a module with functions (I have even done that once), but I can't find the documentation anymore on how to do that. Could please someone provide the URL to this information? Thanks, Ben Engbers

3 3

.
by Christian Grün 24 Feb '22

24 Feb '22

It’s a dire day. We wish safety and the rapid establishment of peace to our fellows and friends in Ukraine. BaseX 9.7 was scheduled for today, it will be postponed.

3 4

string-join with a newline separator?
by Ben Engbers 24 Feb '22

24 Feb '22

Hi, My xml has the structure <base> <element1> <al-group> <al>bla</al> </al-group> </element1> <element1> <al-group> <al>bla</al> </al-group> <al-group> <al>bla</al> </al-group> </element1> </base> The <al-group> element contains 1 to many <al> elements. let $tekst := fn:string-join(fn:data($par//al/text()), ".") concatenates this to: bla.bla.bla But I want it to return: bla bla bla Is it possible to add a newline item-separator to fn:string-join? Ben Engbers

4 4

How to call XQuery function from Java?
by Paul L. Merchant Jr. 23 Feb '22

23 Feb '22

Is it possible for a Java function called from XQuery in BaseX to call back to an XQuery function that's passed to it in BaseX? I haven't been able to find any documentation or examples of this. I got the following example to work, but it doesn't feel right. Is there a better way that might take advantage of the automatic type conversions BaseX performs and doesn't require calling something named "...Internal"? Thanks! ------------------- xquery ------------------- import module namspace java = 'java:my.Module; declare function local:hello($p as xs:string) as xs:string { concat("Hello ", $p) }; java:invoke(local:hello#1) ------------------- Java ------------------- package my; import java.io.UnsupportedEncodingException; import org.basex.query.QueryException; import org.basex.query.QueryModule; import org.basex.query.value.type.AtomType; import org.basex.query.value.item.FuncItem; import org.basex.query.value.item.Str; import org.basex.query.value.Value; import org.basex.util.InputInfo; public class Module extends QueryModule { public String invoke(final FuncItem func) throws UnsupportedEncodingException, QueryException { InputInfo ii = new InputInfo("my.Module.invoke", 0, 0); String s = "Java"; Str p1 = new Str(s.getBytes("UTF-8"), AtomType.STRING); Value[] args = new Value[] {p1}; String r = (String) func.invokeInternal(queryContext, ii, args).toJava(); return r; } }

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk