- BaseX-Talk - mailman.uni-konstanz.de

Optimizing query that reads nodes from two databases
by Matt Dziuban 04 Jun '25

04 Jun '25

Hi all, I'm working on a query that reads from two databases -- OriginalDB and UpdateDB. The structure of each database is the same: <data> <item id="1">...</item> <item id="2">...</item> ... </data> OriginalDB has 2,305,570 items and UpdateDB has 307,019. The query gets all distinct item ids from the two databases, then gets the item with a matching id (if there is one) from each database, and returns the id and both nodes in a JSON object. Here's the query: let $originalDB := db:get('OriginalDB')/data/item let $updateDB := db:get('UpdateDB')/data/item for $id in distinct-values(($originalDB/@id, $updateDB/@id)) let $original := $originalDB[@id = $id] let $update := $updateDB[@id = $id] return json:serialize(map { 'id': xs:long($id), 'original': $original, 'update': $update }, map { 'indent': 'no' }) In its current state this query is very slow -- it's returned only 35 JSON objects in ~30 minutes. How can I go about optimizing it to best take advantage of each database's indices? Thanks in advance, Matt

3 5

BaseX 12.0 is tripping on the XSLT transformation that outputs text, not XML
by ykhabins＠bellsouth.net 04 Jun '25

04 Jun '25

Hello, It seems that BaseX 12.0 beta 2686758, and later, is tripping on the XSLT transformation that outputs text, not XML. I am using SaxonHE12-5J. Even the following XQuery setting is not helping. declare option output:method 'text'; IMHO, BaseX should read the output:method attribute from an XSLT and handle it properly. method? = "xml" | "html" | "xhtml" | "text" | "json" | "adaptive" | eqname https://qt4cg.org/specifications/xslt-40/Overview.html#id-xsl-output-declar… Below is an easy repro. Process3.xslt is working (xsl:output method="xml") Process2.xslt is NOT working (xsl:output method="text") Error: Stopped at C:/Program Files (x86)/BaseX/src/file, 10/15: [FODC0002] "" (Line 1): Content is not allowed in prolog. XQuery ============= (: Outputs the result as xml. :) declare option output:method 'text'; declare variable $input as xs:string := 'e:\Temp\BaseX_XSLT\Input2.xml'; declare variable $stylesheet as xs:string := 'e:\Temp\BaseX_XSLT\Process2.xslt'; xslt:transform($input, $stylesheet) input2.xml ============== <dummy>text</dummy> Process3.xslt ================= <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="xml" omit-xml-declaration="yes"/> <xsl:template match="/dummy"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> Process2.xslt ================ <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output method="text" omit-xml-declaration="yes"/> <xsl:template match="/dummy"> <xsl:value-of select="."/> </xsl:template> </xsl:stylesheet>

3 3

Questions about xslt:transform-text and xslt:transform-report
by Lizzi, Vincent 02 Jun '25

02 Jun '25

Hello BaseX team, I typically use the xslt:transform-text function to apply an XSLT stylesheet and obtain the result that is serialized by Saxon according to the xsl:output instructions in the stylesheet. The xslt:transform-report has the advantage of providing access to messages created by xsl:message instructions. I have a few questions about these functions: Is there a way to use the xsl:transform-report function to obtain messages and the serialized result (instead of document nodes) that is currently returned by xslt:transform-text? Is there a way to provide a base URI in the options of xsl:transform-text (or xsl:transform-report or xsl:transform)? If the input to the transformation is on disk then the base URI is set automatically, however if the input is held in memory then the base URI is not set so it could be helpful to be able to provide a base uri to the transformer as an option. The documentation doesn't mention these capabilities so I'm guessing that these things are not currently possible. Thank you, Vincent ______________________________________________ Vincent M. Lizzi Head of Information Standards | Taylor & Francis Group E-Mail: vincent.lizzi(a)taylorandfrancis.com<mailto:vincent.lizzi@taylorandfrancis.com> Web: www.tandfonline.com<http://www.tandfonline.com/> Time zone: US Eastern Taylor & Francis is a trading name of Informa UK Limited, registered in England under no. 1072954 "Everything should be made as simple as possible, but not simpler." [https://res.public.onecdn.static.microsoft/assets/bookwithme/misc/CalendarP…]<https://outlook.office.com/bookwithme/user/aa80d42cbb5b46dba06a5ad241d7665b…> Book time to meet with me<https://outlook.office.com/bookwithme/user/aa80d42cbb5b46dba06a5ad241d7665b…> Information Classification: General

3 3

difference of type of $schema for different validation functions?
by Martin Honnen 02 Jun '25

02 Jun '25

Why is the function https://docs.basex.org/main/Validation_Functions#validate:xsd documented as taking the schema as item()? while the function https://docs.basex.org/main/Validation_Functions#validate:xsd-report documents the schema as xs:string? ? Is that a documentation error or quirk or are these two functions taking different types of arguments for the schema?

2 1

unexpected results in typecasting functionresult
by Rob Stapper 26 May '25

26 May '25

Christian, I encountered some unexpected behavior with the typecasting of functionresult. it is about the "?" in line 4 of both snippets. [expectedResult.xq], the snippet with the " ?" gives the expected result. [unexpectedResult.xq], the snipped without the "?" , gives an unexpected result. Is this a correct observation or am I missing something? mvgr. Rob Stapper

2 1

BaseX GUI create dialog differences from db:create()?
by Graydon Saunders 26 May '25

26 May '25

Hello and happy Friday -- I'm using BaseX 11.9. I have a processing chain of queries that takes some XHTML documents and loads them into a DB. Another query reads that DB, merges the documents, and writes the merged version to disk. From the disk copy, I create a DB of the merged content, and run another query to update that. (Changing the image references.) I then I use another query to write the updated DB back out to disk. If I create the DB of the merged content with the BaseX GUI "new database" dialog, everything works. If I create the DB of the merged content using db:create('mergedChapters', file:children($mergedDir), (), map { 'ftindex': true(), 'casesens': true(), 'createfilter': '*.html' }) I get a database and I can use db:get() to query it; for example, db:get('mergedChapters')//*:img from a GUI editor window gives me the expected result. (I also get the expected "you can't do that" error if I make the DB the context DB and attempt to run the "load the merged version" query again.) When I try to run the "update the image references" query (also from a GUI editor window, and also using db:get() to access the DB) after creating the mergedChapters dB from XQuery, it does NOT work. So far as I can tell, it doesn't alter the DB at all. I'm very sure I'm using the same database name; it's been copied out of the update-the-image-references query. I don't see an option to make the db changeable in the db:create() options. Any suggestions? Thanks! Graydon

4 4

restxq versus websockets
by Rob Stapper 23 May '25

23 May '25

Christian, if I may ask another question? I have this very interactive client-server application( a web based data-editer). I now use the RESTXQ protocol. I'm wondering if in this scenario a websocket connection would be more efficient and therefor faster? mvgr. Rob Stapper

2 1

preventing tranforming empty element with begin- and endtag to self-closing-tag
by Rob Stapper 23 May '25

23 May '25

Hi Christian, Basex seems to transform 'begin-/endtagged' element with empty content to a single self-closing-tagged element: <div></div> is transformed to: </div> This gives problems in my client application. Both in Chrome, Edge and Firefox transforms Javascript's "insertAdjacentHTML": <div> <div/> <div>tekst</div> <div>tekst</div> </div> to: <div> <div> <div>tekst</div> <div>tekst</div> </div> </div> This is not what is intended. I guess for void-elements this transformation is ok but result in problems like this in case of container-elements. Is it possible in Basex to suppress this transformation? mvgr. Rob Stapper

2 1

Fwd: Re: functionParameterTyping issue
by Rob Stapper 20 May '25

20 May '25

Hi Christian, Good news, I hope. The problem with the dynamic typecasting is not as much a typecasting-issue but an issue with recognizing derived functions. The problem is that derived functions are treated as functioncalls where it shouldn't. I can remenber that once there was an issue with the parity of derived functions. It is something like that. attached some demos: [referentialDemo.xq] shows that dynamic and strict typing works within version-11.9 [buggyDemo.xq] shows that dynamic and strict typing fails in case of derived functions. [workingDemo,xq] show the workaround. Thought you would like to know what the issue is. Rob > ---------- Oorspronkelijk Bericht ---------- > Van: Rob Stapper <r.stapper(a)lijbrandt.nl> > Aan: Christian Grün <christian.gruen(a)gmail.com> > Datum: 14-05-2025 18:43 CEST > Onderwerp: Re: [basex-talk] functionParameterTyping issue > > > >>As can be seen, case "a" can never work if $p is a string. > correct, the error triggered by the system when calling this function is part of the validation mechanisme. Once correctly validated the data wil be stored as XML and wil be garanteed to be a string (in this case). It is not a hidden dependency as such but a managed dependency. > > Thanx for your time, > Rob > > > Op 14-05-2025 18:01 CEST schreef Christian Grün <christian.gruen(a)gmail.com>: > > > > > > > > > It is more like: > > > let $n := "b" > > > let $p := "test" > > > return switch($n) > > > case "a" return fn( $n, $p as map(*) ) { }( ?, $p) > > > case "b" return fn( $n, $p as xs:string ) { }( ?, $p) > > > default return () > > > > > > > As can be seen, case "a" can never work if $p is a string. If there is a (hidden) dependency in the semantics in your code that $p will always be a map if $n is "a", it is something the complete cannot »guess«. In those cases, it is better to omit type declarations in your code. The reason is that the optimizer will also rewrite and simplify those parts of your code that may not be evaluated at runtime. > > > > Hope this helps, > > Christian > > > > > > > > > > > > > > > mvgr. > > Rob Stapper > mvgr. Rob Stapper

2 1

11.9 doc() not working with DOCTYPE declaration
by Graydon Saunders 15 May '25

15 May '25

Hello -- I'm on linux, using 11.9 and using 20:59 graydon % java -version openjdk version "21.0.7" 2025-04-15 OpenJDK Runtime Environment (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6) OpenJDK 64-Bit Server VM (Red_Hat-21.0.7.0.6-1) (build 21.0.7+6, mixed mode, sharing) If I use the basexgui "Create DB" dialog and the internal parser, I can load the whole folder without error; I get the expected DB and the named entities in the content have been resolved. (These are effectively HTML 4 entity names, so that's not completely guaranteeing that the DTD in the DOCTYPE is being found and parsed somewhere.) If I use the doc() function with an absolute file system path to one of the same files that loaded into the DB just fine, I get an error. I get the same error if I try to switch off the internal parser and set a catalog. It's at least several files in the folder that loads fine as a DB. All of this works fine with 11.8; whatever changed changed in 11.9. The error is: Could not execute XQUERY doc('/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml'): Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 11.9 Java: Red Hat, Inc., 21.0.7 OS: Linux, amd64 Stack Trace: javax.xml.catalog.CatalogException: JAXP09040002: No match found for href '/home/graydon/GeneralCode/git/cpc-conversion/Chapter/BU4372-14A.xml' and base '/home/graydon/GeneralCode/git/cpc-conversion/xquery/file'. at java.xml/javax.xml.catalog.CatalogMessages.reportError(CatalogMessages.java:74) at java.xml/javax.xml.catalog.CatalogResolverImpl.resolve(CatalogResolverImpl.java:153) at org.basex.query.StaticContext.resolve(StaticContext.java:156) at org.basex.query.StaticContext.resolve(StaticContext.java:137) at org.basex.query.QueryInput.<init>(QueryInput.java:31) at org.basex.query.func.fn.Docs.queryInput(Docs.java:37) at org.basex.query.func.fn.Docs.lambda$accept$0(Docs.java:67) at org.basex.query.scope.LockVisitor.lock(LockVisitor.java:48) at org.basex.query.func.fn.Docs.accept(Docs.java:56) at org.basex.query.scope.MainModule.databases(MainModule.java:93) at org.basex.query.QueryContext.addLocks(QueryContext.java:412) at org.basex.query.QueryProcessor.addLocks(QueryProcessor.java:230) at org.basex.core.cmd.AQuery.addLocks(AQuery.java:170) at org.basex.core.locks.Locking.acquire(Locking.java:86) at org.basex.core.jobs.Job.register(Job.java:48) at org.basex.core.Command.execute(Command.java:90) at org.basex.gui.GUI.execute(GUI.java:430) at org.basex.gui.GUI.lambda$execute$5(GUI.java:375) at java.base/java.lang.Thread.run(Thread.java:1583) I can go on using 11.8 for now, so this is only somewhat alarming rather than completely alarming. -- Graydon

3 4