July 2024 - BaseX-Talk - mailman.uni-konstanz.de

feature request: control environment of external processes
by Hauke Brandes 16 Jul '24

16 Jul '24

Hello list, I would like to suggest an enhancement for the proc:system/execute/fork family of functions: it should be possible to control the environment variables of the external process. At the moment, the environment is always inherited from the calling basex process, as far as I can tell. There are several reasons why it is desirable to control the environment of external processes: * set variables to values that are not known statically when starting basex * avoid information leaking (restrict environment to the minimal required subset) * control the PATH of the external process It would be great to have an additional, optional entry in the $options map (maybe named "env" or "environment") that contains a map of environment variables for the external process. If absent, the current behavior (inherited environment) should be used. If the option is given, **only** the environment variables in the map should be used for the external process. Effectively, the "env"/"environment" option would behave as if it had a default value of `map:merge( available-environment-variables() ! map:entry(., environment-variable(.)) )` The ProcessBuilder in Java should allow for a straightforward implementation. What do you think? Cheers, Hauke

2 1

How best to handle long-running REST requests?
by Eliot Kimber 15 Jul '24

15 Jul '24

(Using 10.7 for now—upgrade to 11 is planned but not scheduled yet.) In my web app I have implemented client-side on-load JavaScript that implements deferred loading of the results of long-running queries. The JS finds elements that represent callbacks to the server and makes REST calls that are then dispatched to an appropriate worker server based on worker load. These queries can take minutes to run and return a result back to the browser, where the result is added to the HTML page, replacing the original element. In order for the server to maintain the HTTP connection while the query runs I had to turn off the server time out. This works but seems like a crude solution. My question: Is there a better way to manage this kind of client-to-server-to-client communication for long-running queries? For example, is there a way to have the response timeout be more dynamic? I didn’t see anything but I haven’t had time to push on it hard. The result I was getting in the browser with the default 30-second timeout was that when the response eventually returned, the browser reported that the connection had already been closed so it couldn’t do anything with the result. I’m using fetch() to get the result: fetch(url).then(function(response) { return response.text(); }).then(function(data) { const parser = new DOMParser(); const doc = parser.parseFromString(data, 'text/html'); … update web page … } Where “url” is the REST call back to my server. My infrastructure *should* be protecting against long-running queries blocking the main web-serving worker, but it still seems dangerous to have no time out (or a timeout set to 10 minutes or something). Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer Digital Content & Design O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

XSD 1.1 validation: base-uri(.) in the assert
by ykhabins＠bellsouth.net 14 Jul '24

14 Jul '24

Here is the latest development on the subject. I communicated with Mukul Gandhi. He is a chief maintainer of the Xerces-J XSD 1.1 project. Please see below his reply with the good news. >Hi Yitzhak, > >I'm well and hoping same for you. > >I could fix this bug, and the fix for this has been committed to Xerces-J's XML Schema 1.1 dev repos branch at, https://github.com/apache/xerces-j/tree/xml-schema-1.1-dev. > >I've also created a jira issue for this at, https://issues.apache.org/jira/browse/XERCESJ-1765. > >If you wish, you may build and use Xerces-J's XML Schema 1.1 implementation from repos url cited above. > >-- >Regards, >Mukul Gandhi Regards, Yitzhak Khabinsky

1 0

XSD 1.1 validation: base-uri(.) in the assert
by ykhabins＠bellsouth.net 11 Jul '24

11 Jul '24

Hello, We are using the final release BaseX 11.0 and Xerces 2.12.2 XSD 1.1 engine. There is a need to validate XML file and check XML file name for naming convention. Unfortunately, the assert with the base-uri(.) function is not working as expected in the assert expression. Observations: The XPath expression on the XML file emits, as expected, the following: o XPath: base-uri(.) o Results: file:/C:/Users/i300179/Downloads/ATF_NESS/base-uri/input.xml The XPath expression on the XML file emits, as expected, the following: o XPath: tokenize(base-uri(.),"/")[last()] o Results: input.xml I tried multiple expressions around base-uri(.) function. None of them worked. You will find all my attempts in the XSD file. The test case XQuery/XML/XSD is below. We are using Xerces, but the following Saxon excerpt could be pointing to the root cause of the problem. For the reference: https://www.saxonica.com/html/documentation10/functions/fn/base-uri.html "It is possible for the base URI of a document to be absent (unknown). This happens when the document is supplied to Saxon with no location information: for example when the node is represented using a tree model such as DOM that has no base URI information; or when a Source is supplied for which getSystemId() returns null. The most common example of this is a Source created using new StreamSource(new FileInputStream("input.xml")). The problem can be fixed by using the second argument of the StreamSource constructor, or by using the constructor that expects a File rather than an InputStream." XSD11_Validation.xq ======================= declare variable $xml as xs:string external; declare variable $xsd as xs:string external; let $result := validate:xsd-report($xml, $xsd, map { 'http://apache.org/xml/features/validation/cta-full-xpath-checking': true(), 'cache': false() }) return $result input.xml ======================= <root> <event> <id>1</id> </event> </root> input.xsd ======================== <?xml version="1.0" encoding="UTF-8"?> <xs:schema version="1.0.1" attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:vc="http://www.w3.org/2007/XMLSchema-versioning" xmlns:xerces="http://xerces.apache.org" vc:minVersion="1.1"> <xs:element name="root"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" ref="event"/> </xs:sequence>  <xs:assert test="tokenize(base-uri(.), '/')[last()] eq 'input.xml'" xerces:message="Rule #21: XML file name shall match file naming convention">   </xs:assert> </xs:complexType> </xs:element> <xs:element name="event"> <xs:complexType> <xs:sequence> <xs:element ref="id"/> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="id" type="xs:integer"/> </xs:schema> Regards, Yitzhak Khabinsky

2 1

Re: [basex-talk] XSLT with multiple outputs in BaseX 11.0
by ykhabins＠bellsouth.net 03 Jul '24

03 Jul '24

I completely concur. BaseX is a standard bearer of the latest XML standards. We are mostly using BaseX with its HTTP listener for the following: 1. XSD 1.1 validations (via Xerces 12.2.2) 2. XSLT 3.0 transformations (via Saxon engine) and not just as a XML database platform with its XQuery 4.0 support. Regards, Yitzhak Khabinsky

1 0

XSLT with multiple outputs in BaseX 11.0
by ykhabins＠bellsouth.net 03 Jul '24

03 Jul '24

Hello, I am using the latest BaseX 11.0 final. I am running an XSLT 3.0 transformation, by using SaxonHE12-4J, and hit a snag with it. The goal of the XSLT is to split large input XML file into few smaller XML files. The test case files: XML, XSLT, and xq, are below. The XSLT transformation is working as expected, producing multiple output files on the file system. Alas, BaseX GUI emits the following error: Error: Stopped at C:/Program Files (x86)/basex/src/xslt_transform30.xq, 6/15: [FODC0002] "" (Line -1): Premature end of file. My XSLT is a little bit unusual. It is leveraging powerful XSLT 3.0 functionality. It doesn't have one single output, but many output files directly to the file system, IMHO, that's the reason for the BaseX error. BaseX is expecting at least some output back from the XSLT transformation. For comparison, the Oxygen XML IDE is working as expected with such XSLT. What could be done to fix it? May be BaseX behavior needs some adjustment, so it will accept blank output as normal for the xslt:transform($input, $stylesheet)? BaseX is famous for its latest standards support. Modern XSLT versions, i.e. above XSLT 1.0, support multiple outputs. _xslt_transform30.xq =========================================== xquery version "4.0"; declare variable $input as xs:string external; declare variable $stylesheet as xs:string external; xslt:transform($input, $stylesheet) =========================================== Input.xml =========================================== <doc title="example document"> <chapter title="chapter 1"> <page>page 1</page> <page>page 2</page> </chapter> <chapter title="chapter 2"> <page>page 3</page> </chapter> </doc> =========================================== SplitProcess.xslt =========================================== <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:math="http://www.w3.org/2005/xpath-functions/math" exclude-result-prefixes="xs math" version="3.0" expand-text="yes"> <xsl:output indent="yes" method="xml"/>  <xsl:param name="directory" select="'c:/Temp/XSLT30/Split up a file into several files'"/> <xsl:template match="/"> <xsl:apply-templates select="doc/chapter/page"/> </xsl:template> <xsl:template match="page"> <xsl:result-document href="file:///{$directory}/page{position()}.xml"> <xsl:copy-of select="root(snapshot())"/> </xsl:result-document> </xsl:template> </xsl:stylesheet> =========================================== Regards, Yitzhak Khabinsky

5 4

Re: [basex-talk] file:write(...) is not respecting indent parameter correctly
by ykhabins＠bellsouth.net 02 Jul '24

02 Jul '24

The solution was to add a stripws option. After that all output XML files are indented properly because causing the issue whitespaces were removed. Please see the working properly XQuery below. (: https://docs.basex.org/main/Options#stripws :) declare option db:stripws "true"; declare variable $input as xs:string external; declare variable $output_dir as xs:string external; for $x at $i in doc($input)/Events/Event return file:write($output_dir || file:dir-separator() || "event_" || $i || ".xml", $x , map {'method': 'xml', 'indent': 'yes', 'omit-xml-declaration': 'no'}) Regards, Yitzhak Khabinsky

2 1

Command line "-i" directory does not update files when file extension not "xml"
by Zimmel, Daniel 02 Jul '24

02 Jul '24

Hi, is the following behaviour a bug or does it need better documentation? When in a directory of XML files one file extension is not .xml but .xsl then with java -cp BaseX.jar org.basex.BaseX -u -w -i"localpath/" update.xquery only .xml-files get updates (not .xsl) Only with java -cp BaseX.jar org.basex.BaseX -u -w -i"localpath/file.xsl" update.xquery the .xsl-file does get an update. BaseX 11.0 The documentation only says: " Opens the specified XML file, directory with XML files, or database", but not anything about file extensions. https://docs.basex.org/main/Command-Line_Options Thanks, Daniel

2 2

Re: [basex-talk] missing invisible-xml library in BaseX 11.0?
by Gunther Rademacher 02 Jul '24

02 Jul '24

2 1

missing invisible-xml library in BaseX 11.0?
by C. M. Sperberg-McQueen 01 Jul '24

01 Jul '24

Some days ago I downloaded BaseX110.zip and installed it. When I try to invoke the invisible-xml function documented at [1], however, I get the error message: [basex:function] Function invisible-xml requires missing class: de.bottlecaps.markup.Blitz. At first, I thought this might be interference from an earlier version of BaseX (9.0.1, which is what my package manager offered me), but it persists now even after I have uninstalled that earlier version. Did I do something wrong in the installation? Or was there an oversight putting the zip file together? best, Michael Sperberg-McQueen [1] https://docs.basex.org/main/Invisible_XML -- C. M. Sperberg-McQueen Black Mesa Technologies LLC http://blackmesatech.com

1 0