- BaseX-Talk - mailman.uni-konstanz.de

Full Text bug (?) - window ... words
by Hans-Juergen Rennau 26 Sep '22

26 Sep '22

Dear BaseX people, it seems to me there is a bug concerning Full Text Search, using option "window": let $text1 := '1 The usability of a Web site is how well the site' let $text2 := '2 The usability of a Web site is how well the sitx'let $text3 := '3 The usability of a Web site is how well the site'return ( $text1[. contains text 'usability web site' all words window 5 words], $text2[. contains text 'usability web site' all words window 5 words], $text3[. contains text 'usability web site' all words window 10 words] ) This query should return all three, $text1, $text2 and $text3, but it only returns $text2 and $text3. So it seems to me that the implemented logic is: "all matches of the "all words" search must be within a 5-words window, but it should be "there is a match of the "all words" search which is within a 5-words window. More detailed argument in PS. Kind regards,Hans-Jürgen PS: Compare https://www.w3.org/TR/xpath-full-text-30/#ftwindow "A window selection examines the matches generated by the preceding portion of the FTSelection, and selects those for which the matched tokens and phrases (more precisely, the individual StringIncludes of that match) are all found within a window whose size is a specified number of FTUnits (words, sentences, or paragraphs); for each such window, the window selection then generates a match containing the merge of those StringIncludes, plus any StringExcludes that fall within the window." (Italic added by me) The detailed semantics are given in 4.2.4 FTWords [1]. Pseudo-function fts:applyFtWordsAllWord() constructs for each of the words an fts:allMatches element and then performs conjunction of these elements, based on recursive application of 4.3.6.2 FTAnd [2]. Each fts:allMatches element contains one fts:match element for each occurrence of the word in question. The "ANDing" of two fts:allMatches is described by pseudo-function fts:ApplyFTAnd(), which creates one match for each pair of matches found in the operands - in other words, all combinations of operand matches are considered. As an example consider: "foo bar" all words and assume "foo" occurs two times and "bar" occurs two times. The fts:allMatches element representing the result is the result of applying fts:ApplyFTAnd() to two fts:allMatches elements, one obtained for word "foo", one obtained for word "bar". Schematically: $fts:allMatches_foo: <fts:match>foo(1)</fts:match> <fts:match>foo(2)</fts:match> $fts:allMatches_bar: <fts:match>bar(1)</fts:match> <fts:match>bar(2)</fts:match> fts:applyFTAnd($fts:allMatches_foo, $fts:allMatches_bar) is a single fts:allMatches containing four matches, each one of which is a combination of matches found in the operands: <fts:allMatches_allwords> = <fts:match>foo(1), bar(1)</fts:match> <fts:match>foo(1), bar(2)</fts:match> <fts:match>foo(2), bar(1)</fts:match> <fts:match>foo(2), bar(2)</fts:match> Now extend the query to "foo bar" all words window 5 words The result is true, if there is at least one combination of occurrences of "foo" and "bar" found in a window of at most 5 for word "bar". Referring to the semantic intermediates: if <fts:allMatches_allwords> contains at least one <fts:match> element for which all contained matches satisfy the window condition. [1] https://www.w3.org/TR/xpath-full-text-30/#tq-ft-fs-FTWords[2] https://www.w3.org/TR/xpath-full-text-30/#tq-ft-fs-FTAnd

2 1

Java 11
by Giuseppe G. A. Celano 13 Sep '22

13 Sep '22

Hi All, On the BaseX website (https://basex.org/download/ <https://basex.org/download/>), it is specified that “Java 11 is required to install and run BaseX (10.1)” . Since it is not specified “Java 11 or higher”, I am wondering whether this requirement is actually meant to be strict. Thanks! Best, Giuseppe

2 1

current-dateTime() function precision
by ykhabins＠bellsouth.net 09 Sep '22

09 Sep '22

Hello, I am using BaseX v.10.1 It seems that the current-dateTime() function precision is off. I am validating an XML file against an XSD 1.1 file via Xerces 2.12.2 validator. And trying to measure a timing of it. Here is my code. xquery version "4.0"; let $xml := '\\...\AForm-XSD-20211013\PD220224062681.XML' let $xsd := '\\...\AForm-XSD-20211013\Miami-ws-AForm.xsd' let $before_datetime := current-dateTime() let $result := validate:xsd-report($xml, $xsd, map { 'http://apache.org/xml/features/validation/cta-full-xpath-checking': true() }) return <root><metadata> <result>{data($result/status)}</result> <errors>{count($result/message)}</errors> <startTime>{$before_datetime}</startTime> <endTime>{current-dateTime()}</endTime> <xmlFile>{$xml}</xmlFile> <xsdFile>{$xsd}</xsdFile> <xsdProcessor>BaseX {data(db:system()//version)}, EE-Java, {validate:xsd-processor()} 2.12.2</xsdProcessor> <xsdVersion>{validate:xsd-version()}</xsdVersion> </metadata> <messages>{$result/message}</messages> </root> It is emitting the following output: <root> <metadata> <result>invalid</result> <errors>3</errors> <startTime>2022-09-09T11:32:29.667-04:00</startTime> <endTime>2022-09-09T11:32:29.667-04:00</endTime> <xmlFile>\\...\AForm-XSD-20211013\PD220224062681.XML</xmlFile> <xsdFile>\\...\AForm-XSD-20211013\Miami-ws-AForm.xsd</xsdFile> <xsdProcessor>BaseX 10.1, EE-Java, Xerces 2.12.2</xsdProcessor> <xsdVersion>1.1</xsdVersion> </metadata> <messages> ... </messages> </root> The question is why the before and after timing is the same: <startTime>2022-09-09T11:32:29.667-04:00</startTime> <endTime>2022-09-09T11:32:29.667-04:00</endTime> They are identical up to a millisecond. My expectations were that at least the millisecond time portion would be different. Regards, Yitzhak Khabinsky

3 8

Copy Modify Return - base URI preserved?
by Hans-Juergen Rennau 07 Sep '22

07 Sep '22

Dear BaseX people, I just observed that a document obtained by applying Copy/Modify/Return to a source document preserves its base URI, that is, base-uri($resultDoc) = base-uri($sourceDoc). That is marvellous, but can I rely on this as a feature? (In comparison - if I obtained the result document in a "conventional" way by recursive processing of the source document, the base URI would be lost, unless I added an @xml:base.) Kind regards,Hans-Jürgen

2 1

Basex query for checking size of node
by Sarath Surisetty 03 Sep '22

03 Sep '22

Problem statement: one of the project stores the logs in BaseX, and size of the baseX increases nearly 1gb to 10gb . when the size of BaseX increases, while performing queries to baseX, noticed that the CPU utilization exceeded more than 200% and performance is degraded drastically, But when some of logs data removed from BaseX manually BaseX performance is good. could u help in , 1)We are getting issues from customers env , where we don't have access to analyze which node is consuming more storage or which query is consuming more cpu and customer are not supposed to use basex gui. Could you give the query(Xquery) to print list of nodes(xmls) and its sizes ? 2)could you share any suggestions to resolve the performance issues. thanks sarath Regards Sarath

1 0

Am I using proc:exec wrong?
by Graydon Saunders 03 Sep '22

03 Sep '22

Hello -- This is with BaseX 10.1 on a linux box: 18:02 graydon % java -version openjdk version "17.0.4" 2022-07-19 OpenJDK Runtime Environment (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8) OpenJDK 64-Bit Server VM (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8, mixed mode, sharing) I'm trying to invoke CoffeePot, https://coffeepot.nineml.org/ which is an ixml processor. In the BaseX GUI, proc:execute('/usr/bin/java','-version') gives me the expected version information as an error, despite a return code of zero: <result><error>openjdk version "17.0.4" 2022-07-19 OpenJDK Runtime Environment (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8) OpenJDK 64-Bit Server VM (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8, mixed mode, sharing) </error><code>0</code></result> >From a console, /usr/bin/java -cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main works fine, in as much as I get the list of coffeepot usage information I expect if it's run without arguments. proc:execute('/usr/bin/java','-cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main') returns <result><error>Unrecognized option: -cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. </error><code>1</code></result> I suspect I'm doing something wrong, but it's not at all clear to me what. Anyone have suggestions? Thanks! Graydon

3 2

Problem with validate:xsd-report
by Hans-Juergen Rennau 01 Sep '22

01 Sep '22

Dear BaseX people, I encounter a problem with validate:xsd-report(), more particular with using a schema importing the well-known xml.xsd. The imported attribute declaration (xml:base) is not found. I am pretty sure that it is my fault, but I just can't find my error - could you take a look? (1) Document: doc.xml <book xml:base="/a/b/c" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="schema.xsd"> <author>xyz</author></book> (2) Schema: schema.xsd <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"> <xs:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="xml.xsd"/> <xs:element name="book"> <xs:complexType> <xs:sequence> <xs:element name="author" type="xs:string"/> </xs:sequence> <xs:attribute ref="xml:base"/> </xs:complexType> </xs:element></xs:schema> (In the same folder, a downloaded copy of xml.xsd is stored - for a snippet, see PS) (3) Query: validate.xq let $doc := 'doc.xml'let $xsd := 'schema.xsd'return validate:xsd-report($doc, $xsd) (4) Result: basex validate.xq=><report> <status>invalid</status> <message level="Fatal" line="9" column="43" url="file:/C:/projects/d2type/jats/test-validation/schema.xsd">src-resolve: Cannot resolve the name 'xml:base' to a(n) 'attribute declaration' component. </message></report> I do not understand why the import seems to be ignored. Validating with oXygen works fine, but I suppose that oXygen ignores the @schemaLocation. If you see the reason, please let me know. Cheers,Hans-Jürgen PS: Snippet from xml.xsd, stored in the same folder: <?xml version='1.0'?><!DOCTYPE xs:schema PUBLIC "-//W3C//DTD XMLSCHEMA 200102//EN" "XMLSchema.dtd" ><xs:schema targetNamespace="http://www.w3.org/XML/1998/namespace" xmlns:xs="http://www.w3.org/2001/XMLSchema" xml:lang="en"> ... <xs:attribute name="base" type="xs:anyURI"> <xs:annotation> <xs:documentation>See http://www.w3.org/TR/xmlbase/ for information about this attribute.</xs:documentation> </xs:annotation> </xs:attribute> ...</xs:schema>

2 2

Exception handling in XQuery-Java-Binding
by Andreas Hengsbach | nexoma 29 Aug '22

29 Aug '22

Hi, I am currently experimenting with java code loaded as a basex module. There I throw custom exceptions, which I want to catch and handle in XQuery. Unfortunately, the information about the thrown Java exception seems to get lost. What options do I have to throw my own error messages from my java module that do not immediately become "XPTY0004"? Many thanks and best regards Andreas

2 2

Re: [basex-talk] an Earley parser in XQuery?
by C. M. Sperberg-McQueen 25 Aug '22

25 Aug '22

On 23 August 2022, Graydon Saunders wrote: > On Mon, Aug 22, 2022 at 07:42:39PM +0200, Martin Honnen scripsit: > > On 22.08.2022 19:07, Graydon Saunders wrote: > > > I'm trying to test if some extracted sentences validate as productions > > > of a particular context-free grammar expressed in a BNF dialect. > > > > > > Is there an available implementation of a parser in XQuery that can do > > > this? > > > > > In the context of Invisible XML https://invisiblexml.org/ I have heard > > that term "Earley" parser a lot but I am not familiar enough with the > > various implementations to tell whether all are "Earley" parsers, > > https://github.com/cmsmcq/Aparecium is an XQuery implementation of > > Invisible XML. For what it's worth: Of the ixml processors I'm familiar with, all use (or can use) Earley parsers to parse the input string against the input grammar; Norm Walsh's Coffeepot and its related libraries also make a GLL parser available, which is sometimes faster and sometimes slower. About half use the same parser to parse the input grammar against the spec grammar; about half have a hand-written parser for the ixml spec grammar. > And thus interesting, but alas too slow for the application I have. If you are willing and able to send me a copy of your grammar and some sample data, I would be grateful; it might help me improve the situation with respect to Aparecium's performance. Both Norm Walsh's processor and John Lumley's processor are currently a lot faster than Aparecium, though I don't know a simple way to call them directly from XQuery. Another alternative, if you have speed issues and just one grammar you are interested in, is to use Gunther Rademacher's REx parser generator to compile the grammar you are interested in into a parser written in XQuery (other target languages are also available). https://bottlecaps.de/rex/ -- C. M. Sperberg-McQueen Black Mesa Technologies LLC http://blackmesatech.com

2 1

an Earley parser in XQuery?
by Graydon Saunders 23 Aug '22

23 Aug '22

Hello -- I'm trying to test if some extracted sentences validate as productions of a particular context-free grammar expressed in a BNF dialect. Is there an available implementation of a parser in XQuery that can do this? Thanks! Graydon

4 4