- BaseX-Talk - mailman.uni-konstanz.de

Help with a function to return paths
by Bridger Dyson-Smith 17 Oct '20

17 Oct '20

Hi all - I hope you don't mind a question about serializing distinct XPaths. I'm trying to generate some reports for documentation and the available built-in and library functions[1] aren't quite what I need. The output I'm after is: test/aaa/bbb[@type="foo"] test/aaa/bbb[@type="foo"][@enc="bar"] test/aaa/bbb[@type="bzz"][@enc="bar"] test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="start"] test/aaa/bbb[@type="qux"][@enc="bar"][@key="yes"][@point="end"] I have a couple of functions that are getting me close, but I can't quite manage the output strings, and multiple children are causing me trouble (entirely too much like real life). Any help or suggestions would be greatly appreciated. I've created a gist[2] with the following example: Thanks very much for your time and trouble. Best, Bridger ```xquery xquery version "3.1"; declare variable $input := <test> <aaa> <bbb type="foo">bbb content</bbb> </aaa> <aaa> <bbb type="foo" enc="bar">bbb content</bbb> </aaa> <aaa> <bbb type="bzz" enc="bar">bbb content</bbb> <bbb type="qux" enc="bar" key="yes" point="start">bbb content</bbb> <bbb type="qux" enc="bar" key="yes" point="end">bbb content</bbb> </aaa> </test>; declare function local:elem( $nodes as node()* ) as xs:string* { for $node in $nodes return( string-join( if ($node/@*) then (string-join((name($node) || string-join(for $att in $node/@* return local:atty($att))), "/"), local:elem($node/child::*)) else if ($node/child::*) then (for $child in $node/child::* return local:elem($child), local:elem($node/child::*)) else (name($node) || "/" || local:elem($node/child::*)) ) ) }; declare function local:atty( $att as attribute() ) as xs:string* { "[@" || name($att) || "='" || data($att) || "']" }; local:e2($input (: this currently returns test/aaa/bbb[@type='foo']/ / /aaa/bbb[@type='foo'][@enc='bar']/ / /aaa/bbb[@type='bzz'][@enc='bar']/ /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='start']/ /bbb[@type='qux'][@enc='bar'][@key='yes'][@point='end']/ / :) ``` [1] fn:path, and the related functions from functx (functx:distinct-element-paths, functx:path-to-node, and functx:path-to-node-with-pos). The functx functions are really close and awesome, but I need to incorporate attributes into my output. [2] https://gist.github.com/CanOfBees/8cfb435ac06986c9b0b0c215a786f4d7

2 3

Dpi scaling problem - Ubuntu 20.04
by Silamphre 15 Oct '20

15 Oct '20

Hello everyone, I started using BaseX for a class project a few days ago. I'd like to run it on my Ubuntu 20.04. I've installed openjdk version 14 to do so. I've set my Ubuntu display settings at 125% fractional scaling, but it appears that BaseX does not scale with this setting. The GUI consequently appears really tiny, almost unusable. Does someone have a way to solve this issue, and make BaseX scales accordingly to my Ubuntu display settings, please? Thank you for your help. Kind regards, Hugo <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Garanti sans virus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 1

Re: [basex-talk] stemming chinese texts
by Philippe Pons 14 Oct '20

14 Oct '20

Hi Christian, I suppose some of my colleagues would be able to judge the quality of your full-text search results. On the other hand, on code level, I'm not sure I know how to implement an additionnal class that extends abstract Tokenizer class. Thank you for your help Philippe Le 14/10/2020 à 11:00, Christian Grün a écrit : > Hi Philippe, > > Thanks for your mail in private, in which I already gave you a little > assessment on what might be necessary to include the CJK tokenizers in > BaseX: > > The existing Apache code can be adapted and embedded into the BaseX > tokenizer infrastructure. On code level, an additional class needs to > be implemented that extends abstract Tokenizer class [1]. > > As far as I can judge, the 3 Lucene CJK analyzers could all be applied > to traditional and simplified Chinese. If we found someone who could > rate the linguistic quality of our full-text search results, that’d > surely be helpful. > > Hope this helps, > Christian > > [1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/main/java/org/b… > > > > On Tue, Oct 13, 2020 at 12:32 PM Philippe Pons > <philippe.pons(a)college-de-france.fr> wrote: >> Dear Christian, >> >> Thank you very much for this quick and enlightening response. >> >> Without having had (yet) the opportunity to test it, I have indeed read the Japanese text tokenizer. >> Supporting Chinese tokenization would also be a great help. >> >> I have never tested what Lucene offers, especially since I have to manage texts in traditional Chinese and simplified Chinese (without reading either one myself). >> I would like to test Lucene's analyzers, but I don't know how to do it in BaseX? >> >> Best regards, >> Philippe Pons >> >> >> >> Le 12/10/2020 à 12:01, Christian Grün a écrit : >> >> Dear Philippe, >> >> As the Chinese language rarely uses inflection, there is usually no >> need to perform stemming on texts. However, tokenization will be >> necessary indeed. Right now, BaseX provides no tokenizer/analyzer for >> Chinese texts. It should be possible indeed to adopt code from Lucene, >> as we’ve already done for other languages (our software licenses allow >> that). >> >> Have you already worked with tokenization of Chinese texts in Lucene? >> If yes, which of the 3 available analyzers [1] have proven to yield >> the best results? >> >> As you may know, one of our users, Toshio HIRAI, has contributed a >> tokenizer for Japanes texts in the past [2]. If we decide to include >> support for Chinese tokenization, it might as well be interesting to >> compare the results of the Apache tokenizer with our internal >> tokenizer. >> >> Cordiales salutations, >> Christian >> >> [1] https://lucene.apache.org/core/7_2_0/analyzers-common/org/apache/lucene/ana… >> [2] https://docs.basex.org/wiki/Full-Text:_Japanese >> >> >> >> On Mon, Oct 12, 2020 at 11:37 AM Philippe Pons >> <philippe.pons(a)college-de-france.fr> wrote: >> >> Dear BaseX Team, >> >> I'm actually working on chinese texts in TEI. >> I would like to know if stemming chinese text is possible in BaseX, as we can do with other languages (like english or deutsch)? >> Or maybe there is a way to add this functionnality with Lucene? >> >> Best regards, >> Philippe Pons >> >> -- >> Ingénieur d'étude chargé de l'édition de corpus numériques >> Centre de recherche sur les civilisations de l'Asie Orientale >> CRCAO - UMR 8155 (Collège de France, EPHE, CNRS, PSL Research University, Univ Paris Diderot, Sorbonne Paris Cité) >> 49bis avenue de la Belle Gabrielle >> 75012 Paris >> https://cv.archives-ouvertes.fr/ppons >> >>

1 0

stemming chinese texts
by Philippe Pons 14 Oct '20

14 Oct '20

Dear BaseX Team, I'm actually working on chinese texts in TEI. I would like to know if stemming chinese text is possible in BaseX, as we can do with other languages (like english or deutsch)? Or maybe there is a way to add this functionnality with Lucene? Best regards, Philippe Pons -- Ingénieur d'étude chargé de l'édition de corpus numériques Centre de recherche sur les civilisations de l'Asie Orientale CRCAO - UMR 8155 (Collège de France, EPHE, CNRS, PSL Research University, Univ Paris Diderot, Sorbonne Paris Cité) 49bis avenue de la Belle Gabrielle 75012 Paris https://cv.archives-ouvertes.fr/ppons

2 3

Re: [basex-talk] BaseX-Talk Digest, Vol 130, Issue 5
by Duncan Paterson 12 Oct '20

12 Oct '20

> I'm actually working on chinese texts in TEI. > I would like to know if stemming chinese text is possible in BaseX, as > we can do with other languages (like english or deutsch)? > Or maybe there is a way to add this functionnality with Lucene? > > Best regards, > Philippe Pons > Dear Philippe, if by stemming you mean the removal of prefixes and suffixes to arrive at normalized word stems, the concept simply doesn’t apply to Chinese, so no it can’t be done. What you are most likely looking for is the ability to tokenize strings into n-grams, which lucene can do. https://lucene.apache.org/core/6_6_1/analyzers-common/org/apache/lucene/ana… <https://lucene.apache.org/core/6_6_1/analyzers-common/org/apache/lucene/ana…> Greetings Duncan

1 0

XPTY0004 error when calling http:send-request inside of a local function
by Bridger Dyson-Smith 11 Oct '20

11 Oct '20

Hi all - I'm accustomed to writing something like the following: ```xquery declare function local:grab( $url as xs:string, $path as xs:string, $fname as xs:string ) as document-node() { let $req := http:send-request(<http:request method="get"/>, $url) return if (head($req)/@status = "200") then file:write($path || $fname, tail($req), map { "method": "xml" }) else ("something went wrong!") }; ``` but I've noticed that in version 9.4.3 this returns an XPTY error: Stopped at /home/bridger/bin/basex-src/basex-943-xpty0004-error.xq, 1/18: [XPTY0004] Cannot promote xs:string* to document-node(): (let $req_7 := http:send-request(<http:request method="get"/>, " https://dpla.lib.utk.edu/repox/OAIHa.... While this could be a bug, should I be writing this kind of local function differently? Or could I improve my typing on it? Thanks very much for any advice or insights you can provide. Best, Bridger

2 3

XQuery Optimizer Bug?
by Paul L. Merchant Jr. 07 Oct '20

07 Oct '20

Greetings everyone, I'm evaluating whether BaseX 9.4.3 might be suitable for my projects and I think I've run into an XQuery optimizer bug that's easily reproducible. I've got a test database "test-db" with two identical documents containing minimal xml: "<doc title='a-document'/>". The documents are stored in separate directories: test-db |- a/doc.xml |- b/doc.xml The XQuery I'm using I believe should retrieve only the document in path a, but I'm getting different results depending on whether I write the expression in a single let statement or two separate statements: declare function local:show-paths($nodes as node()*) as node()* { for $p in $nodes return <path>{$p/base-uri()}</path> }; let $docs_1 := collection('test-db/a')/doc[@title='a-document'] let $col := collection('test-db/a') let $docs_2 := $col/doc[@title='a-document'] return <result> <single-expr expr_docs="{count($docs_1)}"/> <single-expr-paths>{local:show-paths($docs_1)}</single-expr-paths> <split-expr collection_docs="{count($col)}" expr_docs="{count($docs_2)}"/> <split-expr-paths>{local:show-paths($docs_2)}</split-expr-paths> </result> This code produces this result: <result> <single-expr expr_docs="1"/> <single-expr-paths> <path>/test-db/a/doc.xml</path> </single-expr-paths> <split-expr collection_docs="1" expr_docs="2"/> <split-expr-paths> <path>/test-db/a/doc.xml</path> <path>/test-db/b/doc.xml</path> </split-expr-paths> </result> I'd expect the expr_docs values for both single-expr and split-expr to be the same. I'm happy to file this in the GitHub issue tracker, but as a newcomer to BaseX I thought I should checkin with the list first. What are you thoughts and wishes? Thanks, -- Paul

2 1

king Henry VIII dalenda est
by Liam R. E. Quin 05 Oct '20

05 Oct '20

At https://words.fromoldbooks.org/Search/ a search for henry shows lots of matches, and sodoes a search for henry i, but henry ii and henry viis missing and so is henry viii. I can search for viii and find Henry VIII and also Charles VIII, but i also can't search for Charles VIII. I can search for the king’s feet, and for Henry V, but not Charles II. It looks like words ending in ii are invisible. I'm using ft:search("wobo", $term)/ancestor-or-self::p Might this be related to stemming? i have that turned off This is used to create the db (from the Perl API) # create query instance t("drop db wobo"); t("create db wobo"); t("open wobo"); $session->send("set chop false"); $session->send("set ftindex true"); $session->send("set updindex true"); $session->send("set autooptimize true"); txf("/home/liam/w/Search/wobo.xml"); t("create index attribute"); t("create index text"); t("create index fulltext"); t("optimize"); t("close"); t("quit"); where t() just does a $session->execute() on its argument after printing a trace line, and txf does a delete followed by an add. Probably i can make a test case available if needed. Liam -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org

3 4

Mapping query-params to $options argement of ft:functionj
by Günter Dunz-Wolff 05 Oct '20

05 Oct '20

Hi all, is there a possibility to match the query-parameters more directly to the $options argument of a ft:function? In the moment I have a rather complicated if-else-if-else-if…-scenario. Example: if ($wildcards = "on" and $all = "on" and $case_sensitive = "on" and $using_diacritics = "on") then $collection[.//text() contains text {$query} all words using case sensitive using diacritics sensitive using wildcards] else if and so on. Depending on query-params I want to build the ft:function with $options argument more directly. (But for example string-concatenation doesn’t work.) Thanks for any advice. Guenter

2 3

Re: [basex-talk] Paging results of an XQuery-Search
by Günter Dunz-Wolff 27 Sep '20

27 Sep '20

Thanks Gerrit, invoking search and passing all params per GET is the way to go.

2 1