July 2018 - BaseX-Talk - mailman.uni-konstanz.de

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

BaseX 9.0: The Spring Edition!
by Christian Grün 01 Nov '18

01 Nov '18

Dear BaseX aficionados, We are very happy to announce the release of BaseX 9.0! The new version of our XML database system and XQuery 3.1 processor includes some great new features and a vast number of minor improvements and optimizations. It’s both the usage of BaseX in productive environments as well as the valuable feedback of our open source users that make BaseX better and better, and that allow and motivate us to keep going. Thanks to all of you! Along with the new release, we invite you to visit our relaunched homepage: http://basex.org/. Java 8 is now required to run BaseX. The most prominent features of Version 9.0 are: XQUERY - Comprehensive extensions in the internal XQuery optimizer framework - Dynamic hash joins rewritings of general comparisons - Register query jobs as persistent services - Process very large CSV files via the new CSV 'xquery' format - Update Module: higher-order functions for performing updates - Unified static typing, including maps, arrays and function items - COPYNODE: Lightweight copying of XML fragments - ENFORCEINDEX: Enforce rewriting for index access STORAGE - Improved database compression (short strings, whitespaces) WEB APPLICATIONS - Permissions layer: Unified definition of RESTXQ access patterns - RESTXQ: Full support for client- and server-side quality factors - REST: Run BaseX command scripts - Web server upgrade to Jetty 9 REPOSITORIES - Combined packaging mechanism (XQuery and Java) - Java Packaging: Bundling of additional libraries DISTRIBUTIONS - Revised detection and configuration of BaseX home directory - Windows executable: better detection of Java libraries DBA: BASEX DATABASE ADMINISTRATION - Improved editing and evaluation of XQuery modules - Revised file, session and job management More details can be found on our homepage. We are looking forward to your feedback. Enjoy and have fun, Christian, BaseX Team

7 8

Wasn't there a function, that would walk a website?
by Andreas Mixich 01 Aug '18

01 Aug '18

Hi, somehow it deems to me, that I once saw a function in BaseX extensive module library, that would recursively read a web-page from remote, following all the links in it. But now I can't find it. Am I dreaming? -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

3 6

about xml:base when using multiple nested xi:include
by Marco Randazzo 30 Jul '18

30 Jul '18

Dear all, My example (in attachment) employs xi:include elements to include other xml files. The example is constituted by a main.xml file which includes another file (include1.xml) stored in a subfolder, called include1. include1.xml, in turn, includes another xml file (include2.xml) stored in a subfolder of include1 which is called include2. Overall, the example consists of four folder levels and only the last file, include4.xml, contains data. When I import main.xml in baseX, I obtain the following result: <main xmlns:xi="http://www.w3.org/2001/XInclude"> <level1 xml:base="./include1/include1.xml"> <level2 xml:base="./include1/include2/include2.xml"> <level3 xml:base="./include2/include3/include3.xml"> <level4 xml:base="./include3/include4/include4.xml"> <data>0</data> <data>1</data> </level4> </level3> </level2> </level1> </main> The xml tree is correct, but I have doubts about the value of the attribute xml:base. In particular: - level1: I think that the value of xml:base ( i.e. ./include1/include1.xml) is correct. - level2: xml:base value is correct if its meaning is a path relative to the root file. Otherwise, if xml:base is supposed to store a path relative to the current file, then it should be ./include2/include2.xml. - level3: This seems to be not correct, neither as an absolute path, nor as a relative path. I think it should be ./include1/include2/include3/include3.xml as a path relative to the root file or ./include3/include3.xml as a path relative to the current file. - level4: Same as level3. What do you think? Am I missing something? What are the specifications for the value of xml:base? Thank you, Marco Randazzo

2 3

how to prevent org.basex.core.jobs.JobException error
by radim-havlicek＠post.cz 30 Jul '18

30 Jul '18

Hello, can I extend some value in BaseX to prevent org.basex.core.jobs.JobException error? Best regards, Radim Havlicek

2 1

Messy result with TagSoup
by Andreas Mixich 29 Jul '18

29 Jul '18

Hi, If I use `fetch:xml($url, map{'parser':'html'})` all is fine! The next one gives a correct result (although, in contrary to the browser, without namespaces and doctype): let $url := "https://bartoszmilewski.com/2014/10/28/category-theory-for-programmers-the-…" let $response := http:send-request(<http:request method='get'/>, $url) return $response[2] This creates a mess: let $url := "https://bartoszmilewski.com/2014/10/28/category-theory-for-programmers-the-…" let $response := http:send-request(<http:request method='get'/>, $url) return html:parse($response[2]) gives (partial): <html> <body>Category Theory for Programmers: The Preface | Bartosz Milewski's Programming Cafe/* */ if ( 'function' === typeof WPRemoteLogin ) { document.cookie = "wordpress_test_cookie=test; path=/"; if ( document.cookie.match( /(;|^)\s*wordpress_test_cookie\=/ ) ) { WPRemoteLogin(); } etc. I also tried: let $url := "https://bartoszmilewski.com/2014/10/28/category-theory-for-programmers-the-…" let $response := http:send-request(<http:request method='get'/>, $url) return html:parse($response[2], map{ 'html': false(), 'lexical': true(), 'nocdata': true(), 'nodefaults': true(), 'nons': false() }) This adds only the xhtml namespace, but the rest is the same. html:parser() -> "TagSoup" I don't know enough about Java, to test which TagSoup version is in use via the GUI. I am getting this result, when using the GUI on Windows 10 with BaseX 9.0.1 As I have seen, there was a thread on this list in 2016 about eventual replacement of TagSoup: https://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg08928.ht… What about https://about.validator.nu/htmlparser/ ? Thank you. -- Goody Bye, Minden jót, Mit freundlichen Grüßen, Andreas Mixich

2 3

Is it possible to set the root for resolving include and import in xslt:transform
by Omar Siam 28 Jul '18

28 Jul '18

Hi list! I have an XSL snippet that includes and imports some XSL files. I pass this to xslt:transform and the includes and imports are resolved using the working directory where basex was started from. On my server that was / and on my Windows development machine C:\Users\user\basex\bin. This was not what I expected. Quick way to see the effect: declare namespace xsl="http://www.w3.org/1999/XSL/Transform"; xslt:transform(<root><test/></root>, <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> <xsl:template match="test"><root><xsl:value-of select="static-base-uri()"/></root></xsl:template> </xsl:stylesheet>) Perhaps in a future version it may be possible to set that to the directory the XQuery runs from or to be able to just specify what it should be. Best regards Omar Siam

3 2

More authentication options?
by Peter Villadsen 27 Jul '18

27 Jul '18

I was previously using simple basic authentication, which will not pass muster within our organization. I therefore was happy to go to Digest authentication, which seems to have solved my problems for now. However, I was wondering if there will be an option for more advanced authentication scenarios, like certificates or possibly OAuth? Microsoft's SQL server SQL server has the concept of "windows authentication" where the windows security model is used (users who are already signed into windows do not have to present further creds). I don't know how that DB works when it runs on Linux though. Best Regards Peter Villadsen Principal Architect Microsoft Business Applications Group

1 0

Curious query optimization
by Sebastian Zimmer 27 Jul '18

27 Jul '18

Hi, I have a query which is optimized in a curious way in BaseX 9.0.2 (yesterday's snapshot). This is the original query: xquery version "3.1"; declare namespace tei ="http://www.tei-c.org/ns/1.0"; let $string := "string" let $fuzzy := false() return ( collection('ZK')/tei:TEI[ if (false()) then (.[descendant::text() contains text {$string} using fuzzy]) else (.[descendant::text() contains text {$string}]) ], collection('ZK')/tei:TEI[ if ($fuzzy) then (.[descendant::text() contains text {$string} using fuzzy]) else (.[descendant::text() contains text {$string}]) ] ) And this is the optimized one (newlines inserted by me for better readability): ( ft:search("ZK", "string" using language 'English')/ancestor::tei:TEI[parent::document-node()], ft:search("ZK", "string" using fuzzy using language 'English')/ancestor::tei:TEI[parent::document-node()] ) I'm curious why the second search is using fuzzy, even though the variable $fuzzy is false. I presume that query optimization is independent of the data, so you won't need the data to reproduce. But if you do, I can provide it. A database with enabled full-text index is required obviously. Best regards, Sebastian Zimmer -- Sebastian Zimmer sebastian.zimmer(a)uni-koeln.de <mailto:sebastian.zimmer@uni-koeln.de> Cologne Center for eHumanities <http://cceh.uni-koeln.de> DH Center at the University of Cologne @CCeHum <https://twitter.com/CCeHum>

2 10