BaseX-Talk May 2018

basex-talk@mailman.uni-konstanz.de

28 participants
38 discussions

Global locks
by Andy Bunce 11 Feb '19

11 Feb '19

Running the following in the GUI (8.6.5) ``` let $db:="BEP" return count(collection($db)) ``` The info windows shows $db is inlined Compiling: - inline $db_0 - pre-evaluate collection("BEP") to document-node() sequence - pre-evaluate count((db:open-pre("BEP",0), ...)) to xs:integer - simplify gflwor Optimized Query: 9193 All good. Submitting a job ``` let $q:='let $db:="BEP" return count(collection($db))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` returns: <job id="job90" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.007S" reads="(global)" writes="(none)">let $db:="BEP" return count(collection($db))</job> Note @reads="(global)" But ``` let $q:='count(collection("BEP"))' return jobs:eval($q,(),map{"cache":true()})!(jobs:wait(.),jobs:list-details(.)) ``` <job id="job92" type="ScheduledXQuery" state="cached" user="admin" duration="PT0.006S" reads="BEP" writes="(none)">count(collection("BEP"))</job> Note @reads="BEP" Is this correct and/or of any consequence? /Andy

2 8

BaseX 9.0: The Spring Edition!
by Christian Grün 01 Nov '18

01 Nov '18

Dear BaseX aficionados, We are very happy to announce the release of BaseX 9.0! The new version of our XML database system and XQuery 3.1 processor includes some great new features and a vast number of minor improvements and optimizations. It’s both the usage of BaseX in productive environments as well as the valuable feedback of our open source users that make BaseX better and better, and that allow and motivate us to keep going. Thanks to all of you! Along with the new release, we invite you to visit our relaunched homepage: http://basex.org/. Java 8 is now required to run BaseX. The most prominent features of Version 9.0 are: XQUERY - Comprehensive extensions in the internal XQuery optimizer framework - Dynamic hash joins rewritings of general comparisons - Register query jobs as persistent services - Process very large CSV files via the new CSV 'xquery' format - Update Module: higher-order functions for performing updates - Unified static typing, including maps, arrays and function items - COPYNODE: Lightweight copying of XML fragments - ENFORCEINDEX: Enforce rewriting for index access STORAGE - Improved database compression (short strings, whitespaces) WEB APPLICATIONS - Permissions layer: Unified definition of RESTXQ access patterns - RESTXQ: Full support for client- and server-side quality factors - REST: Run BaseX command scripts - Web server upgrade to Jetty 9 REPOSITORIES - Combined packaging mechanism (XQuery and Java) - Java Packaging: Bundling of additional libraries DISTRIBUTIONS - Revised detection and configuration of BaseX home directory - Windows executable: better detection of Java libraries DBA: BASEX DATABASE ADMINISTRATION - Improved editing and evaluation of XQuery modules - Revised file, session and job management More details can be found on our homepage. We are looking forward to your feedback. Enjoy and have fun, Christian, BaseX Team

7 8

Insert CSV into database
by Ben Engbers 01 Jun '18

01 Jun '18

Hi, My goals is to use my R clientdriver to insert csv-files into a new databases. But before that, I'm experimenting with the GUI. >From the documentation for the CSV-parser, I have taken this code: for $file in file:list("/home/bengbers/DataScience/RBaseX/Examples/Parse", false(), "*.csv") return db:add("CSV_test", $file, "", map { 'parser': 'csv', 'csvparser': map { 'header': 'yes', 'separator': ';' } }) BaseX returns: Error: Stopped at /home/bengbers/DataScience/Eindopdracht/Data/file, 2/14: [FODC0002] Resource 'Test_Parse.csv' does not exist. I didn't enter this path. It was used yesterday when browsing to the datafiles that were inserted into another test-database. let $file := file:list("/home/bengbers/DataScience/RBaseX/Examples/Parse", false(), "*.csv") return $file BaseX returns: Test_Parse.csv Test_Parse (exemplaar).csv If I create a new database, it neatly adds the two csv-files. My questions are: - which query I have to use to insert csv-files? - obviously, BasexGUI uses the wrong path. How should I adjust this path? Cheers, Ben

2 3

BaseX 9.0.2: New Maintenance Release
by Christian Grün 31 May '18

31 May '18

Hi all, A new BaseX maintenance release is available! Please visit http://basex.org to download the latest version. These are the changes: Critical bug fixes: - XQuery: Node ordering of mixed database/fragment instances - XQuery: Index rewritings of paths with nested positional predicates Improvements: - WebDAV: Much faster locking of database resources - XQuery: Enable ENFORCEINDEX for variable predicate arguments - GUI: Larger interaction components, unified font sizes Minor Bug fixes: - DBA: Suppress menu on login page - Commands: Nested evaluation with variable bindings - XQuery: Static typing of lookup expressions Enjoy, Christian

1 0

Serialize csv xquery map
by Andy Bunce 29 May '18

29 May '18

The xqueryformat of CSV records as a sequence of arrays [1] is very useful when parsing but there seems to be no easy way to serialize from this format. Is this correct? map { "records": ["Huber", "Sepp", "Hauptstraße 13", "93547 Hintertupfing"], "names": ["Name", "First_Name", "Address", "City"] } =>csv:serialize() [SENR0001] Items of type map(xs:string, array(xs:string)) cannot be serialized. ... would be nice if it could. /Andy [1] http://docs.basex.org/wiki/CSV_Module#XQuery

2 2

IETF Date in If-Modified-Since header
by Marco Lettere 29 May '18

29 May '18

Hello all, need to use the If-Modified-Since header in one of my APIs. The format requested is the IETF date format: If-Modified-Since: Wed, 21 Oct 2015 07:28:00 GMT as explained in [1]. If I pass the header as prescribed to a restxq made like: declare %rest:path("config") %rest:header-param("If-Modified-Since", "{$modifiedsince-ietf}") %rest:HEAD function config:check($modifiedsince-ietf as xs:string?){ I get the following exception: Cannot convert xs:string+ to xs:string?: ("Wed", "21 Oct 2015 07:28:00 GMT"). Which somehow demonstrates that , is treated as a separator of values for the header. I patched by replacing ? with * and using a string-join on the $modifiedsince-ietf before passing it to the follwoing parse-ietf-date function. But this is hard to explain and probably subject to other issues in the future. Is there a way to pass HTTP headers verbatim to the RestXQ function? thanks! M. [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since

2 2

no value assigned to external variables in xq when running bxs->bxs->xq
by Freihalter Thomas 29 May '18

29 May '18

Hello, I am using basex 9.0.1 and have this 3 files: test.xq: declare variable $test external; $test test.bxs: <commands> <set option="BINDINGS">test=foo</set> <run file="test.xq"/> </commands> run.bxs: <commands> <run file="test.bxs"/> </commands> Running test.bxs is okay basex -v test.bxs BINDINGS: test=foo foo Query "test.xq" executed in 5.4 ms. basex -v run.bxs BINDINGS: test=foo Stopped at test.xq, 1/18: [XPDY0002] No value assigned to $test. It look like the bindings in test.bxs get lost, when executing the superior run.bxs. Is there any solution solving this problem? 8.6.7 does not work as well. regard Thomas echo "declare variable \$test external; \$test" > test.xq; echo "<commands><set option=\"BINDINGS\">test=foo</set><run file=\"test.xq\"/></commands>" > test.bxs; echo "<commands><run file=\"test.bxs\"/></commands>" > run.bxs; basex -v test.bxs; basex -v run.bxs

2 2

progress of query execution
by Ветошкин Владимир 29 May '18

29 May '18

Hi, everybody! My core has written in php and php-script execute xquery-scripts to get needed data. To get the progress of one large query I have to divide it into some small ones. Isn't it? Are there any more options? Maybe it is possible to get number of seconds, which needed to complete execution of query? -- С уважением, Ветошкин Владимир Владимирович

2 1

Controlling json serialization of JSON-XML
by Hondros, Constantine (ELS-AMS) 29 May '18

29 May '18

Hello all, I'm expecting the answer to be no, but is it possible to influence the JSON serialization of <number> elements at all? xml-to-json( <map xmlns="http://www.w3.org/2005/xpath-functions"> <number key="number">10111234</number> </map> ) Results in {"number":1.0111234E7} I read this in the spec, so it looks like the number is always being cast as a double: xs:string(xs:double(fn:string($E))) TIA, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Fwd: BaseX performance improvements
by Christian Grün 28 May '18

28 May '18

Hi Martin (cc to the list), > 1. each XML node => 500 bytes > 2. total size of XML DB => 10million nodes(which will grow continuously as > new files are added). So one node is one document, right? I’m just asking because, in the XML terminology, each XML document has nodes itself: element nodes, attributes nodes, text nodes, etc. If you are working with millions of documents, it can be helpful to work with a "daily" database, which contains auxiliary references on documents that have been deleted or updated. If there is a time interval in which no people access your database (e.g. each night), you can merge these databases and recreate your index structures. If you query your database, you can a few more lines of XQuery: first check your daily database, and (if it doesn’t contain the required document) look up the document in your fully indexed database. > I have to do a update of the nodes as I need to replace them. I chose delete > + insert instead of replace Please note that a replace operation can be much faster: If the structure of the updated document is similar to the old document, the document can be replace in-place. > Also, I am thinking of indexing on the node id, as that is like the primary > key on which I base my operations, any suggestions might also be helpful, Here it may also be interesting to know if you will only address documents or arbitrary nodes of your document. In the first case, addressing the document path via db:open should be sufficient [1]. In the latter case, you could possibly use the existing function db:node-id [2]. Hope this helps, Christian [1] http://docs.basex.org/wiki/Databases [2] http://docs.basex.org/wiki/Database_Module#db:node-id > Thanks again, > Regards > Martin Lourduswamy > > On Sun, May 20, 2018 at 9:28 AM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Martin, >> >> > I am new to BaseX, I would like ot speed up XQuery of insert and delete >> > and >> > replace through options. >> >> Welcome to the list. As there are numerous ways to do updates in >> BaseX, feel free to give us more information on your insert and delete >> operations. Do you work with large single documents or many small >> documents? How large is your database? >> >> > While I query basex through perl, and try to >> > connect through GUI, the perl connection aborts. Is there a parameters >> > for >> > parallel connections, please let me know. >> >> I can’t tell why your perl connection is interrupted by opening the >> GUI (because they should be completely independent from each other). A >> step-by-step description on how you proceeded might be helpful. >> >> Because there is no coupling between GUI and the client/server >> architecture, however, you should avoid running updates outside the >> GUI. Please check out [1] for more information. >> >> Best, >> Christian >> >> [1] http://docs.basex.org/wiki/Startup#Concurrent_Operations >> >> >> >> > >> > Thanks, >> > Regards >> > Martin Lourduswamy > >

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk May 2018