August 2018 - BaseX-Talk - mailman.uni-konstanz.de

Performance (index?) on large intermediate structures
by Sebastian Albert 29 Aug '18

29 Aug '18

Dear BaseX community I am currently working with BaseX to write a rather long XQuery statement with quite large intermediate structures (XML documents / fragments) bound to variables (in FLWOR expressions), which I query by XPath using attributes etc. further down. Having nested loops, this is becoming very slow. Are those intermediate structures automatically indexed? Or how can I achieve indexing on them? Or do I need to write them into a database for this indexing? (Can I specify a datbase as "in-memory"?) Then I guess I need a "BaseX script" rather than a single XQuery? Of course, I am looking for the most elegant/idiomatic way. Or is it best to build index-like intermediate structures myself, probably using maps rather than XML documents? Furthermore, are there profiling tools inside BaseX, or would I need to fire up a Java IDE to see where most time is being spent? Thanks in advance for any suggestions! Sebastian

2 1

[XQUERY] Filter and paging with window clause
by Marco Lettere 28 Aug '18

28 Aug '18

Hi all, here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. But I was wondering whether it's possible to streamline this into one single FLOWR. Two bonus questions: 1. Will this actually be more efficient than the naive (two FLOWR) solution or is the almighty optimizer able to blend the two subsequent FLOWR on its own? 2. What if the input is unordered and I also would like to introduce sorting? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ())

3 7

Strange index values with numerics
by Zachary N. Dean 28 Aug '18

28 Aug '18

Hi, I was recently taking a look into the index data files (for reasons) and came across something that I found strange... When numeric values are in nodes they are put into the index with min/max and distinct token values, which is cool. What's strange is, when negative integer values are in text and attribute nodes the index contains the minimum value correctly, but the maximum value is '4.9E-324' ([0,0,0,0,0,0,0,1]). This doesn't seem to happen with positive values. Now, with small value ranges I assume this is okay, but with many values I would imagine it could slow things down. Not sure if this is a bug or a feature, so I figured I'd bring it up. Here an example: <r> <a>-1</a> <b>0</b> <c>1</c> <d>-50000</d> <d>-49000</d> <e>2</e> <e>3</e> <f a="-1"/> <g>-1</g> <g>1</g> </r> I would have assumed that the index would see that element "d" has a min of -50000 and a max of -49000. Here the index infos: Elements - Structure: Hash - Entries: 8 g 2x, 2 distinct integers [-1, 1], leaf e 2x, 2 distinct integers [2, 3], leaf d 2x, 2 distinct integers [-50000, 4.9E-324], leaf c 1x, integer [1, 1], leaf f 1x, leaf r 1x a 1x, integer [-1, 4.9E-324], leaf b 1x, integer [0, 4.9E-324], leaf Attributes - Structure: Hash - Entries: 1 a 1x, integer [-1, 4.9E-324], leaf If it's a feature, then cool. Keep on rockin'! If not, then I hope this helps a little. Thanks, Zack Dean

3 2

Update functions - Error handling and counting
by Bogdan Bogucki 26 Aug '18

26 Aug '18

Hello, I am playing with XQUF in BaseX ann I encountered following problems. 1) XQuery is functional language and updating function can return empty sequence. How can I count and return result how many nodes was inserted or updated ? Example: declare %private %updating function cfv:upsertVehicles ($vehicles as node()*) as empty-sequence() { let $db := db:open("db")/cars for $v in $vehicles let $dbVehicle := $db/car[vin = $v/vin] return if (not(exists($dbVehicle))) then (insert node $v as last into $db) else if (exists($dbVehicle)) then (replace node $dbVehicle with $v) else () }; I have similar problem with error handling. How can I catch and return any error. Regards Bogdan Bogucki

2 5

archive
by Ветошкин Владимир 23 Aug '18

23 Aug '18

4 7

Forcing reading _only_ from standard input
by Richard Walker 22 Aug '18

22 Aug '18

Hi. I have a script in which I process data from standard input like this: VOCAB_WITH_NEW_VOCAB_NOTE="$(echo "$VOCAB" | "${BASEX}" -i - -q $" copy \$c := . modify ... etc. etc. The script has been working fine. However, due to a defect in an earlier line in the script, during one test run, the value of $VOCAB was (incorrectly) an empty string. But in this case, I got a quite unexpected result from the above invocation of BaseX. I can easily repeat the same phenomenon out of the script's context thus: $ echo '' | basex -i - -q '.' <vocabulary xmlns="http://vocabs.ands.org.au/registry/schema/2017/01/vocabulary" ... etc. etc. In this case, rather than processing the empty string provided as standard input, BaseX fetched data from one of my databases. Can I/how do I force the option "-i -" to mean "read data from standard input, and _only_ from standard input, even if that is empty"? -- Richard Walker Australian Research Data Commons (ARDC) (formerly, Australian National Data Service, ANDS) 101 Liversidge Street The Australian National University Acton ACT 2601, Australia E: Richard.Walker(a)ardc.edu.au T: +61 2 6125 0584 W: ardc.edu.au W: www.ands.org.au

2 4

basex 9.1 and websockets
by Maximilian Gärber 22 Aug '18

22 Aug '18

Hi, how far are we from the wss enabled release? I am ready for beta testing ;-) Br, Max

2 1

XML Merging while reading from the DB
by DK Singh 18 Aug '18

18 Aug '18

Hi All, I have ingested the XML into DB having node "include" and within "include" some file referenced, when i read the file from the DB it merges the each XML , is there any way to disabled include functionality while reading the file. Input ========================== <book xmlns="http://docbook.org/ns/docbook" xmlns:xlink=" http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0-extension bp-0.1" role="fullText" xml:id="b-9781845928315" xml:lang="en" outputformat='e-Only'> <xi:include href="test1.xml"/> <xi:include href="test2.xml"/> </book> i am reading the file using fn:doc() or db:open() both the function Merging included files. Note: i don't have to set the option on the DB label to disable xinclude . Regards Dharmendra Kumar Sing

1 0

archive:create
by Ветошкин Владимир 17 Aug '18

17 Aug '18

2 2

XML Files merging while reading from the DB
by DK Singh 17 Aug '18

17 Aug '18

Hi All, I have ingested the XML into DB having node "include" and within "include" some file referenced, when i read the file from the DB it merges the each XML , is there any way to disabled include functionality while reading the file. Input ========================== <book xmlns="http://docbook.org/ns/docbook" xmlns:xlink=" http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0-extension bp-0.1" role="fullText" xml:id="b-9781845928315" xml:lang="en" outputformat='e-Only'> <xi:include href="test1.xml"/> <xi:include href="test2.xml"/> </book> i am reading the file using fn:doc() or db:open() both the function Merging included files. Note: i don't have to set the option on the DB label to disable xinclude . Regards Dharmendra Kumar Singh

1 0