- BaseX-Talk - mailman.uni-konstanz.de

Bug (?) in fn:generate-id
by Hans-Juergen Rennau 31 Aug '18

31 Aug '18

Dear BaseX team, I face an issue with fn:generate-id. Given: a document $doc parsed from the file system.Nodes obtained by xquery_eval(EXPR, map{'':$doc}) and nodes obtained by $doc/EXPR have the same identity (per $node1 is $node2), but the values generated by fn:generate-id differ. Interestingly, if the document is constructed within the query, rather than read from a file, the values generated by fn:generate-id do not differ. For a test see below. At first sight, the issue may appear too obscure to justify efforts to remove it. HOWEVER, there is an important use case involved: (a) expressions are read from a configuration and evaluated via xquery:eval; a map is constructed which associates for each node in the expression value its node id with other information (e.g. which of the configured expressions yielded this node) ;(b) then, when later nodes are visited via XPath, that other information should be retrieved (lookup in the map, using generate-id($node) as key) - AND THIS FAILS, as the XPath-provided nodes have ids which are different from the xquery:eval-provided nodes. Could you please have a look? With kind regards,Hans-Jürgen Test query:======== let $doc1 := <root><foo/><bar/></root> let $doc2 := doc('foobar.xml')/* let $evalNodes1 := xquery:eval('*', map{'': $doc1}) let $evalNodes2 := xquery:eval('*', map{'': $doc2}) let $xpathNodes1 := $doc1/* let $xpathNodes2 := $doc2/* let $evalIds1 := $evalNodes1/concat(local-name(.), ' id=', generate-id(.)) let $xpathIds1 := $xpathNodes1/concat(local-name(.), ' id=', generate-id(.)) let $evalIds2 := $evalNodes2/concat(local-name(.), ' id=', generate-id(.)) let $xpathIds2 := $xpathNodes2/concat(local-name(.), ' id=', generate-id(.)) return ( 'local doc - evalNodes1 = xpathNodes1 ?', for $i in 1 to 2 return $evalNodes1[$i] is $xpathNodes1[$i], '', 'local doc - evalNodes, ids:', $evalIds1, 'local doc - xpathNodes, ids:', $xpathIds1, '', 'file doc - evalNodes1 = xpathNodes1 ?', for $i in 1 to 2 return $evalNodes2[$i] is $xpathNodes2[$i], '', 'file doc - evalNodes, ids:', $evalIds2, 'file doc - xpathNodes, ids:', $xpathIds2) Document 'foobar.xml':================= <root><foo/><bar/></root> OUTPUT:======= local doc - evalNodes1 = xpathNodes1 ? true # OK! true # OK! local doc - evalNodes, ids: foo id=id10 bar id=id12 local doc - xpathNodes, ids: foo id=id10 # OK! bar id=id12 # OK! file doc - evalNodes1 = xpathNodes1 ? true # OK! true # OK! file doc - evalNodes, ids: foo id=id17 bar id=id18 file doc - xpathNodes, ids: foo id=id31 # SHOULD BE id17bar id=id32 # SHOULD BE id18

1 0

DB connection between classes
by Karina C. Malavazi 30 Aug '18

30 Aug '18

Hello, I'm new on Java programming and also on nonSQL database... hope you can give me some support. I have a group of testcases where every time that a test is executed the database has to be connected again, which lets my tests very slow. Example: public class AllTests { . public static Test suite() throws IOException { TestSuite suite = new TestSuite(); suite.addTestSuite(NumberTest.class); suite.addTestSuite(LetterTest.class); suite.addTestSuite(ImageTest.class); ... } } In the example above, NumberTest.class, LetterTest.class and ImageTest.class are tests made for some interfaces that use the same document from the database. However, each one of them finish (NumberTest and LetterTest), I have to connect to the database again. I'm not "closing" it in any place, only opening. So, why is this being closed every time that a new test is executed? Is there any way to keep the db connection opened during all the tests and close it only in the end? Any idea or tip will be helpful. Thanks, Karina.

2 1

rest parallel
by Ветошкин Владимир 29 Aug '18

29 Aug '18

2 9

ThreadLocal error in Tomcat
by Simon Chatelain 29 Aug '18

29 Aug '18

Hello all, I am currently using BaseX in embedded mode as a XQuery processor in a tomcat webapplication. I am receiving some XML, create a BaseX in memory database, execute some XQuery, then close the context. But when I stop tomcat or reload the web application, I have a lot of errors saying: SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [webap] created a ThreadLocal with key of type [java.lang.ThreadLocal.SuppliedThreadLocal] (value [java.lang.ThreadLocal$SuppliedThreadLocal@60506a99]) and a value of type [org.basex.query.expr.path.PathCache] (value [org.basex.query.expr.path.PathCache@665cfafa]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. Is there a way to avoid that error, by forcibly clearing these cache or something ? Thanks Simon

2 2

Performance (index?) on large intermediate structures
by Sebastian Albert 29 Aug '18

29 Aug '18

Dear BaseX community I am currently working with BaseX to write a rather long XQuery statement with quite large intermediate structures (XML documents / fragments) bound to variables (in FLWOR expressions), which I query by XPath using attributes etc. further down. Having nested loops, this is becoming very slow. Are those intermediate structures automatically indexed? Or how can I achieve indexing on them? Or do I need to write them into a database for this indexing? (Can I specify a datbase as "in-memory"?) Then I guess I need a "BaseX script" rather than a single XQuery? Of course, I am looking for the most elegant/idiomatic way. Or is it best to build index-like intermediate structures myself, probably using maps rather than XML documents? Furthermore, are there profiling tools inside BaseX, or would I need to fire up a Java IDE to see where most time is being spent? Thanks in advance for any suggestions! Sebastian

2 1

[XQUERY] Filter and paging with window clause
by Marco Lettere 28 Aug '18

28 Aug '18

Hi all, here's a question related to XQuery, sorry for being slightly off-topic. I'm struggling to find a way to combine the windowing clause and FLOWR in order to get a paged result of a subset of items which respect a given filter. Of course I'm able to get this by first applying the filter to the whole input and then a second FLOWR for the windowing clause. But I was wondering whether it's possible to streamline this into one single FLOWR. Two bonus questions: 1. Will this actually be more efficient than the naive (two FLOWR) solution or is the almighty optimizer able to blend the two subsequent FLOWR on its own? 2. What if the input is unordered and I also would like to introduce sorting? The closer I get is [1] which is not what I'd need because I get 2,4,6,8,10 as result for page 1 but I'd really want 10 results per page thus 2,4,6,8,10,12,14,16,18,20. Thanks for any help on this in the meanwhile I'll stick to my naive solution. Marco. [1] let $page := 1 let $count := 10 return for tumbling window $w in (1 to 100) start $first at $s when ($s = 1 + ($page - 1) * $count) end $last at $e when ($e - $s = $count - 1) return $w ! (if (. mod 2 = 0 ) then . else ())

3 7

Strange index values with numerics
by Zachary N. Dean 28 Aug '18

28 Aug '18

Hi, I was recently taking a look into the index data files (for reasons) and came across something that I found strange... When numeric values are in nodes they are put into the index with min/max and distinct token values, which is cool. What's strange is, when negative integer values are in text and attribute nodes the index contains the minimum value correctly, but the maximum value is '4.9E-324' ([0,0,0,0,0,0,0,1]). This doesn't seem to happen with positive values. Now, with small value ranges I assume this is okay, but with many values I would imagine it could slow things down. Not sure if this is a bug or a feature, so I figured I'd bring it up. Here an example: <r> <a>-1</a> <b>0</b> <c>1</c> <d>-50000</d> <d>-49000</d> <e>2</e> <e>3</e> <f a="-1"/> <g>-1</g> <g>1</g> </r> I would have assumed that the index would see that element "d" has a min of -50000 and a max of -49000. Here the index infos: Elements - Structure: Hash - Entries: 8 g 2x, 2 distinct integers [-1, 1], leaf e 2x, 2 distinct integers [2, 3], leaf d 2x, 2 distinct integers [-50000, 4.9E-324], leaf c 1x, integer [1, 1], leaf f 1x, leaf r 1x a 1x, integer [-1, 4.9E-324], leaf b 1x, integer [0, 4.9E-324], leaf Attributes - Structure: Hash - Entries: 1 a 1x, integer [-1, 4.9E-324], leaf If it's a feature, then cool. Keep on rockin'! If not, then I hope this helps a little. Thanks, Zack Dean

3 2

Update functions - Error handling and counting
by Bogdan Bogucki 26 Aug '18

26 Aug '18

Hello, I am playing with XQUF in BaseX ann I encountered following problems. 1) XQuery is functional language and updating function can return empty sequence. How can I count and return result how many nodes was inserted or updated ? Example: declare %private %updating function cfv:upsertVehicles ($vehicles as node()*) as empty-sequence() { let $db := db:open("db")/cars for $v in $vehicles let $dbVehicle := $db/car[vin = $v/vin] return if (not(exists($dbVehicle))) then (insert node $v as last into $db) else if (exists($dbVehicle)) then (replace node $dbVehicle with $v) else () }; I have similar problem with error handling. How can I catch and return any error. Regards Bogdan Bogucki

2 5

archive
by Ветошкин Владимир 23 Aug '18

23 Aug '18

4 7

Forcing reading _only_ from standard input
by Richard Walker 23 Aug '18

23 Aug '18

Hi. I have a script in which I process data from standard input like this: VOCAB_WITH_NEW_VOCAB_NOTE="$(echo "$VOCAB" | "${BASEX}" -i - -q $" copy \$c := . modify ... etc. etc. The script has been working fine. However, due to a defect in an earlier line in the script, during one test run, the value of $VOCAB was (incorrectly) an empty string. But in this case, I got a quite unexpected result from the above invocation of BaseX. I can easily repeat the same phenomenon out of the script's context thus: $ echo '' | basex -i - -q '.' <vocabulary xmlns="http://vocabs.ands.org.au/registry/schema/2017/01/vocabulary" ... etc. etc. In this case, rather than processing the empty string provided as standard input, BaseX fetched data from one of my databases. Can I/how do I force the option "-i -" to mean "read data from standard input, and _only_ from standard input, even if that is empty"? -- Richard Walker Australian Research Data Commons (ARDC) (formerly, Australian National Data Service, ANDS) 101 Liversidge Street The Australian National University Acton ACT 2601, Australia E: Richard.Walker(a)ardc.edu.au T: +61 2 6125 0584 W: ardc.edu.au W: www.ands.org.au

2 4