August 2014 - BaseX-Talk - mailman.uni-konstanz.de

serialisation before xslt
by Marc 07 Aug '14

07 Aug '14

Hi, I want to transform with xslt before to output the file. If I do a serialize on the disk, runs the xslt with saxon et write the result the space are conserved (I use chop=false to load the file in the base). But if I transform the file in BaseX, and serialize the file after, the element are passed to the xslt indented and I lost the space at the begining and the end of element. And other thing, if I put the "declare option db:chop 'false';" at the beginning of the XQuery, the file is indented, if I put it out (it's unuseful) the serialisation without indentation is done. I use the 7.9 version.

2 3

Lower level synching
by Marco Lettere 07 Aug '14

07 Aug '14

Hello to everyone who is still there even in deep Summer! I was just wondering whether anyone had experiences with synchronizing two basex databases (same server or not) at a lower level for example using (r)sync or nfs or even Hdfs. Would it be feasible? Thanks, Marco.

2 1

Different behaviour on exception
by Marco Lettere 07 Aug '14

07 Aug '14

Hi all, I have the following code [1] that I expect to raise a xml parser exception but it raises a typecast exception [2]. Instead I get the expected exception [3] when substituting the call to local:h with direct parse -xml() (as for the snippet in the comment). Any clue? Thanks, Marco. [1] declare function local:f($rdf as node()){ $rdf }; declare function local:h(){ parse-xml("xxx") }; declare function local:g(){ let $rdf := local:h() (: parse-xml("xxx") :) let $validate := local:f($rdf) (:$rdf:) return ($validate) }; local:g() [2] Stopped at /home/lettere/tmp/basex80/basex/file, 2/18: [XPTY0004] Cannot cast from empty-sequence() to node(). [3] Stopped at /home/lettere/tmp/basex80/basex/file, 11/26: [FODC0006] SAX: "file:/home/lettere/tmp/basex80/basex/file" (Line 1): Content is not allowed in prolog.

2 1

Unit module sometimes drives me nuts
by Marc van Grootel 07 Aug '14

07 Aug '14

Really, I want to be good and write my tests together with the code. I still don't do it often enough but I know it's important and for the code I'm currently working on I try to cover most of it with tests. I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-) No, seriously, here are a few more general things that bug me about the Unit module. - assertions causing an NPE when an XQuery error is raised but no traceback that tells me what/where the error actually is. [ex1] - assertions causing a failure/error reported, produce a test result but stop running further tests after the one that raises the error [ex2] I tried to reduce these issues into a short example but I think that they highlight a more general issue with the current Unit module. It's mainly the fact that there's no indication of the problem that forces me to add unit:ignore to all tests and one by one removing it, or executing the assertion xquery in a client to find out where the issue lies. I also have an improvement idea: - Personally, I run the tests often (minutes rather than hours), and from the shell. The XML format feedback isn't very helpful. It would be for automated test reports but I think the usability (readability) of the feedback during development is just as, if not more, important. Maybe adding an extra switch (-t for textual feedback and -tx for XML feedback) could help. It should just focus on summarizing clearly the expected, return values, failures and errors, and provide helpful context. Note that I always run the code using `basex -t` from bash and I'm running 8.0 beta 7ba9510 Note too that most of these things happen to me when doing map related stuff. ... and now I stop moaning and get back to coding. --Marc [ex1] example of test that causes an NPE without any helpful output *declare* *%unit:test* *function* *test:npe*() { *unit:assert-equals*(*map* {}('x'), *map* { 'x': 1 }('x')) }; [ex2] example of test that causes further tests to be executed *declare* *%unit:test* *function* *test:fail*() { *unit:assert-equals*(*map* {}, *map* { 'x': 1 }) }; doing map {} = map {'x': 1} causes this error [XPTY0004] map(*) and map(*) cannot be compared.

2 4

Same query, huge difference in performance
by Paul Swennenhuis 05 Aug '14

05 Aug '14

Hi, I have two XQueries that do exactly the same thing, i.e. create a list of waters in the Facts database and display where these waters flow into, if any. But Query 1 executes 10 times faster than Query 2 (in the GUI). The only difference is in the selection of the query context. Query 1 assumes the Facts database is open and does not use an explicit doc() or collection() context. Query 2 uses an explicit collection("Facts") context Query 1 executes in appr. 600 msecs, Query 2 in appr. 60 msecs (!) How can there be such a huge difference? Is it because Query 2 operates twice on the $facts variable and does not need to evaluate that again? Listings: Query1 (: list waters and where they stream to (if any):) for $source in //(sea|river|lake) let $toId := $source/to/@water let $to := (//sea|//river|//lake)[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } Query2 (: list waters and where they stream to (if any):) let $facts := collection("Facts")//(sea|river|lake ) for $source in $facts let $toId := $source/to/@water let $to := $facts[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } As a side-question: I want to extend the query to make it recursive: river "Bahr el-Djebel" streams into river "White Nile" streams into river "Nile" streams into sea "Mediterranean Sea" I think I can find out how to do that, but how can I optimize the recursion process? Would a recursive function be efficient? Thanks, Paul

4 12

UPDINDEX and ever growing index size
by James Ball 04 Aug '14

04 Aug '14

Hello, I’ve finally had some time to look at an issue I’ve been having with databases that have UPDINDEX set to true. I’m now running a BaseX 8.0 beta and was on 7.X when I first encountered this. The issue I’m seeing is that the size of the index grows by approximately 1MB with every updating ‘transaction’ (snapshot?) even if there is no new data for the index. For example if I have a database with 100,000 files and I replace one of those files (with itself so there’s no new data) then the size of the index will go up by around 1MB. If I replace 1000 files in the same transaction (again with themselves) the size of the index will go up again by around 1MB. Dropping and recreating the index returns it to its original size. I have a current project where I’m expecting thousands of files a few at time that need to be added/replaced - I completely ran out of disk space before I spotted what was happening when testing. Is this expected behaviour? I don’t know the format for the index files but I’ve looked at atvl.basex just in a text editor. It looks like for each update to the index around 40k blank lines are being added. I don’t know that they are truly blank lines - but that’s how they’re rendering in the editor. I’ve created a small test case to replicate what I’m seeing. [Mac OS 10.9.4, BaseX 8.0 beta 496c381] Thank you for your help. Regards, James 1) SET UPDINDEX TRUE 2) CREATE DB Index-Test-Updindex-XQ 3) Run an XQuery to populate a reasonable database (I do 10,000 items) ------------------------ let $files_from := 1 let $files := 10000 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 4) Check the size of the index - should be about 325kB 5) Run the XQuery again (it will replace files with identical copies) but for just one file: ------------------------ let $files_from := 1 let $files := 1 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 6) Check the size of the index - it will be about 1MB 7) Run the XQuery again for around 100 files ------------------------ let $files_from := 1 let $files := 100 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 8) Check the size of the index - it will be about 2MB. 9) Drop the index and recreate it. It will be about 325kB again. --------------------------------- James Ball me(a)jamesball.co.uk

2 11

Question about the mailing-list
by Paul Swennenhuis 04 Aug '14

04 Aug '14

A bit off-topic but this weekend I ran into a strange issue with this mailinglist. I tried to send a message from the same e-mail address I am using now (paul(a)swennenhuis.nl) but it bounced back every time I tried. I get the feeling that the subscription is not merely e-mail based but also IP address based? Even weirder: the rejected mails were sent to another e-mail address of mine, p.a.m.swennenhuis(a)gmail.com I can't remember even having subscribed with that address, not does it appear in the subscribers list of the mailing list. Paul

2 1

Copy between databases
by Marco Lettere 04 Aug '14

04 Aug '14

3 3

Réponse automatique
by mab 01 Aug '14

01 Aug '14

1 0