- BaseX-Talk - mailman.uni-konstanz.de

Different behaviour on exception
by Marco Lettere 07 Aug '14

07 Aug '14

Hi all, I have the following code [1] that I expect to raise a xml parser exception but it raises a typecast exception [2]. Instead I get the expected exception [3] when substituting the call to local:h with direct parse -xml() (as for the snippet in the comment). Any clue? Thanks, Marco. [1] declare function local:f($rdf as node()){ $rdf }; declare function local:h(){ parse-xml("xxx") }; declare function local:g(){ let $rdf := local:h() (: parse-xml("xxx") :) let $validate := local:f($rdf) (:$rdf:) return ($validate) }; local:g() [2] Stopped at /home/lettere/tmp/basex80/basex/file, 2/18: [XPTY0004] Cannot cast from empty-sequence() to node(). [3] Stopped at /home/lettere/tmp/basex80/basex/file, 11/26: [FODC0006] SAX: "file:/home/lettere/tmp/basex80/basex/file" (Line 1): Content is not allowed in prolog.

2 1

Unit module sometimes drives me nuts
by Marc van Grootel 07 Aug '14

07 Aug '14

Really, I want to be good and write my tests together with the code. I still don't do it often enough but I know it's important and for the code I'm currently working on I try to cover most of it with tests. I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-) No, seriously, here are a few more general things that bug me about the Unit module. - assertions causing an NPE when an XQuery error is raised but no traceback that tells me what/where the error actually is. [ex1] - assertions causing a failure/error reported, produce a test result but stop running further tests after the one that raises the error [ex2] I tried to reduce these issues into a short example but I think that they highlight a more general issue with the current Unit module. It's mainly the fact that there's no indication of the problem that forces me to add unit:ignore to all tests and one by one removing it, or executing the assertion xquery in a client to find out where the issue lies. I also have an improvement idea: - Personally, I run the tests often (minutes rather than hours), and from the shell. The XML format feedback isn't very helpful. It would be for automated test reports but I think the usability (readability) of the feedback during development is just as, if not more, important. Maybe adding an extra switch (-t for textual feedback and -tx for XML feedback) could help. It should just focus on summarizing clearly the expected, return values, failures and errors, and provide helpful context. Note that I always run the code using `basex -t` from bash and I'm running 8.0 beta 7ba9510 Note too that most of these things happen to me when doing map related stuff. ... and now I stop moaning and get back to coding. --Marc [ex1] example of test that causes an NPE without any helpful output *declare* *%unit:test* *function* *test:npe*() { *unit:assert-equals*(*map* {}('x'), *map* { 'x': 1 }('x')) }; [ex2] example of test that causes further tests to be executed *declare* *%unit:test* *function* *test:fail*() { *unit:assert-equals*(*map* {}, *map* { 'x': 1 }) }; doing map {} = map {'x': 1} causes this error [XPTY0004] map(*) and map(*) cannot be compared.

2 4

Same query, huge difference in performance
by Paul Swennenhuis 05 Aug '14

05 Aug '14

Hi, I have two XQueries that do exactly the same thing, i.e. create a list of waters in the Facts database and display where these waters flow into, if any. But Query 1 executes 10 times faster than Query 2 (in the GUI). The only difference is in the selection of the query context. Query 1 assumes the Facts database is open and does not use an explicit doc() or collection() context. Query 2 uses an explicit collection("Facts") context Query 1 executes in appr. 600 msecs, Query 2 in appr. 60 msecs (!) How can there be such a huge difference? Is it because Query 2 operates twice on the $facts variable and does not need to evaluate that again? Listings: Query1 (: list waters and where they stream to (if any):) for $source in //(sea|river|lake) let $toId := $source/to/@water let $to := (//sea|//river|//lake)[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } Query2 (: list waters and where they stream to (if any):) let $facts := collection("Facts")//(sea|river|lake ) for $source in $facts let $toId := $source/to/@water let $to := $facts[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } As a side-question: I want to extend the query to make it recursive: river "Bahr el-Djebel" streams into river "White Nile" streams into river "Nile" streams into sea "Mediterranean Sea" I think I can find out how to do that, but how can I optimize the recursion process? Would a recursive function be efficient? Thanks, Paul

4 12

UPDINDEX and ever growing index size
by James Ball 04 Aug '14

04 Aug '14

Hello, I’ve finally had some time to look at an issue I’ve been having with databases that have UPDINDEX set to true. I’m now running a BaseX 8.0 beta and was on 7.X when I first encountered this. The issue I’m seeing is that the size of the index grows by approximately 1MB with every updating ‘transaction’ (snapshot?) even if there is no new data for the index. For example if I have a database with 100,000 files and I replace one of those files (with itself so there’s no new data) then the size of the index will go up by around 1MB. If I replace 1000 files in the same transaction (again with themselves) the size of the index will go up again by around 1MB. Dropping and recreating the index returns it to its original size. I have a current project where I’m expecting thousands of files a few at time that need to be added/replaced - I completely ran out of disk space before I spotted what was happening when testing. Is this expected behaviour? I don’t know the format for the index files but I’ve looked at atvl.basex just in a text editor. It looks like for each update to the index around 40k blank lines are being added. I don’t know that they are truly blank lines - but that’s how they’re rendering in the editor. I’ve created a small test case to replicate what I’m seeing. [Mac OS 10.9.4, BaseX 8.0 beta 496c381] Thank you for your help. Regards, James 1) SET UPDINDEX TRUE 2) CREATE DB Index-Test-Updindex-XQ 3) Run an XQuery to populate a reasonable database (I do 10,000 items) ------------------------ let $files_from := 1 let $files := 10000 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 4) Check the size of the index - should be about 325kB 5) Run the XQuery again (it will replace files with identical copies) but for just one file: ------------------------ let $files_from := 1 let $files := 1 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 6) Check the size of the index - it will be about 1MB 7) Run the XQuery again for around 100 files ------------------------ let $files_from := 1 let $files := 100 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 8) Check the size of the index - it will be about 2MB. 9) Drop the index and recreate it. It will be about 325kB again. --------------------------------- James Ball me(a)jamesball.co.uk

2 11

Question about the mailing-list
by Paul Swennenhuis 04 Aug '14

04 Aug '14

A bit off-topic but this weekend I ran into a strange issue with this mailinglist. I tried to send a message from the same e-mail address I am using now (paul(a)swennenhuis.nl) but it bounced back every time I tried. I get the feeling that the subscription is not merely e-mail based but also IP address based? Even weirder: the rejected mails were sent to another e-mail address of mine, p.a.m.swennenhuis(a)gmail.com I can't remember even having subscribed with that address, not does it appear in the subscribers list of the mailing list. Paul

2 1

Copy between databases
by Marco Lettere 04 Aug '14

04 Aug '14

3 3

Réponse automatique
by mab 01 Aug '14

01 Aug '14

1 0

Re: [basex-talk] XML-Validation return value
by Martin Hartmann 31 Jul '14

31 Jul '14

2 1

Cannot instantiate java.net.URLEncoder
by Marc van Grootel 30 Jul '14

30 Jul '14

Hi, What could be the reason that importing java.net.URLDecoder works as expected but doing the same with java.net.URLEncoder doesn't. The following works: import module namespace decoder = "java.net.URLDecoder"; decoder:decode('foo/bar%20baz') => 'foo/bar baz' This gives a "Could not instantiate module 'java.net.URLEncoder' import module namespace encoder = "java.net.URLEncoder"; encoder:encode('foo/bar baz') --Marc

2 3

Re: [basex-talk] How to create xml file dynamically and store into database directly without storing into my disk?
by Dirk Kirsten 30 Jul '14

30 Jul '14

Hello Manish, please always reply to the list, so others can also benefit from questions and answers. String query = "db:add("db","xmlFile")"; will certainly not work, this will throw a java error due to the ". This should work: String query = "db:add('db','xmlFile')"; As I already suggested (multiple times) on StackOverflow, you will have to build and modify the document within BaseX using XQuery Update. This way, the XML file can be kept in main memory and later be added to the database. Please read up in our documentation about XQuery Updates. Cheers, Dirk On 30/07/14 11:37, manish payasi wrote: > Thanks Dirk, > > I am trying to write like > String query = "db:add("db","xmlFile")"; where xmlFile is string > variable which has xml file name, > but it is throwing some syntax error (like Syntax error on tokens, delete > these tokens for both the arguments) . > > Another thing is that in second argument we are passing the xml file which > is still coming from disk which we don't want it rather than i want create > the xml file dynamically in java and store this file directly into database. > > Thanks and Regards > manish payasi. > > > > On Tue, Jul 29, 2014 at 6:29 PM, Dirk Kirsten <dk(a)basex.org> wrote: > >> Hi Manish, >> >> You might want to take a look at our java examples here : >> https://docs.basex.org/wiki/Java_Examples >> >> So to simply execute an XQuery and using db:add() (see >> https://docs.basex.org/wiki/Database_Module#db:add) to add to the >> database would look something like that: >> >> Context context = new Context(); >> String query = "db:add("your-database", <xml>here goes your >> xml</xml>, "myfile.xml") >> QueryProcessor proc = new QueryProcessor(query, context); >> proc.execute(); >> proc.close(); >> context.close(); >> >> the second argument of db:add() is an item(), so you can also >> dynamically generate an XML element here. >> >> Cheers, >> Dirk >> >> On 29/07/14 14:46, manish payasi wrote: >>> How to create xml file dynamically in java and store into BaseX database >>> 'db'. That means it shouldn't not be created into my disk(my system) but >> it >>> should be directly created(store) into database 'db' ????? >>> >>> Please give me some better solution. >>> >> >> -- >> Dirk Kirsten, BaseX GmbH, http://basex.org >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >> > -- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

1 0