Really, I want to be good and write my tests together with the code. I still don't do it often enough but I know it's important and for the code I'm currently working on I try to cover most of it with tests.
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
No, seriously, here are a few more general things that bug me about the Unit module.
- assertions causing an NPE when an XQuery error is raised but no traceback that tells me what/where the error actually is. [ex1]
- assertions causing a failure/error reported, produce a test result but stop running further tests after the one that raises the error [ex2]
I tried to reduce these issues into a short example but I think that they highlight a more general issue with the current Unit module.
It's mainly the fact that there's no indication of the problem that forces me to add unit:ignore to all tests and one by one removing it, or executing the assertion xquery in a client to find out where the issue lies.
I also have an improvement idea:
- Personally, I run the tests often (minutes rather than hours), and from the shell. The XML format feedback isn't very helpful. It would be for automated test reports but I think the usability (readability) of the feedback during development is just as, if not more, important. Maybe adding an extra switch (-t for textual feedback and -tx for XML feedback) could help. It should just focus on summarizing clearly the expected, return values, failures and errors, and provide helpful context.
Note that I always run the code using `basex -t` from bash and I'm running 8.0 beta 7ba9510
Note too that most of these things happen to me when doing map related stuff.
... and now I stop moaning and get back to coding.
--Marc
[ex1] example of test that causes an NPE without any helpful output
*declare* *%unit:test* *function* *test:npe*() { *unit:assert-equals*(*map* {}('x'), *map* { 'x': 1 }('x')) }; [ex2] example of test that causes further tests to be executed
*declare* *%unit:test* *function* *test:fail*() { *unit:assert-equals*(*map* {}, *map* { 'x': 1 }) };
doing
map {} = map {'x': 1} causes this error
[XPTY0004] map(*) and map(*) cannot be compared.
Hi Marc,
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
Thanks for moaning! XQUnit is still sth. we are heavily working on, so any feedback is welcome.
- assertions causing an NPE when an XQuery error is raised but no traceback
that tells me what/where the error actually is. [ex1]
NPEs are usually serious bugs. We'll have a look at that.
- assertions causing a failure/error reported, produce a test result but
stop running further tests after the one that raises the error [ex2]
The reason is that maps are functions, and functions cannot be compared. But the current version of the XQuery spec. will provide support for comparing maps, so it's basically a matter of time when the comparision of maps will be made possible. I'll dig into this one as well; maybe it's not a big deal to provide an early yet stable implementation.
- Personally, I run the tests often (minutes rather than hours), and from
the shell. The XML format feedback isn't very helpful.
It's probably not that trivial to find a proper textual representation for everything that can be done with XQUnit. One thing you can do is to send the output to a second BaseX call. The following command-line code will only return the failures of your XML code:
basex -t . | basex -i - //failure
Of course, you could also call a function that does some more sophisticated stuff with your input.
Hope this helps, Christian
Hi Christian,
Yes, I already learned to write tests to work around not being able to compare maps. It was more the general behaviour of a test run not giving me much to go on. Noticed it on maps mainly.
Thanks for the tip of piping result through an expression, didn't think of that.
Keep up the good work, I'm generally enjoying this :)
--Marc
On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
Thanks for moaning! XQUnit is still sth. we are heavily working on, so any feedback is welcome.
- assertions causing an NPE when an XQuery error is raised but no
traceback
that tells me what/where the error actually is. [ex1]
NPEs are usually serious bugs. We'll have a look at that.
- assertions causing a failure/error reported, produce a test result but
stop running further tests after the one that raises the error [ex2]
The reason is that maps are functions, and functions cannot be compared. But the current version of the XQuery spec. will provide support for comparing maps, so it's basically a matter of time when the comparision of maps will be made possible. I'll dig into this one as well; maybe it's not a big deal to provide an early yet stable implementation.
- Personally, I run the tests often (minutes rather than hours), and from
the shell. The XML format feedback isn't very helpful.
It's probably not that trivial to find a proper textual representation for everything that can be done with XQUnit. One thing you can do is to send the output to a second BaseX call. The following command-line code will only return the failures of your XML code:
basex -t . | basex -i - //failure
Of course, you could also call a function that does some more sophisticated stuff with your input.
Hope this helps, Christian
Hi Marc,
XQUnit can now also be used to compare maps. Feel free to check out the latest snapshot [1].
Cheers, Christian
[1] http://files.basex.org/releases/latest/
On Wed, Jul 30, 2014 at 5:06 PM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
Yes, I already learned to write tests to work around not being able to compare maps. It was more the general behaviour of a test run not giving me much to go on. Noticed it on maps mainly.
Thanks for the tip of piping result through an expression, didn't think of that.
Keep up the good work, I'm generally enjoying this :)
--Marc
On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
Thanks for moaning! XQUnit is still sth. we are heavily working on, so any feedback is welcome.
- assertions causing an NPE when an XQuery error is raised but no
traceback that tells me what/where the error actually is. [ex1]
NPEs are usually serious bugs. We'll have a look at that.
- assertions causing a failure/error reported, produce a test result but
stop running further tests after the one that raises the error [ex2]
The reason is that maps are functions, and functions cannot be compared. But the current version of the XQuery spec. will provide support for comparing maps, so it's basically a matter of time when the comparision of maps will be made possible. I'll dig into this one as well; maybe it's not a big deal to provide an early yet stable implementation.
- Personally, I run the tests often (minutes rather than hours), and
from the shell. The XML format feedback isn't very helpful.
It's probably not that trivial to find a proper textual representation for everything that can be done with XQUnit. One thing you can do is to send the output to a second BaseX call. The following command-line code will only return the failures of your XML code:
basex -t . | basex -i - //failure
Of course, you could also call a function that does some more sophisticated stuff with your input.
Hope this helps, Christian
-- --Marc
Hi Christian,
Got it. Rewrote some tests and it works great.
Thanks. --Marc
On Wed, Aug 6, 2014 at 8:08 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
XQUnit can now also be used to compare maps. Feel free to check out the latest snapshot [1].
Cheers, Christian
[1] http://files.basex.org/releases/latest/
On Wed, Jul 30, 2014 at 5:06 PM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
Yes, I already learned to write tests to work around not being able to compare maps. It was more the general behaviour of a test run not giving
me
much to go on. Noticed it on maps mainly.
Thanks for the tip of piping result through an expression, didn't think
of
that.
Keep up the good work, I'm generally enjoying this :)
--Marc
On Wed, Jul 30, 2014 at 4:48 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Marc,
I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-)
Thanks for moaning! XQUnit is still sth. we are heavily working on, so any feedback is welcome.
- assertions causing an NPE when an XQuery error is raised but no
traceback that tells me what/where the error actually is. [ex1]
NPEs are usually serious bugs. We'll have a look at that.
- assertions causing a failure/error reported, produce a test result
but
stop running further tests after the one that raises the error [ex2]
The reason is that maps are functions, and functions cannot be compared. But the current version of the XQuery spec. will provide support for comparing maps, so it's basically a matter of time when the comparision of maps will be made possible. I'll dig into this one as well; maybe it's not a big deal to provide an early yet stable implementation.
- Personally, I run the tests often (minutes rather than hours), and
from the shell. The XML format feedback isn't very helpful.
It's probably not that trivial to find a proper textual representation for everything that can be done with XQUnit. One thing you can do is to send the output to a second BaseX call. The following command-line code will only return the failures of your XML code:
basex -t . | basex -i - //failure
Of course, you could also call a function that does some more sophisticated stuff with your input.
Hope this helps, Christian
-- --Marc
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Marc van Grootel