Hi,
I am playing around with basex-tests [1]. I cloned the repository, then went to the directory and tried "mvn test". I got a compilation error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile (default-testCompile) on project basex-tests: Compilation failure [ERROR] .../basex-tests/src/test/java/org/basex/test/qt3ts/app/AppXMark.java:[293,26] constant string too long
If I try only "mvn compile", everything goes fine. From there, I could then execute the tests by using (from the same directory):
M2_BASEX=~/.m2/repository/org/basex/basex BASEX=$M2_BASEX/7.2.2-SNAPSHOT/basex-7.2.2-20120514.173900-14.jar
java -cp target/classes/:$BASEX org.basex.tests.w3c.QT3TS \ -p ~/tmp/fots-try
The directory ~/tmp/fots-try/ is not the official FOTS from the WG, but a small sample I wrote for playing with FOTS. It contains the following files:
fots-try/ catalog.xml sample.xml set.xml
The file catalog.xml is the FOTS catalog, the main entry point, sample.xml is a sample XML document, and set.xml is the only one test set in the test suite.
I have two remarks. 1/ The second test case fails because BaseX tries to resolve sample.xml in the current directory, but it should be resolved relatively to the catalog file. The FOTS doc says (for the attribute group "fileAttr"): "The URI will always be relative to the base URI of the XML document in which the attribute appears."
2/ BaseX creates a file "qt3ts.log" in the test suite directory. I couldn't find any infos in the FOTS doc about where to put the result, but Saxon does that in results/saxon/, which is probably better in order to avoid interferences between several processors. The catalog contains:
<catalog xmlns="http://www.w3.org/2010/09/qt-fots-catalog" test-suite="try" version="0.1.0">
<environment name="empty"/>
<!-- set prefix "s" and refer to sample.xml --> <environment name="sample"> <namespace prefix="s" uri="http://example.org/sample"/> <source role="." file="sample.xml" uri="http://example.org/sample.xml"> <description>Some sample input tree.</description> <created by="Florent Georges" on="2012-05-14"/> </source> </environment>
<!-- the only one test set in this suite --> <test-set name="set" file="set.xml"/>
</catalog>
The test set descriptor, set.xml, contains two test cases:
<test-set xmlns="http://www.w3.org/2010/09/qt-fots-catalog" name="set">
<description>Some test set...</description>
<!-- test "hello" is a string with content 'hello' --> <test-case name="sample-001"> <description>Some test case...</description> <created by="Florent Georges" on="2012-05-14"/> <environment ref="empty"/> <test>"hello"</test> <result> <all-of> <assert-eq>'hello'</assert-eq> <assert-type>xs:string</assert-type> </all-of> </result> </test-case>
<!-- test sample.xml contains one child element named sample --> <test-case name="sample-002"> <description>Other test case...</description> <created by="Florent Georges" on="2012-05-14"/> <environment ref="sample"/> <test>sample</test> <result> <assert-type>element(sample)</assert-type> </result> </test-case>
</test-set>
Thank you for making this available to the community!
Regards,
Hi Florent,
thanks for your input.
I am playing around with basex-tests [1]. I cloned the repository, then went to the directory and tried "mvn test". I got a compilation error:
The error seems to point to an automatically generated class, which contains a string with (way) too many characters. Good to know that everything works with "mvn compile", so I'll currently leave it like that for now..
I have two remarks. 1/ The second test case fails because BaseX tries to resolve sample.xml in the current directory, but it should be resolved relatively to the catalog file.
Out of interest: could you try switch to your test directory and run the tests from there? This is probably what most of us do. To be honest, I'm not even sure if someone in the team uses the -p flag at all, although I mentioned it to you in one of my last e-mails.
2/ BaseX creates a file "qt3ts.log" in the test suite directory. I couldn't find any infos in the FOTS doc about where to put the result, but Saxon does that in results/saxon/, which is probably better in order to avoid interferences between several processors.
Yes, this could be done.. But hasn't been considered so far, as we only use the code to test our own implementation. As the QT3TS code is coupled pretty tightly to our implementation, we'll probably stick with the status quo (unless the code will be adapted by someone to also work for other Java implementations). I guess that Leo's XQuery FOTS code (or a slightly enhanced version of it) could be adapted much easier to other implementations, as most of the code is written in pure XQuery 3.0 [1].
Hope this helps, Christian
Christian Grün wrote:
Hi,
The error seems to point to an automatically generated class, which contains a string with (way) too many characters. Good to know that everything works with "mvn compile", so I'll currently leave it like that for now..
Ok. FWIW, it seems extracting them as external files or resources should be enough.
Out of interest: could you try switch to your test directory and run the tests from there? This is probably what most of us do. To be honest, I'm not even sure if someone in the team uses the -p flag at all, although I mentioned it to you in one of my last e-mails.
A bit weird... For what I've seen in the source code, the test suite directory is either set by "-p", or fixed to "g:/XML/w3c/qt3ts/" (static variable QT3TS.qt3tsPath). Because the base URI is not properly set for files in environments defined in the global catalog, they are resolved against the PWD. So the only way to have them resolved, if I am right, is anyway to be in the test suite directory and either to have it at "g:/XML/w3c/qt3ts/" (on a Windows filesystem only) or to use "-p ."
I've tried "-p ." and it works. Thanks!
FWIW the line 236 in QT3TS (b = null;) looks suspect to me. It sets the base URI to resolve files in environents to null (only for envs defined in the global catalog.xml).
I guess that Leo's XQuery FOTS code (or a slightly enhanced version of it) could be adapted much easier to other implementations, as most of the code is written in pure XQuery 3.0 [1].
Yes, this is a good candidate! (even though at first glance it uses map{}, so is restricted to XQuery 3.0 + Map Facility implementations). BTW the goal is not to have one test harness for all implementations, but to be sure that all implementations are able to run FOTS-based test suites ;-)
Thanks for your response. BaseX is definitely one FOTS-ready processor! ;-)
Regards,
-- Florent Georges http://fgeorges.org/ http://h2oconsulting.be/
Yes, this is a good candidate! (even though at first glance it uses map{}, so is restricted to XQuery 3.0 + Map Facility implementations).
True; here, we may have to use element fragments as drawback.
BTW the goal is not to have one test harness for all implementations, but to be sure that all implementations are able to run FOTS-based test suites ;-)
Thanks for looking at this! Have you got some other driver implementations so far? Christian
Christian Grün wrote:
Yes, this is a good candidate! (even though at first glance it uses map{}, so is restricted to XQuery 3.0 + Map Facility implementations).
True; here, we may have to use element fragments as drawback.
I am sure we will find solutions if ever needed ;-)
Thanks for looking at this! Have you got some other driver implementations so far?
That's probably a discussion we should continue on the EXPath mailing list, I don't want to bother BaseX users, but in a few words Mike told me he doesn't have any objection to let us try to adapt the code to Saxon HE (they use Saxon EE internally of course), and Matthias was optimist about Zorba. There are a couple of other leading implementations I'd like to check, but that's a good start.
Also, I think (as far as I know) that FOTS does not cope very well with heterogenous sequence. For instance for the result of the HTTP Client function (returning an element http:response describing the response then one or several items representing the actual content of the response), it is not easy to assert anything about the returned sequence except its length (or to create several test cases with the same function call, one asserting the length, one asserting things on the first item, one asserting things on the second item, ...)
But I think that's a good start! ;-)
Thanks for your responses. Regards,
-- Florent Georges http://fgeorges.org/ http://h2oconsulting.be/
basex-talk@mailman.uni-konstanz.de