Dear BaseX team,
I face a problem with the "validate" module. My impression when using "validate:xsd-info" is the following: (a) if the XSD is supplied as argument - xs:import is NOT resolved (b) if the XSD is inferred from xsi:schemaLocation - xs:import IS resolved
It would be wonderful if imports/includes could be automatically resolved - the "validate" module could then be extremely useful!
Best regards, Hans-Juegen
PS: My experiment:
======================================================
let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS" kann nicht als "type definition"-Komponente aufgelöst werden.</result1> <result2/> </results>
I just want to second that suggestion! In the mean time, Hans, what I do id this:
*let **$xsd-file-path* := *fix-validation:update-file-system-dtds-xsds*( *$xsd*) *return **validate:xsd*(*$file*, *$xsd-file-path*)
...
*declare function **fix-validation:update-file-system-dtds-xsds*(*$dtd* *as **xs:string*) *as **xs:string*{ *let **$to-file-sys*:= *for **$db-url* *in **db:list*('AppResources', 'dtds') *let* *$target-file* := *concat*('BaseX-DTDs/', *substring-after*(*$db-url*, 'dtds/')) (:let $debug := file:write('debug-dtd-2-file.xml', $target-file):) *let* *$dir* := *file:parent*(*$target-file*) *let* *$create-dir* := *if* (*file:exists*(*$dir*)) *then *() *else **file:create-dir*(*$dir*) *return **file:write*(*$target-file*, *db:open*( 'AppResources', *$db-url*))
(: Get the new file path for db equivalent $dtd :) *let* *$dtd-on-file-sys* := *concat*('BaseX-DTDs/', *substring-after*( *$dtd*, 'dtds/')) (: Return path to file system copy of the DTD :) *return **$dtd-on-file-sys* };
Our XSDs change enough and our drop cycle is fast enough, that it's worth doing this on each validation to avoid confusion between schema versions in 2 locations. We need the .xsd in BaseX because we parse them for dynamic documentation. Otherwise, you might want to keep the .xsd files outside of BaseX.
I don't remember if that worked with .xsd, but with .dtds and .css saving the files as raw instead of text enabled the resolution of dependents. Of course, if you need to xquery your schemas, saving them as raw is not an option.
Thank you, France, for sharing this. But I am not sure if I understand well: it seems to me you copy DTDs from the database to the file system, before using them for validation, and this seems to point at a problem with resolving references which exists only if DTDs reside in the database, rather than in the file system - is this so? But my problems with XSD validation (involving imports) occur in the file system - databases are not involved.
Kind regards, Hans-Juergen
France Baril france.baril@architextus.com schrieb am 15:31 Donnerstag, 11.September 2014:
I just want to second that suggestion! In the mean time, Hans, what I do id this:
let $xsd-file-path:= fix-validation:update-file-system-dtds-xsds($xsd) return validate:xsd($file, $xsd-file-path)
...
declare function fix-validation:update-file-system-dtds-xsds($dtd as xs:string) as xs:string{ let $to-file-sys:= for $db-url in db:list('AppResources', 'dtds') let $target-file := concat('BaseX-DTDs/', substring-after($db-url, 'dtds/')) (:let $debug := file:write('debug-dtd-2-file.xml', $target-file):) let $dir := file:parent($target-file) let $create-dir := if (file:exists($dir)) then () else file:create-dir($dir) return file:write($target-file, db:open('AppResources', $db-url))
(: Get the new file path for db equivalent $dtd :) let $dtd-on-file-sys := concat('BaseX-DTDs/', substring-after($dtd, 'dtds/')) (: Return path to file system copy of the DTD :)
return $dtd-on-file-sys };
Our XSDs change enough and our drop cycle is fast enough, that it's worth doing this on each validation to avoid confusion between schema versions in 2 locations. We need the .xsd in BaseX because we parse them for dynamic documentation. Otherwise, you might want to keep the .xsd files outside of BaseX.
I don't remember if that worked with .xsd, but with .dtds and .css saving the files as raw instead of text enabled the resolution of dependents. Of course, if you need to xquery your schemas, saving them as raw is not an option.
Sorry for the delay, yes that's what I am doing. I didn't realize your issue was on the file system. I haven't had issue validating from the file system.
France
On Thu, Sep 11, 2014 at 2:57 PM, Hans-Juergen Rennau hrennau@yahoo.de wrote:
Thank you, France, for sharing this. But I am not sure if I understand well: it seems to me you copy DTDs from the database to the file system, before using them for validation, and this seems to point at a problem with resolving references which exists only if DTDs reside in the database, rather than in the file system - is this so? But my problems with XSD validation (involving imports) occur in the file system - databases are not involved.
Kind regards, Hans-Juergen
France Baril france.baril@architextus.com schrieb am 15:31 Donnerstag, 11.September 2014:
I just want to second that suggestion! In the mean time, Hans, what I do id this:
*let **$xsd-file-path* := *fix-validation:update-file-system-dtds-xsds*( *$xsd*) *return **validate:xsd*(*$file*, *$xsd-file-path*)
...
*declare function **fix-validation:update-file-system-dtds-xsds*(*$dtd* *as **xs:string*) *as **xs:string*{ *let **$to-file-sys*:= *for **$db-url* *in **db:list*('AppResources', 'dtds') *let* *$target-file* := *concat*('BaseX-DTDs/', *substring-after*(*$db-url*, 'dtds/')) (:let $debug := file:write('debug-dtd-2-file.xml', $target-file):) *let* *$dir* := *file:parent*(*$target-file*) *let* *$create-dir* := *if* (*file:exists*(*$dir*)) *then *() *else **file:create-dir*(*$dir*) *return **file:write*(*$target-file*, *db:open*( 'AppResources', *$db-url*))
(: Get the new file path for db equivalent $dtd :) *let* *$dtd-on-file-sys* := *concat*('BaseX-DTDs/', *substring-after*(*$dtd*, 'dtds/')) (: Return path to file system copy of the DTD :)
*return **$dtd-on-file-sys* };
Our XSDs change enough and our drop cycle is fast enough, that it's worth doing this on each validation to avoid confusion between schema versions in 2 locations. We need the .xsd in BaseX because we parse them for dynamic documentation. Otherwise, you might want to keep the .xsd files outside of BaseX.
I don't remember if that worked with .xsd, but with .dtds and .css saving the files as raw instead of text enabled the resolution of dependents. Of course, if you need to xquery your schemas, saving them as raw is not an option.
It would be wonderful if imports/includes could be automatically resolved - the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS" kann nicht als "type definition"-Komponente aufgelöst werden.</result1> <result2/> </results>
I think I have had this problem, and my recollection is
let $xsd:=doc( $xsdURI) validate:xsd-info($doc1, $xsd)
does not work, but
validate:xsd-info($doc1, $xsdURI)
does. I guess the problem with the first case is that the base-uri info is not getting though to the validation component. The bigger problem of includes in XSLT or XSD not working for sources in the database would seem to require a custom resolver class. Something like those discussed [1] and [2]
/Andy
[1] http://stackoverflow.com/questions/7236291/saxon-error-with-xslt-import-stat... [2] http://stackoverflow.com/questions/22538071/java-xml-schema-validator-with-c...
On 12 September 2014 16:31, Christian Grün christian.gruen@gmail.com wrote:
It would be wonderful if imports/includes could be automatically
resolved -
the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS"
kann
nicht als "type definition"-Komponente aufgelöst werden.</result1>
<result2/> </results>
It looks to me that in the case where the schema is presented as XML it is written to a temporary file [1] so relative includes/imports will not be resolved. /Andy
[1] https://github.com/BaseXdb/basex/blob/7e7aad58feae2f7e36bb1bcad9641cbf944f45...
On 12 September 2014 17:03, Andy Bunce bunce.andy@gmail.com wrote:
I think I have had this problem, and my recollection is
let $xsd:=doc( $xsdURI) validate:xsd-info($doc1, $xsd)
does not work, but
validate:xsd-info($doc1, $xsdURI)
does. I guess the problem with the first case is that the base-uri info is not getting though to the validation component. The bigger problem of includes in XSLT or XSD not working for sources in the database would seem to require a custom resolver class. Something like those discussed [1] and [2]
/Andy
[1] http://stackoverflow.com/questions/7236291/saxon-error-with-xslt-import-stat... [2] http://stackoverflow.com/questions/22538071/java-xml-schema-validator-with-c...
On 12 September 2014 16:31, Christian Grün christian.gruen@gmail.com wrote:
It would be wonderful if imports/includes could be automatically
resolved -
the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS"
kann
nicht als "type definition"-Komponente aufgelöst werden.</result1>
<result2/> </results>
It looks to me that in the case where the schema is presented as XML it is written to a temporary file [1] so relative includes/imports will not be resolved.
Thanks, and yes, this seems to be the problem. We could think about writing the temporary file to the query directory, but this may cause problems in certain environments.
C.
/Andy
[1] https://github.com/BaseXdb/basex/blob/7e7aad58feae2f7e36bb1bcad9641cbf944f45...
On 12 September 2014 17:03, Andy Bunce bunce.andy@gmail.com wrote:
I think I have had this problem, and my recollection is
let $xsd:=doc( $xsdURI) validate:xsd-info($doc1, $xsd)
does not work, but
validate:xsd-info($doc1, $xsdURI)
does. I guess the problem with the first case is that the base-uri info is not getting though to the validation component. The bigger problem of includes in XSLT or XSD not working for sources in the database would seem to require a custom resolver class. Something like those discussed [1] and [2]
/Andy
[1] http://stackoverflow.com/questions/7236291/saxon-error-with-xslt-import-stat... [2] http://stackoverflow.com/questions/22538071/java-xml-schema-validator-with-c...
On 12 September 2014 16:31, Christian Grün christian.gruen@gmail.com wrote:
It would be wonderful if imports/includes could be automatically resolved - the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS" kann nicht als "type definition"-Komponente aufgelöst werden.</result1> <result2/> </results>
Well it is not clear to me what should be expected to work when some XML is provided as a schema that has no clear location but contains relative uri references. It looks like in Hans-Juegen's case just supplying $xsdURI instead of doc( $xsdURI) would work. (or base-uri(doc( $xsdURI)) )
I think one could avoid the temporary file all together and use a resourceResolver [1] /Andy
[1] http://docs.oracle.com/javase/7/docs/api/javax/xml/validation/Validator.html...)
On 16 September 2014 16:29, Christian Grün christian.gruen@gmail.com wrote:
It looks to me that in the case where the schema is presented as XML it
is
written to a temporary file [1] so relative includes/imports will not be resolved.
Thanks, and yes, this seems to be the problem. We could think about writing the temporary file to the query directory, but this may cause problems in certain environments.
C.
/Andy
[1]
https://github.com/BaseXdb/basex/blob/7e7aad58feae2f7e36bb1bcad9641cbf944f45...
On 12 September 2014 17:03, Andy Bunce bunce.andy@gmail.com wrote:
I think I have had this problem, and my recollection is
let $xsd:=doc( $xsdURI) validate:xsd-info($doc1, $xsd)
does not work, but
validate:xsd-info($doc1, $xsdURI)
does. I guess the problem with the first case is that the base-uri info
is
not getting though to the validation component. The bigger problem of includes in XSLT or XSD not working for sources in the database would seem to require a custom resolver class. Something like those discussed [1] and [2]
/Andy
[1]
http://stackoverflow.com/questions/7236291/saxon-error-with-xslt-import-stat...
[2]
http://stackoverflow.com/questions/22538071/java-xml-schema-validator-with-c...
On 12 September 2014 16:31, Christian Grün christian.gruen@gmail.com wrote:
It would be wonderful if imports/includes could be automatically resolved - the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ',
$xsdURI)},
$doc1/*/(@*, node()) } }return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name
"communication:ShoppingCartFetchRS"
kann nicht als "type definition"-Komponente aufgelöst werden.</result1>
<result2/> </results>
Thank you very much, Christian, Andy and France!
I'm very glad that Andy revealed the simple cause & simple remedy - to use the XSD URI, rather than the XSD doc. This works fine for the scenarios I am currently dealing with, which are not performance critical. (I am not sure, but for reasons of performance it might nevertheless make sense to try and solve the problem arising with using the XSD doc.)
At any rate, I made a naive experiement - adding an @xml:base attribute to the XSD doc. Unfortunately, it did not help. Test code below.
Thanks again, good night gentlemen - Hans-Juergen
==================== let $docURI := 'addonFetchRQ.xml' let $xsdURI := 'external/AddonFetchRQ-1_0.xsd'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc := doc($docURI) let $docSloc := document { element {node-name($doc/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc/*/(@*, node()) } } let $xsdBase := document { element {node-name($xsd/*)} { attribute xml:base {$xsd/document-uri(.)}, $xsd/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc, $xsd)}</result1>, (: ERROR :) <result2>{validate:xsd-info($docURI, $xsd)}</result2>, (: ERROR :) <result3>{validate:xsd-info($docURI, $xsdBase)}</result3>, (: ERROR :) <result4>{validate:xsd-info($docSloc)}</result4>, (: OK :) <result5>{validate:xsd-info($docURI, $xsdURI)}</result5> (: OK, thanks ANDY :) }</results>
====================
Andy Bunce bunce.andy@gmail.com schrieb am 18:19 Dienstag, 16.September 2014:
Well it is not clear to me what should be expected to work when some XML is provided as a schema that has no clear location but contains relative uri references. It looks like in Hans-Juegen's case just supplying $xsdURI instead of doc( $xsdURI) would work. (or base-uri(doc( $xsdURI)) )
I think one could avoid the temporary file all together and use a resourceResolver [1] /Andy
[1] http://docs.oracle.com/javase/7/docs/api/javax/xml/validation/Validator.html...)
On 16 September 2014 16:29, Christian Grün christian.gruen@gmail.com wrote:
It looks to me that in the case where the schema is presented as XML it is
written to a temporary file [1] so relative includes/imports will not be resolved.
Thanks, and yes, this seems to be the problem. We could think about writing the temporary file to the query directory, but this may cause problems in certain environments.
C.
/Andy
[1] https://github.com/BaseXdb/basex/blob/7e7aad58feae2f7e36bb1bcad9641cbf944f45...
On 12 September 2014 17:03, Andy Bunce bunce.andy@gmail.com wrote:
I think I have had this problem, and my recollection is
let $xsd:=doc( $xsdURI) validate:xsd-info($doc1, $xsd)
does not work, but
validate:xsd-info($doc1, $xsdURI)
does. I guess the problem with the first case is that the base-uri info is not getting though to the validation component. The bigger problem of includes in XSLT or XSD not working for sources in the database would seem to require a custom resolver class. Something like those discussed [1] and [2]
/Andy
[1] http://stackoverflow.com/questions/7236291/saxon-error-with-xslt-import-stat... [2] http://stackoverflow.com/questions/22538071/java-xml-schema-validator-with-c...
On 12 September 2014 16:31, Christian Grün christian.gruen@gmail.com wrote:
It would be wonderful if imports/includes could be automatically resolved - the "validate" module could then be extremely useful!
As we use external code for processing XSD, I'm not quite sure if we can do something about this. - Could you possibly provide us with a self-contained example that we can use for testing?
Thanks, Christian
Best regards, Hans-Juegen
PS: My experiment:
====================================================== let $docURI := '... a document URI ...' let $xsdURI := '... the corresponding schema URI ...'
let $xsd := doc($xsdURI) let $tns := $xsd/xs:schema/@targetNamespace let $doc1 := doc($docURI) let $doc2 := document { element {node-name($doc1/*)} { attribute xsi:schemaLocation {concat($tns, ' ', $xsdURI)}, $doc1/*/(@*, node()) } } return <results>{ <result1>{validate:xsd-info($doc1, $xsd)}</result1>, <result2>{validate:xsd-info($doc2)}</result2> }</results> ======================================================
=>
<results> <result1>Fatal:src-resolve: Name "communication:ShoppingCartFetchRS" kann nicht als "type definition"-Komponente aufgelöst werden.</result1> <result2/> </results>
basex-talk@mailman.uni-konstanz.de
-
Andy Bunce -
Christian Grün -
France Baril -
Hans-Juergen Rennau