Re: [basex-talk] Unable to Make My Custom EXPath Module Work
I guess you’re right. We haven’t revised EXPath packaging for a long time now. Actually, I’m not sure how many people use it at all ;) Anyone out there? On Mon, Jan 24, 2022 at 4:08 PM Eliot Kimber <eliot.kimber@servicenow.com> wrote:
I did confirm that if the package @name URI matches the URI of a module, then the module is resolved, i.e.:
<package xmlns="http://expath.org/ns/pkg" name="http://servicenow.com/xquery/module/now-dita-utils" abbrev="now-xquery" version="0.1" spec="1.0"> <title>XQuery modules for ServiceNow Product Content processing and support</title> <xquery> <namespace>http://servicenow.com/xquery/module/database-from-git</namespace> <file>database-from-git.xqm</file> </xquery>
…
</package>
That implies that each module needs to be in a separate XAR file in order to also be in a separate namespace.
I don’t think that is consistent with the EXPath packaging spec.
In the description of the XQuery entry it says:
An XQuery library module is referenced by its namespace URI. Thus the xquery element associates a namespace URI to an XQuery file. An importing module just need to use an import statement of the form import module namespace xx = "<namespace-uri>";. An XQuery main module is associated a public URI. Usually an XQuery package will provide functions through library modules, but in some cases one can want to provide main modules as well. This implies to me that the value of the <namespace> element in the <xquery> is what should be used to resolve the package reference, not the package’s @name value, which simply serves to identify the package.
Is my analysis correct or have I misunderstood the package mechanism?
Cheers, E.
So maybe the better question is: How best to manage deployment of XQuery modules and top-level scripts from a source environment to a running remote server so that using scripts can import modules using just the module namespace? In my old DITA for Small Teams link manager application I was copying files into the BaseX repo location and then baking it into a Docker container. But for my current application we have a remote VM acting as a server, so I was looking to use XAR as a way to remotely install my application code to the running server as our internal infrastructure makes copying files directly to the server punishingly difficult. Thus my focus on using an EXPath package. For my immediate purposes I can update my build process to make separate XAR files for each library (I’m using an Ant script to make the packages and then install them to BaseX). I looked quickly at the code for EXPath package processing but I didn’t spent much time in it, but it probably wouldn’t be too hard to enhance it to behave as I expected. I could possibly pursue that if anyone else would use it. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen@gmail.com> Date: Monday, January 24, 2022 at 9:12 AM To: Eliot Kimber <eliot.kimber@servicenow.com>, BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Unable to Make My Custom EXPath Module Work [External Email] I guess you’re right. We haven’t revised EXPath packaging for a long time now. Actually, I’m not sure how many people use it at all ;) Anyone out there? On Mon, Jan 24, 2022 at 4:08 PM Eliot Kimber <eliot.kimber@servicenow.com> wrote:
I did confirm that if the package @name URI matches the URI of a module, then the module is resolved, i.e.:
<package xmlns="https://urldefense.com/v3/__http://expath.org/ns/pkg__;!!N4vogdjhuJM!X1Svs1N... " name="http://servicenow.com/xquery/module/now-dita-utils" abbrev="now-xquery" version="0.1" spec="1.0"> <title>XQuery modules for ServiceNow Product Content processing and support</title> <xquery> <namespace>http://servicenow.com/xquery/module/database-from-git</namespace> <file>database-from-git.xqm</file> </xquery>
…
</package>
That implies that each module needs to be in a separate XAR file in order to also be in a separate namespace.
I don’t think that is consistent with the EXPath packaging spec.
In the description of the XQuery entry it says:
An XQuery library module is referenced by its namespace URI. Thus the xquery element associates a namespace URI to an XQuery file. An importing module just need to use an import statement of the form import module namespace xx = "<namespace-uri>";. An XQuery main module is associated a public URI. Usually an XQuery package will provide functions through library modules, but in some cases one can want to provide main modules as well. This implies to me that the value of the <namespace> element in the <xquery> is what should be used to resolve the package reference, not the package’s @name value, which simply serves to identify the package.
Is my analysis correct or have I misunderstood the package mechanism?
Cheers, E.
How best to manage deployment of XQuery modules and top-level scripts from a source environment to a running remote server so that using scripts can import modules using just the module namespace?
If you don’t care about version conflicts and other potential dependency errors, you could use the following expression (the argument of fetch:binary can be an arbitrary URI): basex "archive:extract-to(db:option('repopath'), fetch:binary('....zip'))"
I looked quickly at the code for EXPath package processing but I didn’t spent much time in it, but it probably wouldn’t be too hard to enhance it to behave as I expected. I could possibly pursue that if anyone else would use it.
Thanks for the offer, that would certainly be appreciated.
Hi, I have used EXPath packaging, often to package some Java code and a calling xquery module [1].
the URL of the expath-pkg.xml library module which will be imported from outside must match the name of the package In my experience this is not required, see [2] for example. I think the problem may be that only the first xquery referenced in expath-pkg works. I believe your example should work.
One problem I had when trying to use the repository feature for more complex packaging occurs when the package exposes some xqm and that xqm imports a utility library xqm that is in the package, but not exposed in the expath-pkg.xml, and the Xquery that imports the packaged xqm also imports a different version of the utility library. Conflicts can arise between the two versions. They should be isolated [3] /Andy [1] https://github.com/expkg-zone58 [2] https://github.com/expkg-zone58/ex-thumbnailator/blob/master/src/main/expath... [3] https://github.com/BaseXdb/basex/issues/2048 On Mon, 24 Jan 2022 at 17:43, Christian Grün <christian.gruen@gmail.com> wrote:
How best to manage deployment of XQuery modules and top-level scripts from a source environment to a running remote server so that using scripts can import modules using just the module namespace?
If you don’t care about version conflicts and other potential dependency errors, you could use the following expression (the argument of fetch:binary can be an arbitrary URI):
basex "archive:extract-to(db:option('repopath'), fetch:binary('....zip'))"
I looked quickly at the code for EXPath package processing but I didn’t spent much time in it, but it probably wouldn’t be too hard to enhance it to behave as I expected. I could possibly pursue that if anyone else would use it.
Thanks for the offer, that would certainly be appreciated.
I see that I can just directly install the individual module files, i.e.: repo install file:/Users/eliot.kimber/git/dita-build-tools/src/main/xquery/modules/now-dita-utils.xqm And that works, so I think I was just overthinking the problem. In the context of an Ant deployment script on the server can just run the repo install command or, as Christian suggests, use a zip package that reflects the result of this install. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen@gmail.com> Date: Monday, January 24, 2022 at 11:43 AM To: Eliot Kimber <eliot.kimber@servicenow.com> Cc: BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Unable to Make My Custom EXPath Module Work [External Email]
How best to manage deployment of XQuery modules and top-level scripts from a source environment to a running remote server so that using scripts can import modules using just the module namespace?
If you don’t care about version conflicts and other potential dependency errors, you could use the following expression (the argument of fetch:binary can be an arbitrary URI): basex "archive:extract-to(db:option('repopath'), fetch:binary('....zip'))"
I looked quickly at the code for EXPath package processing but I didn’t spent much time in it, but it probably wouldn’t be too hard to enhance it to behave as I expected. I could possibly pursue that if anyone else would use it.
Thanks for the offer, that would certainly be appreciated.
I see that I can just directly install the individual module files, i.e.: repo install file:/Users/eliot.kimber/git/dita-build-tools/src/main/xquery/modules/now-dita-utils.xqm
True! I overlooked that one. With the repo:install, it can optionally be combined with a FLWOR expression.
Here’s what I have working from my little Ant deployment script: <target name="build-basex-zip-package" depends="init, clean" description="Prepares the files for the BaseX now-xquery repo Zip module package" > <mkdir dir="${basex.zip.package.dir}/com/servicenow/xquery/module"/> <!-- The BaseX convention is that the package namespace URI is translated to a directory path, i.e., com/servicenow/xquery/module/ that then contains the module file. --> <copy todir="${basex.zip.package.dir}/com/servicenow/xquery/module"> <fileset dir="${xquery.src}/modules"> <include name="*"/> </fileset> </copy> </target> <target name="package-basex-zip-package" depends="build-basex-zip-package" description="Packages the now-xquery package as Zip file" > <zip basedir="${basex.zip.package.dir}" destfile="${target.dir}/${basex.zip.package.name}.zip" > <include name="**/*"/> </zip> </target> <target name="deploy-now-modules-to-basex" depends="package-basex-zip-package" description="Deploys the now-xquery module package to the BaseX repo directory" > <exec executable="${basex.command}" > <arg value="-c"/> <arg value="xquery archive:extract-to(db:option('repopath'), fetch:binary('${target.dir}/${basex.zip.package.name}.zip'))"/> </exec> <exec executable="${basex.command}" > <arg value="-c"/> <arg value="repo list"/> </exec> </target> I’m making a Zip file to allow for a future where the Zip file is fetched from somewhere else but for now the source is on the same server as BaseX so I just deploy from there. Easy peasy. Ideally I’d be doing this via Maven but I haven’t quite gotten there yet. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow> From: Christian Grün <christian.gruen@gmail.com> Date: Monday, January 24, 2022 at 1:31 PM To: Eliot Kimber <eliot.kimber@servicenow.com> Cc: BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Unable to Make My Custom EXPath Module Work [External Email]
I see that I can just directly install the individual module files, i.e.: repo install file:/Users/eliot.kimber/git/dita-build-tools/src/main/xquery/modules/now-dita-utils.xqm
True! I overlooked that one. With the repo:install, it can optionally be combined with a FLWOR expression.
participants (3)
-
Andy Bunce -
Christian Grün -
Eliot Kimber