Re: [basex-talk] Basex Docker Container: How to Control Options
Hi Eliot, Did you try to specify the option in the local section of the .basex file (at the bottom, after all other options)? Best, Christian Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber@contrext.com>: I’ve worked around the configuration file problem by setting the options on the Java command line by setting BASEX_JVM in my Dockerfile: ENV BASEX_JVM="-Dorg.basex.CHOP=false -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true" That seems to work. I guess I could also set this environment variable in my docker-compose.yml file, although except for CATFILE, these are invariant for this application. Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:44 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber@contrext.com> wrote: Perhaps a more useful question is: is there a workaround for this? That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options set when I load files into a database, which I’m doing through the remote API. Is there either a separate options file that I should be using or a way to specify these options at document load time? Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:23 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I dug into the code and I think this is a code bug but I don’t understand the overall flow well enough to know where the bug is, but I suspect it’s a simple change. If I debug the BasexGUI class and have a ~/.basex file that includes CATFILE=foo it will fail with an unknown option error because the option class being used is StaticOptions and StaticOptions does not define CATFILE as an option. The CATFILE option, along with a few others, is defined on MainOptions: private static final Option<?>[] INHERIT = { CHOP, INTPARSE, STRIPNS, DTD, XINCLUDE, CATFILE }; Looking at BaseXHTTP it uses HTTPContext which only uses StaticOptions, not MainOptions, which means it can never allow the INHERIT options. So something must have changed from 8.1 ish to now to change how options are handled since these options worked before I moved to the latest BaseX container and versions. But there definitely seems to be an oversight in the initial handling of the options file in the BaseXHTTP server (and in the BaseXGUI as well since I get the same behavior there as well). I would make a pull request but I don’t presume to know what policies or general practices underlie the current options mechanism deisgn. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 10:20 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I found my original mail thread from January 2016: https://mailman.uni-konstanz.de/pipermail/basex-talk/2016- January/010171.html Basically I had to append my options to the existing .basex file rather than simply provide a file. However, that isn’t an option here because there is no existing /srv/.basex file (the place where the file needs to be). As an experiment I tried doing this: RUN touch /srv/.basex && \ echo "DTD=true" >> .basex && \ echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> .basex && \ echo "SKIPCORRUPT=true" >> .basex && \ echo "CHOP=false" >> .basex && \ echo "DBPATH=/srv/BaseXData" >> .basex && \ echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ chown basex /srv/.basex RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo "=======" And I get this from the build command: /srv/.basex: =========== DTD=true CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml SKIPCORRUPT=true CHOP=false DBPATH=/srv/BaseXData REPOPATH=/srv/BaseXRepo WEBPATH=/srv/BaseXWeb ======= When I start up this container I get this: /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, no difference in result (but the messages reflect the order change). So it must be something about how these particular options are processed rather than an issue with the .basex file, at least as far as I can see. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 3:29 PM, "Eliot Kimber" <basex-talk-bounces@mailman. uni-konstanz.de on behalf of ekimber@contrext.com> wrote: Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup. Made some progress but something is still not working. I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable. However, if I then add this: COPY basex/dot_basex /srv/.basex As before, on startup I get this: docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was. Cheers, E. -- Eliot Kimber http://contrext.com On 3/26/17, 1:40 PM, "Eliot Kimber" < basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com
Yes, the options were at the end. But looking at the code I don’t see how that could have mattered. Perhaps I missed a detail? Cheers, E. -- Eliot Kimber http://contrext.com From: Christian Grün <christian.gruen@gmail.com> Date: Sunday, April 2, 2017 at 11:38 AM To: Eliot Kimber <ekimber@contrext.com> Cc: "basex-talk@mailman.uni-konstanz.de" <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Basex Docker Container: How to Control Options Hi Eliot, Did you try to specify the option in the local section of the .basex file (at the bottom, after all other options)? Best, Christian Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber@contrext.com>: I’ve worked around the configuration file problem by setting the options on the Java command line by setting BASEX_JVM in my Dockerfile: ENV BASEX_JVM="-Dorg.basex.CHOP=false -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true" That seems to work. I guess I could also set this environment variable in my docker-compose.yml file, although except for CATFILE, these are invariant for this application. Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:44 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: Perhaps a more useful question is: is there a workaround for this? That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options set when I load files into a database, which I’m doing through the remote API. Is there either a separate options file that I should be using or a way to specify these options at document load time? Cheers, E. -- Eliot Kimber http://contrext.com On 4/1/17, 10:23 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I dug into the code and I think this is a code bug but I don’t understand the overall flow well enough to know where the bug is, but I suspect it’s a simple change. If I debug the BasexGUI class and have a ~/.basex file that includes CATFILE=foo it will fail with an unknown option error because the option class being used is StaticOptions and StaticOptions does not define CATFILE as an option. The CATFILE option, along with a few others, is defined on MainOptions: private static final Option<?>[] INHERIT = { CHOP, INTPARSE, STRIPNS, DTD, XINCLUDE, CATFILE }; Looking at BaseXHTTP it uses HTTPContext which only uses StaticOptions, not MainOptions, which means it can never allow the INHERIT options. So something must have changed from 8.1 ish to now to change how options are handled since these options worked before I moved to the latest BaseX container and versions. But there definitely seems to be an oversight in the initial handling of the options file in the BaseXHTTP server (and in the BaseXGUI as well since I get the same behavior there as well). I would make a pull request but I don’t presume to know what policies or general practices underlie the current options mechanism deisgn. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 10:20 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I found my original mail thread from January 2016: https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-January/010171.htm... Basically I had to append my options to the existing .basex file rather than simply provide a file. However, that isn’t an option here because there is no existing /srv/.basex file (the place where the file needs to be). As an experiment I tried doing this: RUN touch /srv/.basex && \ echo "DTD=true" >> .basex && \ echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> .basex && \ echo "SKIPCORRUPT=true" >> .basex && \ echo "CHOP=false" >> .basex && \ echo "DBPATH=/srv/BaseXData" >> .basex && \ echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ chown basex /srv/.basex RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo "=======" And I get this from the build command: /srv/.basex: =========== DTD=true CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml SKIPCORRUPT=true CHOP=false DBPATH=/srv/BaseXData REPOPATH=/srv/BaseXRepo WEBPATH=/srv/BaseXWeb ======= When I start up this container I get this: /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, no difference in result (but the messages reflect the order change). So it must be something about how these particular options are processed rather than an issue with the .basex file, at least as far as I can see. Cheers, Eliot -- Eliot Kimber http://contrext.com On 3/26/17, 3:29 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup. Made some progress but something is still not working. I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable. However, if I then add this: COPY basex/dot_basex /srv/.basex As before, on startup I get this: docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was. Cheers, E. -- Eliot Kimber http://contrext.com On 3/26/17, 1:40 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote: I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs. As part of this setup I need to add several additional options. I also want to include the dba app. Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used. It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them. Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something. In my Dockerfile I have: COPY basex/dot_basex /srv/.basex If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work. Thanks, Eliot -- Eliot Kimber http://contrext.com
Hi Eliot, If you start with a new version of BaseX, in which the .basex configuration file is still empty or non-existent, there may be no "Local Options" comment. I hope the revised Wiki makes this more obvious [1]. Best, Christian [1] http://docs.basex.org/wiki/Options On Sun, Apr 2, 2017 at 12:45 PM, Eliot Kimber <ekimber@contrext.com> wrote:
Yes, the options were at the end. But looking at the code I don’t see how that could have mattered. Perhaps I missed a detail?
Cheers,
E.
--
Eliot Kimber
From: Christian Grün <christian.gruen@gmail.com> Date: Sunday, April 2, 2017 at 11:38 AM To: Eliot Kimber <ekimber@contrext.com> Cc: "basex-talk@mailman.uni-konstanz.de" <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Basex Docker Container: How to Control Options
Hi Eliot,
Did you try to specify the option in the local section of the .basex file (at the bottom, after all other options)?
Best,
Christian
Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber@contrext.com>:
I’ve worked around the configuration file problem by setting the options on the Java command line by setting BASEX_JVM in my Dockerfile:
ENV BASEX_JVM="-Dorg.basex.CHOP=false -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true"
That seems to work. I guess I could also set this environment variable in my docker-compose.yml file, although except for CATFILE, these are invariant for this application.
Cheers,
E.
-- Eliot Kimber http://contrext.com
On 4/1/17, 10:44 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote:
Perhaps a more useful question is: is there a workaround for this?
That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options set when I load files into a database, which I’m doing through the remote API. Is there either a separate options file that I should be using or a way to specify these options at document load time?
Cheers,
E.
-- Eliot Kimber http://contrext.com
On 4/1/17, 10:23 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote:
I dug into the code and I think this is a code bug but I don’t understand the overall flow well enough to know where the bug is, but I suspect it’s a simple change.
If I debug the BasexGUI class and have a ~/.basex file that includes CATFILE=foo it will fail with an unknown option error because the option class being used is StaticOptions and StaticOptions does not define CATFILE as an option.
The CATFILE option, along with a few others, is defined on MainOptions:
private static final Option<?>[] INHERIT = { CHOP, INTPARSE, STRIPNS, DTD, XINCLUDE, CATFILE };
Looking at BaseXHTTP it uses HTTPContext which only uses StaticOptions, not MainOptions, which means it can never allow the INHERIT options.
So something must have changed from 8.1 ish to now to change how options are handled since these options worked before I moved to the latest BaseX container and versions.
But there definitely seems to be an oversight in the initial handling of the options file in the BaseXHTTP server (and in the BaseXGUI as well since I get the same behavior there as well).
I would make a pull request but I don’t presume to know what policies or general practices underlie the current options mechanism deisgn.
Cheers,
Eliot -- Eliot Kimber http://contrext.com
On 3/26/17, 10:20 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote:
I found my original mail thread from January 2016:
https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-January/010171.htm...
Basically I had to append my options to the existing .basex file rather than simply provide a file.
However, that isn’t an option here because there is no existing /srv/.basex file (the place where the file needs to be).
As an experiment I tried doing this:
RUN touch /srv/.basex && \ echo "DTD=true" >> .basex && \ echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> .basex && \ echo "SKIPCORRUPT=true" >> .basex && \ echo "CHOP=false" >> .basex && \ echo "DBPATH=/srv/BaseXData" >> .basex && \ echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ chown basex /srv/.basex RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo "======="
And I get this from the build command:
/srv/.basex: =========== DTD=true CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml SKIPCORRUPT=true CHOP=false DBPATH=/srv/BaseXData REPOPATH=/srv/BaseXRepo WEBPATH=/srv/BaseXWeb =======
When I start up this container I get this:
/srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file.
Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, no difference in result (but the messages reflect the order change).
So it must be something about how these particular options are processed rather than an issue with the .basex file, at least as far as I can see.
Cheers,
Eliot
-- Eliot Kimber http://contrext.com
On 3/26/17, 3:29 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote:
Discovered that if I use USER root in the Dockerfile I have to then do USER basex before the end so that the container will be running as basex, otherwise it does not start up correctly. I also have to chown the /srv/.basex file to basex otherwise it can’t overwrite it on startup.
Made some progress but something is still not working.
I now have a Dockerfile based on basex/dba that results in a running system with the both the dba and my custom app runnable.
However, if I then add this:
COPY basex/dot_basex /srv/.basex
As before, on startup I get this:
docker run --name linkmgr-test -p 8984:8984 link-manager-test /srv/.basex: Unknown option 'CATFILE'. /srv/.basex: Unknown option 'DTD'. /srv/.basex: Unknown option 'SKIPCORRUPT'. /srv/.basex: Unknown option 'CHOP'. /srv/.basex: writing new configuration file. Creating /srv/BaseXWeb/WEB-INF/web.xml Creating /srv/BaseXWeb/WEB-INF/jetty.xml
I think this Unknown option this is actually an issue with the file itself but I don’t recall what the exact cause and/or solution was.
Cheers,
E.
-- Eliot Kimber http://contrext.com
On 3/26/17, 1:40 PM, "Eliot Kimber" <basex-talk-bounces@mailman.uni-konstanz.de on behalf of ekimber@contrext.com> wrote:
I’m trying to set up a Docker container with my own Web app using the latest official containers as a base and following the instructions in the latest BaseX docs.
As part of this setup I need to add several additional options.
I also want to include the dba app.
Following the instructions in the docs I’m putting my additional options in /srv/.basex and these take effect. However, if I use the basex/dba image as a base, the other options, such as the custom REPOPATH and WEBPATH settings do not get used.
It’s not clear to me from looking at the various Dockerfiles how the dba container sets the options and how I can then add to them without overwriting them.
Of course I can duplicate the options in my Dockerfile but it seems like I should be able to add my options additively but I’m not seeing how to do it, so I feel I’m missing something.
In my Dockerfile I have:
COPY basex/dot_basex /srv/.basex
If I don’t include this line then the container otherwise works in that my Web app is there but it requires these additional options in order to function properly. If I include it then the effective options reflect only my changes and not those required for the custom Web apps to work.
Thanks,
Eliot
-- Eliot Kimber http://contrext.com
Are you really saying that the “# Local options” comment is a necessary part of the configuration file? Really? Because I don’t think a comment should ever a required part of any syntax… If the configuration file needs have labeled components use YAML or XML or something. Cheers, e. -- Eliot Kimber http://contrext.com On 4/2/17, 2:25 PM, "Christian Grün" <christian.gruen@gmail.com> wrote: Hi Eliot, If you start with a new version of BaseX, in which the .basex configuration file is still empty or non-existent, there may be no "Local Options" comment. I hope the revised Wiki makes this more obvious [1]. Best, Christian [1] http://docs.basex.org/wiki/Options On Sun, Apr 2, 2017 at 12:45 PM, Eliot Kimber <ekimber@contrext.com> wrote: > Yes, the options were at the end. But looking at the code I don’t see how > that could have mattered. Perhaps I missed a detail? > > > > Cheers, > > > > E. > > > > -- > > Eliot Kimber > > http://contrext.com > > > > > > > > From: Christian Grün <christian.gruen@gmail.com> > Date: Sunday, April 2, 2017 at 11:38 AM > To: Eliot Kimber <ekimber@contrext.com> > Cc: "basex-talk@mailman.uni-konstanz.de" > <basex-talk@mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Basex Docker Container: How to Control Options > > > > Hi Eliot, > > > > Did you try to specify the option in the local section of the .basex file > (at the bottom, after all other options)? > > > > Best, > > Christian > > > > > > > > > > > > Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber@contrext.com>: > > I’ve worked around the configuration file problem by setting the options on > the Java command line by setting BASEX_JVM in my Dockerfile: > > ENV BASEX_JVM="-Dorg.basex.CHOP=false > -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml > -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true" > > That seems to work. I guess I could also set this environment variable in my > docker-compose.yml file, although except for CATFILE, these are invariant > for this application. > > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > On 4/1/17, 10:44 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > Perhaps a more useful question is: is there a workaround for this? > > That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options > set when I load files into a database, which I’m doing through the remote > API. Is there either a separate options file that I should be using or a way > to specify these options at document load time? > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > > On 4/1/17, 10:23 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I dug into the code and I think this is a code bug but I don’t > understand the overall flow well enough to know where the bug is, but I > suspect it’s a simple change. > > If I debug the BasexGUI class and have a ~/.basex file that includes > CATFILE=foo it will fail with an unknown option error because the option > class being used is StaticOptions and StaticOptions does not define CATFILE > as an option. > > The CATFILE option, along with a few others, is defined on > MainOptions: > > private static final Option<?>[] INHERIT = { CHOP, INTPARSE, > STRIPNS, DTD, XINCLUDE, CATFILE }; > > Looking at BaseXHTTP it uses HTTPContext which only uses > StaticOptions, not MainOptions, which means it can never allow the INHERIT > options. > > So something must have changed from 8.1 ish to now to change how > options are handled since these options worked before I moved to the latest > BaseX container and versions. > > But there definitely seems to be an oversight in the initial > handling of the options file in the BaseXHTTP server (and in the BaseXGUI as > well since I get the same behavior there as well). > > I would make a pull request but I don’t presume to know what > policies or general practices underlie the current options mechanism deisgn. > > Cheers, > > Eliot > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 10:20 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I found my original mail thread from January 2016: > > > https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-January/010171.htm... > > Basically I had to append my options to the existing .basex file > rather than simply provide a file. > > However, that isn’t an option here because there is no existing > /srv/.basex file (the place where the file needs to be). > > As an experiment I tried doing this: > > RUN touch /srv/.basex && \ > echo "DTD=true" >> .basex && \ > echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> > .basex && \ > echo "SKIPCORRUPT=true" >> .basex && \ > echo "CHOP=false" >> .basex && \ > echo "DBPATH=/srv/BaseXData" >> .basex && \ > echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ > echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ > chown basex /srv/.basex > RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo > "=======" > > And I get this from the build command: > > /srv/.basex: > =========== > DTD=true > CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml > SKIPCORRUPT=true > CHOP=false > DBPATH=/srv/BaseXData > REPOPATH=/srv/BaseXRepo > WEBPATH=/srv/BaseXWeb > ======= > > When I start up this container I get this: > > /srv/.basex: Unknown option 'DTD'. > /srv/.basex: Unknown option 'CATFILE'. > /srv/.basex: Unknown option 'SKIPCORRUPT'. > /srv/.basex: Unknown option 'CHOP'. > /srv/.basex: writing new configuration file. > > Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, > and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, > no difference in result (but the messages reflect the order change). > > So it must be something about how these particular options are > processed rather than an issue with the .basex file, at least as far as I > can see. > > Cheers, > > Eliot > > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 3:29 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > Discovered that if I use USER root in the Dockerfile I have > to then do USER basex before the end so that the container will be running > as basex, otherwise it does not start up correctly. I also have to chown the > /srv/.basex file to basex otherwise it can’t overwrite it on startup. > > Made some progress but something is still not working. > > I now have a Dockerfile based on basex/dba that results in a > running system with the both the dba and my custom app runnable. > > However, if I then add this: > > COPY basex/dot_basex /srv/.basex > > As before, on startup I get this: > > docker run --name linkmgr-test -p 8984:8984 > link-manager-test > /srv/.basex: Unknown option 'CATFILE'. > /srv/.basex: Unknown option 'DTD'. > /srv/.basex: Unknown option 'SKIPCORRUPT'. > /srv/.basex: Unknown option 'CHOP'. > /srv/.basex: writing new configuration file. > Creating /srv/BaseXWeb/WEB-INF/web.xml > Creating /srv/BaseXWeb/WEB-INF/jetty.xml > > > I think this Unknown option this is actually an issue with > the file itself but I don’t recall what the exact cause and/or solution was. > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 1:40 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I’m trying to set up a Docker container with my own Web > app using the latest official containers as a base and following the > instructions in the latest BaseX docs. > > As part of this setup I need to add several additional > options. > > I also want to include the dba app. > > Following the instructions in the docs I’m putting my > additional options in /srv/.basex and these take effect. However, if I use > the basex/dba image as a base, the other options, such as the custom > REPOPATH and WEBPATH settings do not get used. > > It’s not clear to me from looking at the various > Dockerfiles how the dba container sets the options and how I can then add to > them without overwriting them. > > Of course I can duplicate the options in my Dockerfile > but it seems like I should be able to add my options additively but I’m not > seeing how to do it, so I feel I’m missing something. > > In my Dockerfile I have: > > COPY basex/dot_basex /srv/.basex > > If I don’t include this line then the container > otherwise works in that my Web app is there but it requires these additional > options in order to function properly. If I include it then the effective > options reflect only my changes and not those required for the custom Web > apps to work. > > Thanks, > > Eliot > > -- > Eliot Kimber > http://contrext.com > > > > > > > > > > > > > > > > > > > > > > > >
Hi, side note: I see a level one heading there (markdown interpretes # as <h1> ), not a comment. Anyways, thanks for this conversation - this sheds some light on similar issues I had. Br, Max 2017-04-02 20:36 GMT+02:00 Eliot Kimber <ekimber@contrext.com>:
Are you really saying that the “# Local options” comment is a necessary part of the configuration file? Really? Because I don’t think a comment should ever a required part of any syntax…
If the configuration file needs have labeled components use YAML or XML or something.
Cheers,
e. -- Eliot Kimber http://contrext.com
On 4/2/17, 2:25 PM, "Christian Grün" <christian.gruen@gmail.com> wrote:
Hi Eliot,
If you start with a new version of BaseX, in which the .basex configuration file is still empty or non-existent, there may be no "Local Options" comment. I hope the revised Wiki makes this more obvious [1].
Best, Christian
[1] http://docs.basex.org/wiki/Options
On Sun, Apr 2, 2017 at 12:45 PM, Eliot Kimber <ekimber@contrext.com> wrote: > Yes, the options were at the end. But looking at the code I don’t see how > that could have mattered. Perhaps I missed a detail? > > > > Cheers, > > > > E. > > > > -- > > Eliot Kimber > > http://contrext.com > > > > > > > > From: Christian Grün <christian.gruen@gmail.com> > Date: Sunday, April 2, 2017 at 11:38 AM > To: Eliot Kimber <ekimber@contrext.com> > Cc: "basex-talk@mailman.uni-konstanz.de" > <basex-talk@mailman.uni-konstanz.de> > Subject: Re: [basex-talk] Basex Docker Container: How to Control Options > > > > Hi Eliot, > > > > Did you try to specify the option in the local section of the .basex file > (at the bottom, after all other options)? > > > > Best, > > Christian > > > > > > > > > > > > Am 02.04.2017 10:52 schrieb "Eliot Kimber" <ekimber@contrext.com>: > > I’ve worked around the configuration file problem by setting the options on > the Java command line by setting BASEX_JVM in my Dockerfile: > > ENV BASEX_JVM="-Dorg.basex.CHOP=false > -Dorg.basex.CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml > -Dorg.basex.DTD=true -Dorg.basex.SKIPCORRUPT=true" > > That seems to work. I guess I could also set this environment variable in my > docker-compose.yml file, although except for CATFILE, these are invariant > for this application. > > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > On 4/1/17, 10:44 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > Perhaps a more useful question is: is there a workaround for this? > > That is, I need to have the DTD, CATFILE, CHOP, and SKIPCORRUPT options > set when I load files into a database, which I’m doing through the remote > API. Is there either a separate options file that I should be using or a way > to specify these options at document load time? > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > > On 4/1/17, 10:23 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I dug into the code and I think this is a code bug but I don’t > understand the overall flow well enough to know where the bug is, but I > suspect it’s a simple change. > > If I debug the BasexGUI class and have a ~/.basex file that includes > CATFILE=foo it will fail with an unknown option error because the option > class being used is StaticOptions and StaticOptions does not define CATFILE > as an option. > > The CATFILE option, along with a few others, is defined on > MainOptions: > > private static final Option<?>[] INHERIT = { CHOP, INTPARSE, > STRIPNS, DTD, XINCLUDE, CATFILE }; > > Looking at BaseXHTTP it uses HTTPContext which only uses > StaticOptions, not MainOptions, which means it can never allow the INHERIT > options. > > So something must have changed from 8.1 ish to now to change how > options are handled since these options worked before I moved to the latest > BaseX container and versions. > > But there definitely seems to be an oversight in the initial > handling of the options file in the BaseXHTTP server (and in the BaseXGUI as > well since I get the same behavior there as well). > > I would make a pull request but I don’t presume to know what > policies or general practices underlie the current options mechanism deisgn. > > Cheers, > > Eliot > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 10:20 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I found my original mail thread from January 2016: > > > https://mailman.uni-konstanz.de/pipermail/basex-talk/2016-January/010171.htm... > > Basically I had to append my options to the existing .basex file > rather than simply provide a file. > > However, that isn’t an option here because there is no existing > /srv/.basex file (the place where the file needs to be). > > As an experiment I tried doing this: > > RUN touch /srv/.basex && \ > echo "DTD=true" >> .basex && \ > echo "CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml" >> > .basex && \ > echo "SKIPCORRUPT=true" >> .basex && \ > echo "CHOP=false" >> .basex && \ > echo "DBPATH=/srv/BaseXData" >> .basex && \ > echo "REPOPATH=/srv/BaseXRepo" >> .basex && \ > echo "WEBPATH=/srv/BaseXWeb" >> .basex && \ > chown basex /srv/.basex > RUN echo "/srv/.basex:\n===========" && cat /srv/.basex && echo > "=======" > > And I get this from the build command: > > /srv/.basex: > =========== > DTD=true > CATFILE=/opt/dita-ot/DITA-OT/catalog-dita.xml > SKIPCORRUPT=true > CHOP=false > DBPATH=/srv/BaseXData > REPOPATH=/srv/BaseXRepo > WEBPATH=/srv/BaseXWeb > ======= > > When I start up this container I get this: > > /srv/.basex: Unknown option 'DTD'. > /srv/.basex: Unknown option 'CATFILE'. > /srv/.basex: Unknown option 'SKIPCORRUPT'. > /srv/.basex: Unknown option 'CHOP'. > /srv/.basex: writing new configuration file. > > Note that it’s only complaining about DTD, CATFILE, SKIPCORRUPT, > and CHOP, but not DBPATH, REPOPATH, and WEBPATH. I tried reordering things, > no difference in result (but the messages reflect the order change). > > So it must be something about how these particular options are > processed rather than an issue with the .basex file, at least as far as I > can see. > > Cheers, > > Eliot > > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 3:29 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > Discovered that if I use USER root in the Dockerfile I have > to then do USER basex before the end so that the container will be running > as basex, otherwise it does not start up correctly. I also have to chown the > /srv/.basex file to basex otherwise it can’t overwrite it on startup. > > Made some progress but something is still not working. > > I now have a Dockerfile based on basex/dba that results in a > running system with the both the dba and my custom app runnable. > > However, if I then add this: > > COPY basex/dot_basex /srv/.basex > > As before, on startup I get this: > > docker run --name linkmgr-test -p 8984:8984 > link-manager-test > /srv/.basex: Unknown option 'CATFILE'. > /srv/.basex: Unknown option 'DTD'. > /srv/.basex: Unknown option 'SKIPCORRUPT'. > /srv/.basex: Unknown option 'CHOP'. > /srv/.basex: writing new configuration file. > Creating /srv/BaseXWeb/WEB-INF/web.xml > Creating /srv/BaseXWeb/WEB-INF/jetty.xml > > > I think this Unknown option this is actually an issue with > the file itself but I don’t recall what the exact cause and/or solution was. > > Cheers, > > E. > > -- > Eliot Kimber > http://contrext.com > > > > On 3/26/17, 1:40 PM, "Eliot Kimber" > <basex-talk-bounces@mailman.uni-konstanz.de on behalf of > ekimber@contrext.com> wrote: > > I’m trying to set up a Docker container with my own Web > app using the latest official containers as a base and following the > instructions in the latest BaseX docs. > > As part of this setup I need to add several additional > options. > > I also want to include the dba app. > > Following the instructions in the docs I’m putting my > additional options in /srv/.basex and these take effect. However, if I use > the basex/dba image as a base, the other options, such as the custom > REPOPATH and WEBPATH settings do not get used. > > It’s not clear to me from looking at the various > Dockerfiles how the dba container sets the options and how I can then add to > them without overwriting them. > > Of course I can duplicate the options in my Dockerfile > but it seems like I should be able to add my options additively but I’m not > seeing how to do it, so I feel I’m missing something. > > In my Dockerfile I have: > > COPY basex/dot_basex /srv/.basex > > If I don’t include this line then the container > otherwise works in that my Web app is there but it requires these additional > options in order to function properly. If I include it then the effective > options reflect only my changes and not those required for the custom Web > apps to work. > > Thanks, > > Eliot > > -- > Eliot Kimber > http://contrext.com > > > > > > > > > > > > > > > > > > > > > > > >
participants (3)
-
Christian Grün -
Eliot Kimber -
Maximilian Gärber