First steps with client/server on Linux - bulk import; error message in dba
Hi all, i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation? First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents.... When accessing the same db using WebDAV (Oxygen), one can nicelly navigate using the collection structure. With REST, adding the collection names to the URL will restrict the output to the db content of those collections. But without such a restriction, it would be great if the REST web interface would just return the first level of collections without any XML resource entries. Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections? I've found that Oxygen "translates" subdirectories into collections when using the "import directories" feature. Is there somehing similar using the command line client? And another issue: The "Logs" page of the DBA webapp shows the following error warning: "Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..." Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?). Yours sincerely. Florian
Am 10.05.2022 um 16:29 schrieb Florian Schmitt:
... And another issue: The "Logs" page of the DBA webapp shows the following error warning:
"Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..."
Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?).
After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry: [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R... That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
Thanks for the observation, Florian.
After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry:
I hope the latest snapshot resolves the Yajsw/.logs bug [1].
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at
I’ve added Vincent Lizzi to the conversation, who has gratefully added the documentation on YAJSW: Maybe it would be sensible to move the wrapper-basex.log file out of the .logs directory? I don’t expect any complications if it’s stored in data/. I’ll additionally check if we can prevent admin:logs from crashing if the log input does not conform to the usual pattern. Best, Christian [1] https://github.com/BaseXdb/basex/issues/2105
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R...
That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
Hi Christian and Florian, I recall that the DBA web interface didn't like the YAJSW log files, although that didn't seem surprising because the YAJSW log files are in a different format than BaseX log files. It's been a while since I tried this so I don't have the details now. YAJSW can be configured to save its log files in a different location. Kind regards, Vincent _____________________________________________ Vincent M. Lizzi Head of Information Standards | Taylor & Francis Group vincent.lizzi@taylorandfrancis.com<mailto:vincent.lizzi@taylorandfrancis.com> Information Classification: General From: Christian Grün <christian.gruen@gmail.com> Sent: Thursday, May 12, 2022 8:03 AM To: Florian Schmitt <ml-basex@florian-schmitt.net>; Lizzi, Vincent <Vincent.Lizzi@taylorandfrancis.com> Cc: BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] First steps with client/server on Linux - bulk import; error message in dba Thanks for the observation, Florian.
After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry:
I hope the latest snapshot resolves the Yajsw/.logs bug [1].
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at
I've added Vincent Lizzi to the conversation, who has gratefully added the documentation on YAJSW: Maybe it would be sensible to move the wrapper-basex.log file out of the .logs directory? I don't expect any complications if it's stored in data/. I'll additionally check if we can prevent admin:logs from crashing if the log input does not conform to the usual pattern. Best, Christian [1] https://github.com/BaseXdb/basex/issues/2105<https://github.com/BaseXdb/basex/issues/2105>
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R...
That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
Thanks, Vincent. I have adapted the path in the documention. On Thu, May 12, 2022 at 3:24 PM Lizzi, Vincent <Vincent.Lizzi@taylorandfrancis.com> wrote:
Hi Christian and Florian,
I recall that the DBA web interface didn’t like the YAJSW log files, although that didn’t seem surprising because the YAJSW log files are in a different format than BaseX log files. It’s been a while since I tried this so I don’t have the details now. YAJSW can be configured to save its log files in a different location.
Kind regards,
Vincent
_____________________________________________
Vincent M. Lizzi
Head of Information Standards | Taylor & Francis Group
vincent.lizzi@taylorandfrancis.com
Information Classification: General
From: Christian Grün <christian.gruen@gmail.com> Sent: Thursday, May 12, 2022 8:03 AM To: Florian Schmitt <ml-basex@florian-schmitt.net>; Lizzi, Vincent <Vincent.Lizzi@taylorandfrancis.com> Cc: BaseX <basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] First steps with client/server on Linux - bulk import; error message in dba
Thanks for the observation, Florian.
After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry:
I hope the latest snapshot resolves the Yajsw/.logs bug [1].
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at
I’ve added Vincent Lizzi to the conversation, who has gratefully added the documentation on YAJSW: Maybe it would be sensible to move the wrapper-basex.log file out of the .logs directory? I don’t expect any complications if it’s stored in data/.
I’ll additionally check if we can prevent admin:logs from crashing if the log input does not conform to the usual pattern.
Best, Christian
[1] https://github.com/BaseXdb/basex/issues/2105
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R...
That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
I agree with Florian that navigation of db contents is important and support is desirable - perhaps in analogy to file:children and file:descendants. Kind regards, Hans-Jürgen PS: The same applies to the contents of archives. Am Dienstag, 10. Mai 2022, 16:28:52 MESZ hat Florian Schmitt <ml-basex@florian-schmitt.net> Folgendes geschrieben: Hi all, i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation? First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents.... When accessing the same db using WebDAV (Oxygen), one can nicelly navigate using the collection structure. With REST, adding the collection names to the URL will restrict the output to the db content of those collections. But without such a restriction, it would be great if the REST web interface would just return the first level of collections without any XML resource entries. Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections? I've found that Oxygen "translates" subdirectories into collections when using the "import directories" feature. Is there somehing similar using the command line client? And another issue: The "Logs" page of the DBA webapp shows the following error warning: "Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..." Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?). Yours sincerely. Florian
Hi Florian,
i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation?
Thanks for the offer. You are invited to register for a docs.basex.org Wiki acount. If you prefer, you can give us some specific hints on how to revise existing texts in the documentation.
First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents....
I agree. The REST interface was included many years ago, but in the meantime there are various ways – e.g. via db:dir() – to restrict the database view to specific directories. I have added an issue for that [1]. As it will be a breaking change, we’ll probably tackle it with BaseX 10 or later.
Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections?
If you create a new database, you can use the CREATE command or db:create and specify a directory as initial input. The existing file hierarchy will be mirrored in the resulting database. Hope this helps, Christian [1] https://github.com/BaseXdb/basex/issues/2106
Hi Christian,
Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections?
If you create a new database, you can use the CREATE command or db:create and specify a directory as initial input. The existing file hierarchy will be mirrored in the resulting database.
it seems i'm not quite used with the lightweight concept of databases in BaseX :-) - since it's very easy to create and use multiple databases, filling the db initially is a good and reasonable solution. Regarding the server installation: i'm not sure what went wrong yesterday - anyway, i've ended up with another basex subdirectory in the home directory of my baseX user, containing an empty webapp directory - thus there was no DBA web interface and no logs to consult... but now (esp. after i've found the extremely valuable information about config files: https://docs.basex.org/wiki/Configuration) everything works smooth and nice. Thanks again for your help! Florian
Hi Florian,
I agree. The REST interface was included many years ago, but in the meantime there are various ways – e.g. via db:dir() – to restrict the database view to specific directories. I have added an issue for that [1]. As it will be a breaking change, we’ll probably tackle it with BaseX 10 or later. [1] https://github.com/BaseXdb/basex/issues/2106
The REST output was revised: Resources in subdirectories have been replaced with a summarizing <dir> element. A new BaseX 10 snapshot is available [1]. Best, Christian [1] https://files.basex.org/maven/org/basex/basex/10.0-SNAPSHOT/basex-10.0-20220...
participants (4)
-
Christian Grün -
Florian Schmitt -
Hans-Juergen Rennau -
Lizzi, Vincent