BaseX-Talk

basex-talk@mailman.uni-konstanz.de

12 participants
5243 discussions

Fwd: Securing All Endpoints
by Adam Law 27 Jan '21

27 Jan '21

Hi Christian, Thanks for the permissions pointer- it works well. As an aside I had to use let $vbFailure1 := ($perm?authorization != 'example' or exists( $perm?authorization )=false()) Rather than let $vbFailure1 := ($perm?authorization != 'example' ) To handle the case of no Authorisation. I am not sure why. I should be able to use request:cookie for ws (when I need to) which is great. I wasn't able to work out a catchall rest:path solution but it is not really important for me. Thanks again for the pointer. Adam On Mon, Jan 18, 2021 at 8:33 PM Christian Grün <christian.gruen(a)gmail.com> wrote: > You could have a look at the "Catch HTTP Errors" section in our > documentation. > > [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling > > On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw(a)gmail.com> wrote: > > > > Another thought - a 'catch all' path for fail2ban to catch attempted > accessess to undefined paths. > > I haven't tested this to see if there is such a thing in BaseX > > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün < > christian.gruen(a)gmail.com> wrote: > >> > >> Hi Adam, > >> > >> You could have a loot at our Permissions layer [1]. It can be used for > >> all kinds of checks, not only those specific to authentication. I > >> think we should generalize the concept in a future version of BaseX > >> and introduce additional post-hooks. > >> > >> Hope this helps? > >> Christian > >> > >> [1] https://docs.basex.org/wiki/Permissions > >> > >> > >> > >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw(a)gmail.com> > wrote: > >> > > >> > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > >> > > >> > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > >> > > >> > It seems that I have to secure each endpoint individually. > >> > > >> > Adam >

2 4

Shrinking view in Manage Database Dialog
by Markus Schmid 27 Jan '21

27 Jan '21

Hallo in Basex v9.4.5 im Dialog Manage Database wird die linke View zusammengeschrumpft wenn die rechte Detail View wegen langer Pfad- oder Datenbanknamen viel Platz braucht. Die Views ändern die Breite während man die Einträge in der Liste der Datenbanken durchläuft und auswählt. Beides nervt leicht. Freundliche Grüße Markus S.

2 2

cross-platform file name parameters
by Graydon Saunders 26 Jan '21

26 Jan '21

Hello -- So I'm on a Linux box and my colleague is on a Windows system. We're both using a query that takes before and after versions of a file, converts each to a single string, and writes that back out as two additional files. (This exists to check if we still have all the words in the same order between before and after.) There must be a sensible way to parameterize the file names so they can be relative to the query on both platforms/easily loaded from a file/something that isn't hand-editing a variable (external or not), but I can't think of it. What's the sensible way to do this? Thanks! Graydon

3 2

[Configuration improvements] Separate client and server config, stop overwriting config files
by Volodymyr Prokopyuk 26 Jan '21

26 Jan '21

Hello the BaseX team, First of all I'm really impressed with the quality of the BaseX XML database. I'd particularly highlight the high compliance with the standards XQuery, XPath and the outstanding performance! Thank you very much for the great product! I'm new to the system and I've faced several configuration deficiencies while trying to configure the BaseX database to fit my needs. What follows is my use case (I hope it is a basic and classical one required by much wider audience), the configuration problems I've faced (I hope these can be easily solved by extending your current implementation) and my proposal to solve the above mentioned configuration problems (I hope this will further improve the quality of your product). *My use case (access the basexserver run as a systemd service via the basexclient)* - I've installed the basex package on Manjaro (Arch linux) into /usr/share/java/basex/ - I want to run the basexserver as a systemd service under the already created basex system user (see the attached basex.server unit file) - I want to store the XML databases and the BaseX logs under /var/lib/basex/data /var/lib/basex/log respectively - I want to manage the basexserver configuration (namely DBPATH, LOGPATH, SERVERPORT) separately from the basexclient configuration (namely HOST, PORT, USER, PASSWORD) in a separate files under /etc/basex and ~/ respectively. I do not want to get my configuration overwritten with unnecessary default options that are already in the system and so clutter my clean just-what-you-need configuration files - I want to manage the XML databases and execute queries against the server using the basexclient from my console by providing XQuery command and queries in a file *Configuration problems (mixed config for client and server in a single .basex file under hardly linked home directory)* - My .basex file mixes server and client configuration in one single file so I cannot separate the configuration into server-related at /etc/basex and client-retated at ~/ - My .basex file gets overwritten with unnecessary default options that clutter my minimal and clean configuration - When I start the basexclient a new .basex configuration file is created in the current working directory. I start the basexclient from many different direcotries, but I want to have the basexclient configuration only in my home ~/ directory - It seems that the location of the .basex file is hardly linked to the home directory forcing me to place data, log and config directories under a single home directory. I want to manage my data in /var/lib/basex/data, my logs in /var/lib/basex/log and my server and client configuration in /etc/basex and ~/ respectively - I cannot find any way (via environment variables or command line options) to instruct both the basexserver and the basexclient to look for the configuration file .basex *Proposal to improve configuration* - Do not overwrite the user-provided configuration files with unnecessary default options that are already in the system or at least use a different file (e. g. .basex-default) - Separate the server-related configuration from the client-related configuration into different files (e. g. .basexserver, .basexclient) - Do not mandate the location of configuration files to be under the home directory because there are different aspects of database management that are usually managed in different directories with different policies and quotas (e. g. data, log, client and server config) - Provide separate options for basexserver and basexclient to locate the corresponding configuration file (e. g. via environment variables BASEX_SERVER_CONFIG, BASEX_CLIENT_CONFIG or command line options -server-config, -client-config) - In order to introduce the proposed improvement you can keep your current implementation unchanged (following the open-closed principle) and just extend the implementation with the alternative locations for the separate configuration files via environment variables or command line options. The only change that is required is to stop overwriting the user-provided configuration files with the default options Can the above described use case be configured with the current implementation of the configuration management? Thank you in advance! I'm open to further discussion and collaboration mostly on the design side to get these improvements to work for all other use cases not covered here. Thank you, Vlad

2 1

BaseX python replace bug
by Daniel Kaplan 22 Jan '21

22 Jan '21

Hello, I'm experiencing some issue(s) when using BaseX... wanted to report it here, but not sure what information you need. I'm accessing baseX via python. Attached is some code that reproduces it for me. It doesn't seem to occur when I am not manually flushing, and only seems to occur with replace, but not add. How should I proceed? Cheers, Daniel PS: I am using a folder containing all the items in question.

3 5

Securing All Endpoints
by Adam Law 18 Jan '21

18 Jan '21

Golang has great middleware functionality for endpoints that allows a person to readily secure all of them. I was wondering there would be anyway doing similar in BaseX for restxq & websockets. For example by requiring all requests to have a particular query parameter (very basic example). ... to protect say against browser / javascript port scanning. Eg https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware and check that request contains a particular parameter (very basic security) It seems that I have to secure each endpoint individually. Adam

2 6

error "lock file does not exist" after "add" command
by Thomas Spitaler 13 Jan '21

13 Jan '21

Hi, in one of our projects we are using BaseX. So far we are very happy with our choice, but recently we got an error that we could not fix. The only solution that worked was to create a new, empty database. When trying to add a document to the database, we get an exception and a message „lock file does not exist“: $ open cap10_cap_store_prod Database 'cap10_cap_store_prod' was opened in 219.99 ms. $ add /home/thomas/tmp/test.xml Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.0.1 Java: Debian, 11.0.9 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: cap10_cap_store_prod: lock file does not exist. at org.basex.util.Util.notExpected(Util.java:61) at org.basex.data.DiskData.finishUpdate(DiskData.java:246) at org.basex.core.cmd.ACreate.update(ACreate.java:97) at org.basex.core.cmd.Add.run(Add.java:56) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:92) at org.basex.core.CLI.execute(CLI.java:76) at org.basex.BaseX.console(BaseX.java:196) at org.basex.BaseX.<init>(BaseX.java:171) at org.basex.BaseX.main(BaseX.java:42) Which lock file is the error referring to? Can we create it manually to make it work again? The error only happens in the database that we have been using so far (cap10_cap_store_prod). After creating and using a new one (cap10_cap_store_prod2) the problem does not occur. Here is the output of running list on the database: $ list Name Resources Size Input Path cap10_cap_store_prod 233387 1583797639 cap10_cap_store_prod2 241 1912414 testdb 1 4611 3 database(s). Did we hit a size limit of BaseX? I checked the server and it has enough free disk space and free inodes available. Some more background info: The system runs on a cluster of servers. BaseX runs on one of them, bound to port 1984. The other nodes in the cluster connect to the BaseX-Server via the network. In my understanding, this type of concurrency is supported by BaseX. The software on the nodes is implemented in Elixir. We wrote our own Elixir client library that is a thin wrapper around https://github.com/zadean/basexerl, an existing open source Erlang client library for BaseX. Our Elixir client library keeps open a pool of connections to the BaseX server. So far and until we encountered the above error everything worked perfectly. Also, after creating a new database everything works fine again. This error also occurs after shutting down all the nodes that connect to our BaseX instance. We checked using the lsof command and there are no connections open to the database other then the client we use to reproduce the error. Any help would be greatly appreciated! Kind regards, Thomas Kind regards, Thomas

2 4

file:read-text
by Giuseppe G. A. Celano 12 Jan '21

12 Jan '21

Hi, I am trying to open a bunch of files with file:read-text, but I get the error [file:io-error] invalid XML character: #0, even I am not dealing with XML. Any idea why this happens? Thanks! Best, Giuseppe

2 3

Brew upgrade for 9.4.6 fails with SHA256 mismatch
by Johan Mörén 08 Jan '21

08 Jan '21

Just wanted to let you know that the latest BaseX version can't be installed or upgraded via homebrew. I get the following error message: brew upgrade basex ==> Upgrading 1 outdated package: basex 9.4.5 -> 9.4.6 ==> Upgrading basex 9.4.5 -> 9.4.6 ==> Downloading https://files.basex.org/releases/9.4.6/BaseX946.zip ######################################################################## 100.0% Error: SHA256 mismatch Expected: c6dd9ac56e72de19c88152c76a6e4052deccd4f0db7a4b072aaaa560b92318a5 Actual: 96012025d749062540f9b16f830d7b99876696b66534da2fe27c833d1b1fc43b File: /Users/johmor/Library/Caches/Homebrew/downloads/befef7ec63e74dbf695bd24e06e16773055a9fb0c7f69d9d7e9fd1787766a377--BaseX946.zip To retry an incomplete download, remove the file above. Regards, Johan

2 4

Improper use/potential bug error
by Eliúd Santiago Meza y Rivera 08 Jan '21

08 Jan '21

Hi, I just update to the latest version of BaseX 9.4.6, and the web server now do not refer correctly the link, checking every individually broken link, the server response this message: I attach the code. In advance I appreciate your help. HTTP ERROR 500 java.lang.NoSuchMethodError: org.basex.query.value.type.SeqType.cast(Lorg/basex/query/value/Value;Lorg/basex/query/QueryContext;Lorg/basex/query/StaticContext;Lorg/basex/util/InputInfo;)Lorg/basex/query/value/Value; URI: /app/static/pngkit_realistic-spider-web-png_1049388.png STATUS: 500 MESSAGE: java.lang.NoSuchMethodError: org.basex.query.value.type.SeqType.cast(Lorg/basex/query/value/Value;Lorg/basex/query/QueryContext;Lorg/basex/query/StaticContext;Lorg/basex/util/InputInfo;)Lorg/basex/query/value/Value; SERVLET: RESTXQ CAUSED BY: java.lang.NoSuchMethodError: org.basex.query.value.type.SeqType.cast(Lorg/basex/query/value/Value;Lorg/basex/query/QueryContext;Lorg/basex/query/StaticContext;Lorg/basex/util/InputInfo;)Lorg/basex/query/value/Value;Caused by: java.lang.NoSuchMethodError: org.basex.query.value.type.SeqType.cast(Lorg/basex/query/value/Value;Lorg/basex/query/QueryContext;Lorg/basex/query/StaticContext;Lorg/basex/util/InputInfo;)Lorg/basex/query/value/Value; at org.basex.http.web.WebFunction.bind(WebFunction.java:137) at org.basex.http.restxq.RestXqFunction.bind(RestXqFunction.java:210) at org.basex.http.restxq.RestXqResponse.bind(RestXqResponse.java:60) at org.basex.http.web.WebResponse.create(WebResponse.java:61) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:73) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:738) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905) at java.lang.Thread.run(Unknown Source) ------------------------------ Powered by Jetty:// 9.4.34.v20201102 <https://eclipse.org/jetty> -- LSC Eliud Santiago Meza y Rivera

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk