Good morning (in Oregon), Eric,

You've brought up SQLite many times as an example of how you want BaseX to behave. The "embeddable" nature of SQLite is not a feature the developer decided to add to an existing project, but the very purpose of its existence. SQLite was designed by a US military contractor specifically to be included in applications on naval ships without requiring the installation of a MySQL or PostgreSQL server.

In contrast, BaseX uses a server/client architecture like MySQL and PostgreSQL. It needs to be installed and configured system-wide and started as a service, so there's no point in allowing users to define multiple different DBPATHs. This isn't an unusual or limiting structure, just different from what you've envisioned for your particular application.

For example, anyone who wants to put WordPress on a web server needs to first install and configure the MySQL server package on the system. WordPress doesn't ship with embedded databases, just SQL statements to build the necessary tables. Similarly, to transfer an existing WordPress website to a new server, the database needs to be dumped as SQL and reconstructed from that source, the same way I reconstruct BaseX databases from the XML documents when I move the project around. And if the server admin wants to run other applications that use MySQL databases, they will all live in the same central service, not from files read from user-specified directories. I've rarely encountered projects that use SQLite, but I often find ones that require MySQL, so this workflow is not a significant problem for users.

In this thread, we've offered suggestions of ways you could structure your application to use BaseX, and how each of your concerns could be addressed. You can either consider them and experiment, or you can find a different tool you could use to fully embed your data and access it without a system-wide service.

-Tamara

On Mon, Feb 21, 2022 at 5:01 PM Eric Levy <contact@ericlevy.name> wrote:

On Mon, 2022-02-21 at 15:50 -0800, Tamara Marnell wrote:
> Assuming every distribution of your project will contain the original
> files, or work based off of user-supplied files, there is no
> limitation to putting all databases under one directory. Your
> instructions for installation only need to include specifying that
> directory in the .basex file, or in a custom file of definitions if
> you want to set it programmatically; and running a script that
> constructs the BaseX database on that computer from the documents.
> Whenever I move our project to a new machine--which I did frequently
> during development--our scripts rebuilt our databases of ~42K
> documents, including full-text and custom indexes, in about five
> minutes.

I think you're not understanding the use, as your general framing is
quite far from what I have tried to explain. I'm not sure it would be
helpful for me to address each point you raised in detail.

I feel my explanations are generally clear, especially in light of the
context I offered, and I don't want to keep repeating myself. Perhaps
the SQLite documentation, especially the section "Appropriate Uses For
SQLite" [1], would provide a more robust or accessible representation
of embedded database applications than has emerged so far in the
discussion.

1. https://www.sqlite.org/whentouse.html

On Tue, 2022-02-22 at 01:16 +0100, Imsieke, Gerrit, le-tex wrote:
> Or to say it more bluntly: Eric, start using BaseX productively
> before
> making feature requests that seem out-of-sync with BaseX’s
> architecture
> to core developers and to long-time users alike. The fact that no
> one
> has raised such a request in the last decade – no one has come forth
> to
> say “wow, if I could specify each database’s own directory, I could
> do
> things with less friction” – speaks volumes. This is not a request
> that
> a BaseX user would make, even if they thought about ways to embed
> the
> database engine in an application.

I made a request, and explained the use case, drawing from examples of
established design patterns. The request may not be one you see as
aligned to the project objectives, which is fine, but the analysis
meets me as rather circular.