I’ve created the GitHub project ekimbernow/basex-orchestration: https://github.com/ekimbernow/basex-orchestration
This contains the orchestration module and a “database administration” module that serves as a test bed for orchestration and as a set of utilities for manipulating databases with error checking and logging.
Cheers,
E.
_____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow
From: Eliot Kimber eliot.kimber@servicenow.com Date: Sunday, February 20, 2022 at 7:55 AM To: Jonathan Robie jonathan.robie@gmail.com, BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Database 'name' is being updated, or update was not completed. The approach I’ve been taking is to have two databases: a “production” database that is used for reads by the consuming code (i.e, RESTXQ-served web pages that provide reports or views of the content) and a “temp” database where updates happen.
I then swap the temp for the production when the data processing is done.
If I understand your requirement it sounds like you’d need a query that does the work to make a “snapshot”, commit that state, copy it to the production database, and then run the query to make the next snapshot, etc.
I use two or more BaseX instances to do the database processing, all sharing the same data directory (and thus the same set of databases). The main instance, running on the default ports, serves the web application and only reads from the databases. The “worker” instances, started on other ports of course, do the writing to the temp databases.
The main logistical challenge I’ve faced is how to orchestrate the database swapping and data processing.
If your set of database names is static, then it’s easy to do with simple basex scripts that run individual XQueries, i.e.:
DROP DB temp COPY prod temp XQUERY (: Call to code that does some work to put data in temp :) ALTER prod prod_backup ALTER temp prod
My challenge has been that I have a number of databases created dynamically, so I need a bit more flexibility, thus my work to do the orchestration with BaseX jobs directly within XQuery.
Cheers,
E.
_____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow
From: BaseX-Talk basex-talk-bounces@mailman.uni-konstanz.de on behalf of Jonathan Robie jonathan.robie@gmail.com Date: Saturday, February 19, 2022 at 3:03 PM To: BaseX basex-talk@mailman.uni-konstanz.de Subject: [basex-talk] Database 'name' is being updated, or update was not completed. [External Email]
I have some processes that update databases, taking a long time to do so.
I would like other processes to be able to read the last complete snapshot while this happens. Is that possible?
Jonathan