Hi all, ## Notes on Implementing Non-Locking If implementing such non-locking I’d propose following: - If documented at all, provide a large red warning message in the docs. Something about really do not use this feature if you don’t know the internals very well and voiding all support or something like that. - We already have some locking options. I’d recommend to use them, but disable the recognition of which databases to lock (thus do not call the visitor, I think it should be in `AQuery#databases`). Do not break locking for commands, they’re able to recognize what they need to lock very well. - Add some function like `boolean lockTest(String db)`. See if `db` is actually locked before opening it. There is one function always called in the `Data` class, this is a great point for such a hook, I used it when writing a test implementation of optimistic concurrency control for my [thesis]. If the lock is not set, fail early (throw some QueryException). Your XQuery developers (and database consistency) will be very thankful. The current `DBLocking` implementation expects conservative locking, thus fetching all locks in the beginning of the transaction and will fail if you try to fetch more locks afterwards. If you fetch locks using the existing XQuery options you will not have any problems with that. We used that to prevent deadlocks to prevent failure in case of external side effects which occur rather often, but this resulted in rather bad parallelism for complex queries (which means —- sadly —- most relevant ones). ## Efforts on Further Improved Concurrency Control We have some discussion on extending our locking for use cases which can deal with external side effects (thus, restarting the transaction in case of deadlocks), but this is a rather large project as it delves deep in the core _and_ large parts of the query processor, but this will definitely take some time and effort. The concurrency control part is not too complicated itself (and there is at least a somewhat working, yet rather hacky, concept implementation), but making BaseX capable of gracefully terminating and restarting transactions will take some effort. Kind regards from Lake Constance, Jens [thesis]: http://kops.ub.uni-konstanz.de/handle/urn:nbn:de:bsz:352-235049 -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B Am 19.11.2013 um 20:30 schrieb Christian Grün <christian.gruen@gmail.com>:
What I actually did was add a DISABLELOCKING option which when true will stop all locking in the DBLocking class, then I use a map of locks in my java library to grab a write lock for a given database (using the db name as the key) when doing any update. However there is one problem with this, when we release the lock at the end of the query, as I understand it the pending update list may not have been flushed. To fix this I'm going to have to release the lock after a server side forward, but is there any better way of doing this?
…to say more, we’ll probably have to look into your implementation first.
Thanks for your feedback, Christian
I'm planning on submitting a pull request for the DISABLELOCKING feature soon.
Thanks, Joe
On Tue, Nov 19, 2013 at 5:26 AM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Joe,
I'm going to try and build my own version of basex with the option to turn off all locking. If I get something acceptable I will send a pull request for it to be merged in.
did you already have to look at the code?
My impression is that we could quite easily add a QUERYLOCKING that, when disabled, only considers manual locking flag, which can e.g. be seat via the query:read-lock and query:write-lock options. But I dont’ know if that would help, as you mentioned you’d have to choose the database dynamically.
You mentioned that you are expecting the database to be locked for 1-2 seconds. Maybe there is some chance to speed up your write operations?
Best, Christian