- BaseX-Talk - mailman.uni-konstanz.de

Ripple of effect of db corruption
by France Baril 19 Jul '14

19 Jul '14

Hi, In the end of May we had issue with a BaseX release that was corrupting our DBs. We were able to identify this pattern: 1) Update/insert @status on the root element of multiple files. Eveything seems fine here, but something occured that made the DB unstable. We can read data without problem. 2) Change the status a second time, DBs gets corrupted. We didn't track all operations that occured during that period, but now that we are running a new translation process. We are updating the various language DBs and we are ovewhelmed by corrupted DBs. We thought we had rolled back all the problematic DBs, and apparently we missed some as issues are popping here and there. I am writing with the hope that you remember an issue with the end of May release and that you can send details on tracking issues before they show up and/or know of a way to resolve the issue before the second run of updates. Rolling back means that we lose a full run of translation and we have nearly 30 languages, which means 30 DBs, so it's not a small job to roll them back and run translation again. And once a DB is corrupted, I can't even open it. Any clue would help. Thanks, France -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 1

internal database format changes between 7.X and 8.0 versions
by Fabrice Etanchaud 19 Jul '14

19 Jul '14

Dear all @BaseX, We are currently in the process of reloading a huge volume of data (about 47 Go of zipped xml), And wondering if we can safely do that in the 8.0 beta version. I heard format will change with the 8.0 version. If we stay with the 7.9 version, will OPTIMIZE ALL upgrade the format (or any other way to do so) ? Best regards, Fabrice Questel/Orbit

2 1

JSON or JSONML output formatting
by Zachary Pride 16 Jul '14

16 Jul '14

Hello, I've been playing with BaseX lately and all-in-all am very impressed. Something I'm wrestling with however is the JSON output options. I've got various web applications that would really be simplified by receiving JSON straight away for the obvious scripting benefits. The JSON method has been giving me a lot of grief, stipulating rules I'm not sure how to follow. right now I have an element that "is typed as string and cannot be nested." I've been able to get JSONML to output but the JSON is this morass of nested Arrays... Attributes seem to get transformed into Objects but elements/tags and their values are pushed into Arrays for some rationale - which I suppose is on purpose. My question is how can I get Objects for simple key:value properties from <tag>value</tag> 's. Are there options I'm not utilizing? Would it be feasible to transform the XML output to some middle form first - setting it up to become Objects, before serializing it to JSON? Thanks! Zach Pride Associate Software Engineer Ultra Electronics, ProLogic 1000 Green River Drive, Ste. 201 Fairmont, WV 26554 zachary.pride(a)ultra-prologic.com Tel: +1 304 333 2680 ext.308 www.ultra-prologic.com This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. This e-mail from Ultra Electronics, ProLogic and any attachments to it are confidential to the intended recipient and may also be privileged or controlled by U.S. export control laws and regulations. No technical data, information or other items provided by Ultra Electronics, ProLogic in connection with this email shall be shared or provided to any foreign persons, entities, or subsidiaries without the expressed written authorization of Ultra Electronic, ProLogic. If you have received it in error please notify the sender and delete it from your system. If you are not the intended recipient you must not copy it or use it for any purpose nor disclose or distribute its contents to any other person. All communications may be subject to interception or monitoring for operational and/or security purposes. Please rely on your own virus checking as the sender cannot accept any liability for any damage arising from any bug or virus infection. Please consider the environment before printing this email.

4 4

Cannot get MIXUPDATES working
by Marc van Grootel 15 Jul '14

15 Jul '14

This is with basex 7.9 on Windows I am trying to get (RESTXQ) queries to run with MIXUPDATES. I tried adding MIXUPDATES = true to each of the sections in the .basex file. For all but the last section (# Local Options) the MIXUPDATES line would be deleted upon startup of basexhttp.bat So if I would add the line to the top under # General Options and then start the http server I would get: >basexhttp -d C:/apps/basex/.basex: Unknown option 'MIXUPDATES'. C:/apps/basex/.basex: writing new configuration file. DEBUG: true PASSWORD: admin USER: admin DEBUG: true WEBPATH: F:\home\data\basex\webapp BaseX 7.9 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) When I put it under # Local Options it won't complain but it won't pick up MIXUPDATES so my queries still fail. >basexhttp -d DEBUG: true PASSWORD: admin USER: admin DEBUG: true WEBPATH: F:\home\data\basex\webapp BaseX 7.9 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) Am I doing something wrong here? --Marc

2 2

Re: [basex-talk] Understanding the DBLocking mechanism
by Maximilian Gärber 15 Jul '14

15 Jul '14

Hi, ok - guess I need to dig deeper into constraints of a "two phase commit" then... So, when using the core-API in embedded mode, the preferred way would be to create a new Thread per query - somewhat like the ClientListener does in server mode? Regards, Max 2014-07-14 16:09 GMT+02:00 Jens Erat <jens.erat(a)uni-konstanz.de>: > Hi Max, > > BaseX uses strict two phase locking to prevent deadlocks, therefor it is not allowed to fetch any locks after releasing the first one. > > Enhancing locking (and relaxing the conditions) is still considered, but there is no estimate when it will be available (and what kind of concurrency control). > > Regards, > Jens > > -- > Jens Erat > > [phone]: tel:+49-151-56961126 > [mail]: mailto:email@jenserat.de > [jabber]: xmpp:jabber@jenserat.de > [web]: http://www.jenserat.de > > PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B > > > > Am 14.07.2014 um 16:05 schrieb Maximilian Gärber <mgaerber(a)arcor.de>: > >> Hi, >> >> while trying to understand the locking internals I stumbled over >> DBLocking.java#L106 (in core) >> which throws an Exception when the same thread tries to get a lock again. >> >> The Java documentation states that ReentrantReadWriteLock should be >> able to handle this [1], are there any extra-conditions that make this >> neccessary? >> >> >> Regards, >> >> Max >> >> [1] "The method will return immediately if the current thread already >> owns the lock." >> http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/Reentra… >> http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/locks/Reentra… >

2 1

Understanding the DBLocking mechanism
by Maximilian Gärber 14 Jul '14

14 Jul '14

Hi, while trying to understand the locking internals I stumbled over DBLocking.java#L106 (in core) which throws an Exception when the same thread tries to get a lock again. The Java documentation states that ReentrantReadWriteLock should be able to handle this [1], are there any extra-conditions that make this neccessary? Regards, Max [1] "The method will return immediately if the current thread already owns the lock." http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/Reentra… http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/locks/Reentra…

1 0

JSON output in PHP client
by Paul Swennenhuis 11 Jul '14

11 Jul '14

I've been playing a bit with BaseX the last couple of days - and very excited about the product while doing so - but now I am having problems outputting JSON from a PHP script. I have this XQuery: declare option output:method "json"; declare option output:json "format=jsonml"; <json type="object"> { for $user in collection("saveresult")//user[_id="1f2cda8f-a18a-44ba-8d17-73626d472306"] return <testResults> <testId>{$user/test/_id}</testId> <grade>{$user/user_info/user_grade}</grade> </testResults> } </json> In the BaseX GUI this yields: ["json", {"type":"object"}, ["testResults", ["testId", ["_id", "Bio-1"]], ["grade", ["user_grade", "7"]]], ["testResults", ["testId", ["_id", "Bio-2"]], ["grade", ["user_grade", "1"]]]] Which is what I expected after reading about the JSONML format. (Not too happy with that format, but that's another story). Running exactly the same XQuery form within a PHP script however yields this: <json type="object"> <testResults> <testId> <_id>Bio-1</_id> </testId> <grade> <user_grade>7</user_grade> </grade> </testResults> <testResults> <testId> <_id>Bio-2</_id> </testId> <grade> <user_grade>1</user_grade> </grade> </testResults> </json> (script is attached; note that the $ signs have been escaped there so that PHP will not try to evaluate them) In other words, the result is being output as XML with an enclosing <json> root tag, not as JSON. What is happening here? And how can I have BaseX return JSON from PHP? Paul Swennenhuis

3 7

Distributed features in BaseX
by dooon＠sina.com 11 Jul '14

11 Jul '14

Hello, We are now choosing a xml database and BaseX is one of the candidates. I want to know if BaseX can support distributed storage and distributed query. If it can support these features, pls give me some advices or documents. Thanks Chen

3 2

Highlight missing when getting the results using {{@link Item#toJava()}
by Javier Couto 10 Jul '14

10 Jul '14

Hi, I have a problem with ft:mark()that seems to be a bug. Given the following query: for $sentence in //sentence where $sentence[text() contains text "biological"] return ft:mark($sentence[text() contains text "biological"], 'b') And the following document structure: <article> <articleInfo> <date>27-06-14</date> <author>Author</author> </articleInfo> <section id="1"> <title id="1.1">Title Section here</title> <paragraph id="1.1.1"> <sentence id="1.1.1.1.1">Induction of NF-KB during monocyte differentiation by HIV type 1 infection.</sentence> <sentence id="1.1.1.1.2">Electrophoretic mobility shift assays and Southwestern blotting experiments were used to detect the binding of cellular transactivation factor NF-KB to the double repeat-KB enhancer sequence located in the long terminal repeat.</sentence> </paragraph> ... In Java, if I use the methods query(String query), process(String query) or serialize(String query) from https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/o…, I get the desired results (i.e. the term “biological” is highlighted): <sentence id="445">A novel fluorescent silica tracer for <b>biological</b> silicification studies</sentence> <sentence id="319125">Cell <b>biological</b> basis of biomineralization</sentence> ... But when I try to access the Java object, using the method iterate(String query), the highlight is missing: QueryProcessor proc = new QueryProcessor(query, context); Iter iter = proc.iter(); ArrayList<BXElem> results = new ArrayList<BXElem>(); for(Item item; (item = iter.next()) != null;) results.add((BXElem) item.toJava()); proc.close(); // print results for(BXElem elem: results) System.out.println(elem.getTextContent()); Output: A novel fluorescent silica tracer for biological silicification studies Cell biological basis of biomineralization ... Is this a known bug or I am doing something wrong? Thank you in advance. Javier

2 1

generate-id() is not unique in loop
by Sergey Alekhin 10 Jul '14

10 Jul '14

Hello, this seems to be critical bug in [BaseX7.8.2]. The following code let $a := <element></element> for $i in (1 to 5) return copy $b := $a modify ( insert node attribute id { generate-id() } into $b ) return $b will return elements with the same IDs, i.e.: <element id="id31442"/> <element id="id31442"/> <element id="id31442"/> <element id="id31442"/> <element id="id31442"/>

2 1