Hi Arjan, thanks again for your helpful report. It pointed me to a generally buggy behavior, which may occur when document updates occur in databases that are opened by multiple clients. I have opened a new issue [1]. As I'll be on the road for some (more) days, it may take some time until this is fixed, but be sure that this is something we won't sweep under the carpet. Christian [1] https://github.com/BaseXdb/basex/issues/575 ___________________________ On Sun, Sep 30, 2012 at 12:24 AM, Arjan van Bentem <arjan.vanbentem@bidnetwork.org> wrote:

Hi all,

I noticed that TCP/IP clients are not seeing new documents that are created using db:add(...) in another client:

Once a query has been executed, subsequent queries using the very same database connection do not see/return new documents that were meanwhile created using another connection. When creating some (different) document using one's own existing connection, then everything seems to be in-sync for a moment. But right after that, any new documents are again not seen.

Updates made in other clients are visible just fine. So, the documents that are returned are up-to-date. It's just that not all documents are retrieved.

I can see that in the query plans, the size in <DBNodeSeq size="..."> does not change, and is lower than the size that clients with a newer connection would show.

Should it be okay to keep a connection open? Or is the expected behaviour to need a new connection to see new documents?

Details below.

Thanks, Arjan.

1. Start the server and a client in debugging mode:

./basexserver -d -S ./basexclient -U admin -P admin -d -x

2. Create and open a database:

create database db1 open db1

3. Keep the client open, and make another client insert some documents. Like every second for about 2 minutes, each time using a new connection:

for i in {1..120}; \ do doc="<test><a>`uuidgen`</a></test>"; \ echo $i, $doc; \ ./basexclient -U admin -P admin "db:add('db1', '$doc', 'dummy')"; \ sleep 1; \ done

4. While Step 3 is running, back in the first client, execute:

xquery count(/test)

The log in data/.logs shows:

REQUEST XQUERY count(/test) OK Query executed in 25.74 ms. 223.77 ms

So far, so good.

5. Repeat Step 4 to see the result NOT change, while the loop from Step 3 is still adding new documents. Logs show a faster result, but otherwise are the same. Also the query plans are always the same. (Typically: the size in <DBNodeSeq size="..."> does not change.)

REQUEST XQUERY count(/test) OK Query executed in 5.48 ms. 7.29 ms

6. Using "close" followed by "open db1" gets one a more up-to-date value for "count". Also, using the connection to add some random other type of document, updates the value for "count(/test)" as well:

xquery db:add('db1', '<another-test><xyz>123</xyz></another-test>', 'dummy')

7. It's not just "count" that does not change its value. Also, one can only query documents up to that count as returned by Step 4. Like if Step 4 returned "15", then the following would return 5 results, not 10:

xquery (/test)[position()=10 to 20]

Also, the following would only return the count as returned earlier:

xquery /test

Like noted, UPDATES made in other clients are visible just fine:

xquery replace value of node /another-test/xyz with '456' xquery for $n in /test/a return replace value of node $n with fn:generate-id($n)

Tested in both 7.3 and 7.5-20120929.201251 from http://files.basex.org/releases/latest/, in Oracle Java 7 on Mac OS X 10.8.2. I first noticed that my running application did not see new documents created by a test script. But the command line tools show the same issue.

Any clue?

_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk