Just one more hint, if we parallelly to the ongoing insertions post one directly with basexclient, we don't experience this lag ... M.
On 20/02/2015 15:53, Marco Lettere wrote:
It's me again .... sorry. ;-)
One question that puzzles me in these last hours .. This is the scenario on 2 DBs (same server) with the following sizes:
136746 docs ~1 Gb 194608 docs ~0.5 Gb
declare %rest:path('{$project}/some/url/') %rest:POST('{$body}') %rest:consumes("application/xml") function _:insert($project, $body) { prof:time(
__:insert_impl($project, $body),false(), "POST ACCOUNT") };
The function insert() is an RestXQ entry point for document insertion. It delegates to the actual implementing function. Insertion occurs continuosly. The implementing function performs queries for checking uniqueness of what is inserted and then posts the actual update which is a db:add or db:replace (BTW does this make a difference from a performance point of view?????). Insertion might occur either on one or on both the DBs. The time T output by the profile:time is a steady amount of time. This means that queries and update schedule is working in constant time. The time reported by Jetty (and perceived at the client) is a T + d. The issue is that this d may vary from something around few hundreds of milliseconds to something which is 4 or 5 times T itself. And this happens in waves. When it's slow, it's slow for few minutes and then it gets better for a few minutes and then back again ... When staring at the tcpdump it's evident that all the time is spent between the ack of the request and the sending of the response. Thus network overhead is negligible. This makes me think that the overhead is in the commit of the PUL. Is this possible? I'm using basex 8.0 official. Openjdk 7 and monitoring through JMX doesn't show anything particularly critical. Any clues on what I have to better look into? Thanks and sorry for bothering ... M.