Currently all document we installed is in binary format. So we use bin:decode-String(db:retrieve(‘schema’, path)

I see. In that case, the degradation is due to your filesystem (find some details in [1]). You could either try to run BaseX on a partition with a file system that is capable of handling a larger amount of files, or you can distribute your files in different sub-paths.

0/1/file01.txt 1/2/file12.txt

Out of interest: BaseX is particularly good at handling XML structures. Why do you store everything are binary data?

[1] http://docs.basex.org/wiki/Binary_Data

Thanks

Yvonne

On 6/21/16, 1:44 PM, "Christian Grün" christian.gruen@gmail.com wrote:

...

Hi Yvonne,

could you give us some hints how you retrieve documents? Do you use one of the existing APIs, db:open, or any other functions?

TIA Christian

...

We noticed that when # of documents increased in BaseX database, it will double time to retrieve documents in database. Our test cases execution time double when # of documents increased from 200 to 400. And If number of document reach to 5000, The test case execution time increase much more.

Is that any performance testing BaseX team can share with us to compare, or there is any documentation or suggestion for BaseX database turning that we should follow?

Thanks

Yvonne

Actually, they are json format files, during testing, we only use very small size of .json format files, But our test cases including insert, update, replace, get and delete using bases db functions.

Did you consider using the JSON Module [1]? It gives you great flexibility when it comes to navigating and updating JSON in the XML format, and it can easily be output as JSON again.

[1] http://docs.basex.org/wiki/Json_Module

On Wed, Jun 22, 2016 at 7:17 PM, Wang, Yvonne yvonne.wang@sap.com wrote:

Actually, they are json format files, during testing, we only use very small size of .json format files, But our test cases including insert, update, replace, get and delete using bases db functions.

Thanks

Yvonne

On 6/21/16, 11:41 PM, "Christian Grün" christian.gruen@gmail.com wrote:

...

...

Currently all document we installed is in binary format. So we use bin:decode-String(db:retrieve(‘schema’, path)

I see. In that case, the degradation is due to your filesystem (find some details in [1]). You could either try to run BaseX on a partition with a file system that is capable of handling a larger amount of files, or you can distribute your files in different sub-paths.

0/1/file01.txt 1/2/file12.txt

Out of interest: BaseX is particularly good at handling XML structures. Why do you store everything are binary data?

[1] http://docs.basex.org/wiki/Binary_Data

...

Thanks

Yvonne

On 6/21/16, 1:44 PM, "Christian Grün" christian.gruen@gmail.com wrote:

...

Hi Yvonne,

could you give us some hints how you retrieve documents? Do you use one of the existing APIs, db:open, or any other functions?

TIA Christian

...

We noticed that when # of documents increased in BaseX database, it will double time to retrieve documents in database. Our test cases execution time double when # of documents increased from 200 to 400. And If number of document reach to 5000, The test case execution time increase much more.

Is that any performance testing BaseX team can share with us to compare, or there is any documentation or suggestion for BaseX database turning that we should follow?

Thanks

Yvonne