Re: [basex-talk] Fwd: Re : Basex Query Optimization Support

18 Sep 2015

      Hi Adi,     The attached code you sent is for uploading the files (by the way, I believe you don't need the tempFlag on each iteration). I'd think that would take a long time to upload, but it should only run once, right?. So you're query now should be something like this:
for $details incollection('Sample_Collection')/
                       SUBCUSTBRK[AN[@NO = (987654321)]][DOB >='2014-04-01' and DOB <= '2015-03-31']/CNOS/ROW)
Have you isolated that this query is the one that's taking a long time to execute? Have you used the GUI to find out the query plan that's being used? You could start by using just one condition, then add the others. The query plan should show you whether it's using indexes or not.     You could also double check that the data has been indexed by using the index:text and index:attributes functions, though they might return a lot of data, but they will tell you which data has been indexed.
Hope it helps,     Martín.
Date: Tue, 15 Sep 2015 16:49:22 +0530
Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
From: adibabu.b@gmail.com
To: christian.gruen@gmail.com
CC: ferrari_martin@hotmail.com; basex-talk@mailman.uni-konstanz.de
Hai Chistian,Please find the attachment sample piece of code.
Thanks & RegardsAdi
On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Adi,
How does your new code look like? How did you rewrite the collection function?
Christian
On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu adibabu.b@gmail.com wrote:
...
Hi Martin,
...
I am working with collection data base only from starting on wards. Through
...
mail it is not possible to share the large collection db(more than 7GB)
...
thats why I am sharing sample query(with xml doc but actually I am using
...
collection only).
...
I have tried all your suggestions in the trail mail ,but it takes more time
...
in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
...
...
Thanks & Regards,
...
Adi
...
...
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari
...
ferrari_martin@hotmail.com wrote:
...
...
...
...
HI Adi,
...
...
 Have you tried Max's suggestion of adding the xml data to a BaseX db

...
...
first? Based on this:
...
...
...
...
for $details in
...
...
...
...
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
...
...
...
...
...
...
it looks like you're using the file Freq_Called_Sample_XML_Data.xml
...
...
directly from the filesystem, which means it has no indexes and must be
...
...
fully scanned to get results. You should create a DB and upload that xml
...
...
file with TEXTINDEX and ATTRINDEX enabled (which is the default).  You might
...
...
need to run the OPTIMIZE ALL command after uploading the xml file, you can
...
...
run the INFO DB command and it will show if your database has up-to-date
...
...
indexes or not. Once you have a db, you can also use the GUI to check the
...
...
query plan that's being used for a particular query.
...
...
...
...
...
...
Martín.
...
...
...
...
...
...

...
...
Date: Fri, 11 Sep 2015 18:37:05 +0530
...
...
From: adibabu.b@gmail.com
...
...
To: mgaerber@arcor.de
...
...
CC: basex-talk@mailman.uni-konstanz.de
...
...
Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
...
...
...
...
Dear Max,
...
...
Currently I am using collection only(size 7.96 GB), I am sharing the query
...
...
for your reference only. I tried the latest version 8.2.3 also but it takes
...
...
more time than 7.9.
...
...
Can you please assist me to proceed further.
...
...
...
...
Thanks & Regards,
...
...
Adi
...
...
...
...
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber mgaerber@arcor.de
...
...
wrote:
...
...
...
...
Hi Adi,
...
...
...
...
some general pointers:
...
...

Server version 7.9 is missing all the latest perf improvements,

...
...
current version is 8.2.3
...
...

You are accessing the files in the file system - try adding them to

...
...
the database first, then you can make use of the text index etc. See
...
...
http://docs.basex.org/wiki/Indexes#Text_Index
...
...
...
...
Regards,
...
...
...
...
Max
...
...
...
...
...
...
2015-09-04 8:31 GMT+02:00 Adi Babu adibabu.b@gmail.com:
...
...
...
Dear Team,
...
...
...
I am facing performance issue taking long time to execute the
...
...
...
Xquery in Basex(Client Server architecture), can you please guide me to
...
...
...
optimize the query performance.
...
...
...
...
...
...
Below are the statistics & Basex server hardware specifications.
...
...
...
...
...
...
Statistics :
...
...
...

...
...
...
Total Records count : 6000000
...
...
...
Fetch count         : 10000
...
...
...
Hits Parallel hits  : 100
...
...
...
Time Taking         : 750 seconds
...
...
...
Architecture        : Client Server Architecture
...
...
...
...
...
...
Hardware Specifications of Basex Sever :
...
...
...

...
...
...
Operating System         : Red Hat Enterprise Linux 64bit
...
...
...
OS Version : Linux 2.6.32
...
...
...
Hard Disk : 500 GB
...
...
...
Memory Capacity     : 16 GB Ram
...
...
...
Processor Family         : Intel (R) Xeon(R) CPU E5 -2680 V2
...
...
...
Processor Speed         : 2.80 GHz
...
...
...
Physical Processors     : 8
...
...
...
Virtual Processors/cores : 4
...
...
...
Server Version : basex 7.9
...
...
...
...
...
...
Note : I have attached the sample xml data and Xquery .
...
...
...
...
...
...
...
...
...
Thanks & Regards
...
...
...
Adi
...
...
...
...
...
...
...
...

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Re: [basex-talk] Fwd: Re : Basex Query Optimization Support