Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance. Below are the statistics & Basex server hardware specifications. Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9 Note : I have attached the sample xml data and Xquery . Thanks & Regards Adi
Hi Adi, some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index Regards, Max 2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further. Thanks & Regards, Adi On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
HI Adi, Have you tried Max's suggestion of adding the xml data to a BaseX db first? Based on this: for $details in (collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml") it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must be fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query. Martín. Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support Dear Max,Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9.Can you please assist me to proceed further. Thanks & Regards,Adi On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote: Hi Adi, some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index Regards, Max 2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team,
I am facing performance issue taking long time to execute the
Xquery in Basex(Client Server architecture), can you please guide me to
optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics :
-------------
Total Records count : 6000000
Fetch count : 10000
Hits Parallel hits : 100
Time Taking : 750 seconds
Architecture : Client Server Architecture
Hardware Specifications of Basex Sever :
----------------------------------------------------------
Operating System : Red Hat Enterprise Linux 64bit
OS Version : Linux 2.6.32
Hard Disk : 500 GB
Memory Capacity : 16 GB Ram
Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2
Processor Speed : 2.80 GHz
Physical Processors : 8
Virtual Processors/cores : 4
Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards
Adi
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than 7GB) thats why I am sharing sample query(with xml doc but actually I am using collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further. Thanks & Regards, Adi On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari <ferrari_martin@hotmail.com
wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a BaseX db first? Based on this:
for $details in (collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must be fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
------------------------------ Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Hi Adi, How does your new code look like? How did you rewrite the collection function? Christian On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than 7GB) thats why I am sharing sample query(with xml doc but actually I am using collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards, Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari <ferrari_martin@hotmail.com> wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a BaseX db first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must be fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
________________________________ Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Hai Chistian, Please find the attachment sample piece of code. Thanks & Regards Adi On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Adi,
How does your new code look like? How did you rewrite the collection function?
Christian
On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than 7GB) thats why I am sharing sample query(with xml doc but actually I am using collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards, Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari <ferrari_martin@hotmail.com> wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a BaseX
db
first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must be fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You
might
need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
________________________________ Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Hi Adi, I may not be up-to-date, as I expected you to send me an updated version of your XQuery code. Did you try what Martín suggested? Christian On Tue, Sep 15, 2015 at 1:19 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hai Chistian, Please find the attachment sample piece of code.
Thanks & Regards Adi
On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Adi,
How does your new code look like? How did you rewrite the collection function?
Christian
On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than 7GB) thats why I am sharing sample query(with xml doc but actually I am using collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards, Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari <ferrari_martin@hotmail.com> wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a BaseX db first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must be fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
________________________________ Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de> wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Hi Chirstian, I tried what Martin suggested. Please find the below is the sample code what I am using for creating collection. /* This method takes the xml files list as input and process the list Create the xml collection DB, adds xml to the collection if already exists replace it */ public static void createCollection(ArrayList<String> listFiles){ ClientSession clientSession = null; String strIndexedFile=""; try { clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, "<<USER>>", "<<PWD>>"); String strDbName = "Sample_Collection"; String strKeysFileLocation = "D:\\TEMP\\7GB_Collection_TestingTeam\\Files"; String res = clientSession.execute("XQUERY db:exists('" + strDbName + "')"); //DB existence checking if (!res.equalsIgnoreCase("true")) { clientSession.execute("CREATE DB " + strDbName); } else { clientSession.execute("OPEN " + strDbName); } for (int i = 0; i < listFiles.size(); i++) { strIndexedFile = listFiles.get(i); String tempFlag=clientSession.execute("XQUERY db:exists('" + strDbName + "','" + strIndexedFile + "')"); clientSession.execute("ADD " + strKeysFileLocation + File.separator + strIndexedFile); } System.out.println("InFormation:: "+clientSession.execute("INFO")); //optimizing the current opend DB using OPTIMIZE ALL command clientSession.execute("OPTIMIZE ALL"); System.out.println("InFormation:: "+clientSession.execute("INFO")); } catch (Exception e) { System.out.println("Exception files DB Updating: " + e.getMessage()); e.printStackTrace(); } finally { if (clientSession != null) { try { clientSession.close(); } catch (IOException ex) { ex.printStackTrace(); } } } } Thanks & Regards, Adi On Tue, Sep 15, 2015 at 4:52 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Hi Adi,
I may not be up-to-date, as I expected you to send me an updated version of your XQuery code. Did you try what Martín suggested?
Christian
On Tue, Sep 15, 2015 at 1:19 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hai Chistian, Please find the attachment sample piece of code.
Thanks & Regards Adi
On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi Adi,
How does your new code look like? How did you rewrite the collection function?
Christian
On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hi Martin, I am working with collection data base only from starting on wards. Through mail it is not possible to share the large collection db(more than
7GB)
thats why I am sharing sample query(with xml doc but actually I am using collection only). I have tried all your suggestions in the trail mail ,but it takes more time in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards, Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari <ferrari_martin@hotmail.com> wrote:
HI Adi, Have you tried Max's suggestion of adding the xml data to a
BaseX
db first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml directly from the filesystem, which means it has no indexes and must
be
fully scanned to get results. You should create a DB and upload that xml file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might need to run the OPTIMIZE ALL command after uploading the xml file, you can run the INFO DB command and it will show if your database has up-to-date indexes or not. Once you have a db, you can also use the GUI to check the query plan that's being used for a particular query.
Martín.
________________________________ Date: Fri, 11 Sep 2015 18:37:05 +0530 From: adibabu.b@gmail.com To: mgaerber@arcor.de CC: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max, Currently I am using collection only(size 7.96 GB), I am sharing the query for your reference only. I tried the latest version 8.2.3 also but it takes more time than 7.9. Can you please assist me to proceed further.
Thanks & Regards, Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de
wrote:
Hi Adi,
some general pointers: * Server version 7.9 is missing all the latest perf improvements, current version is 8.2.3 * You are accessing the files in the file system - try adding them to the database first, then you can make use of the text index etc. See http://docs.basex.org/wiki/Indexes#Text_Index
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team, I am facing performance issue taking long time to execute the Xquery in Basex(Client Server architecture), can you please guide me to optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics : ------------- Total Records count : 6000000 Fetch count : 10000 Hits Parallel hits : 100 Time Taking : 750 seconds Architecture : Client Server Architecture
Hardware Specifications of Basex Sever : ---------------------------------------------------------- Operating System : Red Hat Enterprise Linux 64bit OS Version : Linux 2.6.32 Hard Disk : 500 GB Memory Capacity : 16 GB Ram Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 Processor Speed : 2.80 GHz Physical Processors : 8 Virtual Processors/cores : 4 Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards Adi
Hi Adi, The attached code you sent is for uploading the files (by the way, I believe you don't need the tempFlag on each iteration). I'd think that would take a long time to upload, but it should only run once, right?. So you're query now should be something like this: for $details incollection('Sample_Collection')/ SUBCUSTBRK[AN[@NO = (987654321)]][DOB >='2014-04-01' and DOB <= '2015-03-31']/CNOS/ROW) Have you isolated that this query is the one that's taking a long time to execute? Have you used the GUI to find out the query plan that's being used? You could start by using just one condition, then add the others. The query plan should show you whether it's using indexes or not. You could also double check that the data has been indexed by using the index:text and index:attributes functions, though they might return a lot of data, but they will tell you which data has been indexed. Hope it helps, Martín. Date: Tue, 15 Sep 2015 16:49:22 +0530 Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support From: adibabu.b@gmail.com To: christian.gruen@gmail.com CC: ferrari_martin@hotmail.com; basex-talk@mailman.uni-konstanz.de Hai Chistian,Please find the attachment sample piece of code. Thanks & RegardsAdi On Tue, Sep 15, 2015 at 4:34 PM, Christian Grün <christian.gruen@gmail.com> wrote: Hi Adi, How does your new code look like? How did you rewrite the collection function? Christian On Tue, Sep 15, 2015 at 12:48 PM, Adi Babu <adibabu.b@gmail.com> wrote:
Hi Martin,
I am working with collection data base only from starting on wards. Through
mail it is not possible to share the large collection db(more than 7GB)
thats why I am sharing sample query(with xml doc but actually I am using
collection only).
I have tried all your suggestions in the trail mail ,but it takes more time
in 8.2.3 than 7.9 version. Can you please guide me to proceed further.
Thanks & Regards,
Adi
On Fri, Sep 11, 2015 at 10:04 PM, Martín Ferrari
<ferrari_martin@hotmail.com> wrote:
HI Adi,
Have you tried Max's suggestion of adding the xml data to a BaseX db
first? Based on this:
for $details in
(collection("E:\Web-Projects\VodafoneUK\Docs\Cloud_Test_Stats\Performance_Analysis\Freq_Called_Sample_XML_Data.xml")
it looks like you're using the file Freq_Called_Sample_XML_Data.xml
directly from the filesystem, which means it has no indexes and must be
fully scanned to get results. You should create a DB and upload that xml
file with TEXTINDEX and ATTRINDEX enabled (which is the default). You might
need to run the OPTIMIZE ALL command after uploading the xml file, you can
run the INFO DB command and it will show if your database has up-to-date
indexes or not. Once you have a db, you can also use the GUI to check the
query plan that's being used for a particular query.
Martín.
________________________________
Date: Fri, 11 Sep 2015 18:37:05 +0530
From: adibabu.b@gmail.com
To: mgaerber@arcor.de
CC: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Fwd: Re : Basex Query Optimization Support
Dear Max,
Currently I am using collection only(size 7.96 GB), I am sharing the query
for your reference only. I tried the latest version 8.2.3 also but it takes
more time than 7.9.
Can you please assist me to proceed further.
Thanks & Regards,
Adi
On Tue, Sep 8, 2015 at 9:15 PM, Maximilian Gärber <mgaerber@arcor.de>
wrote:
Hi Adi,
some general pointers:
* Server version 7.9 is missing all the latest perf improvements,
current version is 8.2.3
* You are accessing the files in the file system - try adding them to
the database first, then you can make use of the text index etc. See
Regards,
Max
2015-09-04 8:31 GMT+02:00 Adi Babu <adibabu.b@gmail.com>:
Dear Team,
I am facing performance issue taking long time to execute the
Xquery in Basex(Client Server architecture), can you please guide me to
optimize the query performance.
Below are the statistics & Basex server hardware specifications.
Statistics :
-------------
Total Records count : 6000000
Fetch count : 10000
Hits Parallel hits : 100
Time Taking : 750 seconds
Architecture : Client Server Architecture
Hardware Specifications of Basex Sever :
----------------------------------------------------------
Operating System : Red Hat Enterprise Linux 64bit
OS Version : Linux 2.6.32
Hard Disk : 500 GB
Memory Capacity : 16 GB Ram
Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2
Processor Speed : 2.80 GHz
Physical Processors : 8
Virtual Processors/cores : 4
Server Version : basex 7.9
Note : I have attached the sample xml data and Xquery .
Thanks & Regards
Adi
participants (4)
-
Adi Babu -
Christian Grün -
Martín Ferrari -
Maximilian Gärber