HI Christian,
Could not reply to your mail.Here's the reply please go through and suggest us the way forward.
#Sateesh: This would be only one time activity only,but one problem with this approach is lets suppose i have 1gb of xml file when i create db for this xml it would result in approximatly 2gb of database and for this i need to maintain TB's of hard disk […]
This sounds like a general architectural challenge to me, which may not necessarily linked to BaseX itself. What do you need all those user-specific databases for?
#Sateesh1: We create databases and we use them to retrieve user information,as each time creating databases and deleting them takes some time what we are doing is we run an offline tool and creates those databases so that when user chks his details online the users will get the response more faster.With this approach what is happening is as we have around 10k users and each user will have a xml getting generated every month and also basex DB's getting created for those xml's is requiring TB's of storage,for this is there any option in basex which will create DB sizes equal to xml size or less than xml size's(Currently if xml size is 10mb DB size is creating as 20mb approx).
Thanks & Regards SAteesh.A ________________________________________ From: Christian Grün christian.gruen@gmail.com Sent: Thursday, October 9, 2014 2:32 PM To: Sateesh Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Fw: Suggestion for a requirement
#Sateesh: This would be only one time activity only,but one problem with this approach is lets suppose i have 1gb of xml file when i create db for this xml it would result in approximatly 2gb of database and for this i need to maintain TB's of hard disk […]
This sounds like a general architectural challenge to me, which may not necessarily linked to BaseX itself. What do you need all those user-specific databases for?
which we are not able to defend with clients.
What do you mean by that?
#Sateesh: Per minute we can expect 100 parallel hits,in this hits some queries might be complex queries and some might be returning huge volumes of data in response.
Once again, this sounds more like a general issue. If the queries are really complex in terms of runtime (O(n²) or more?), and if the returned data is huge, you will always come across limits in terms of CPU power and bandwidth, no matter which system you are using. Maybe you could analyze if you really need to return the complete amount of data, or if the client can live with a much smaller result. Think of a world map: the contained amount of information is immense, but on a certain level, you will only see parts of the data. The same applies to the TreeMap in BaseX, which only retrieves parts of the database that are required for the actual visual representation. As you may know, XQuery is a very powerful language, which does not only allow you to do aggregation, ordering or filtering in a query. You can also use it to create completely new XML fragments that are comprised of the contents of existing databases. The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.