Re: [basex-talk] Need help on Xquery create Collection using java
Hi Riyaz, that's a good starting point - a huge load of things changed with BaseX since 671, so just have a look. Are you using the full text index? Regards, Lukas On Thu, Oct 18, 2012 at 11:20 AM, riyaz <riyaz.basha@intense.in> wrote:
Dear Lukas,
Thanks for the quick response.
1) I am using Basex671.jar and basex-api.jar. Seems it is not latest one. I will check with latest jar and update you.
and also
2) while iterating in a for loop we are adding one by one xml and where each xml is of 100kb on an average to collection.
199MB of multiple files( around 2000+ xml documents) to a single collection is taking 440 mb on harddisk.
//strAtr is a string array and contains file paths for (int j = 0; j < strAtr.length; j++) { new Add(strAtr[j]).execute(context); }
Please let me know if anything can be done, or anything needed for more clarification.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
On 10/18/2012 2:18 PM, Lukas Kircher wrote:
just a little addition:
Depending on the nature of the document you add and the kind of active indexes there will be some overhead in size and space.
400MB/3min for a 199MB document sounds a little special though ...
On Thu, Oct 18, 2012 at 10:33 AM, Lukas Kircher <lukaskircher1@gmail.com> <lukaskircher1@gmail.com>wrote:
Hi Riyaz,
1) do you use the [latest stable snapshot] or [latest release] of BaseX?
2) Could you equip us with some more facts about your case, like the exact query, the document (structure) or maybe even an SSCCE? You could also send it directly to one of us if you don't want to post it to the list.
Regards, Lukas
[latest stable snapshot] http://files.basex.org/releases/latest/ [latest release] http://files.basex.org/releases/7.3/
On Thu, Oct 18, 2012 at 6:54 AM, riyaz <riyaz.basha@intense.in> <riyaz.basha@intense.in> wrote:
Dear Team,
This is Riyaz from Hyderabad , India. I have been using X query since one and half year. I need help on the following query.
When I add xmls of 199 MB to create collection using " new Add(xmlfilepath).execute( context)" (JAVA), it is taking 440 MB on hard disk and 3 mins to complete the process. Is there any other way to reduce the size and time?
Thanks a ton in advance and would appreciate the earliest response. Please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
_______________________________________________ BaseX-Talk mailing listBaseX-Talk@mailman.uni-konstanz.dehttps://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Dear lukas, Would there be major code changes required to create collection if I migrate from basex671 to basex-7.3 jar. Because in the first step I am getting new BXCollection(collection, true, objContext) - Can't find symbol constructor. It is taking BxDatabase as third argument. I need to see any example codes on creating collection using basex7.3 , so that I can do it on my own. Would you please help me on this. -- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd. /"What you risk reveals what you value."/ On 10/18/2012 3:02 PM, Lukas Kircher wrote:
Hi Riyaz,
that's a good starting point - a huge load of things changed with BaseX since 671, so just have a look.
Are you using the full text index?
Regards, Lukas
On Thu, Oct 18, 2012 at 11:20 AM, riyaz <riyaz.basha@intense.in> wrote:
Dear Lukas,
Thanks for the quick response.
1) I am using Basex671.jar and basex-api.jar. Seems it is not latest one. I will check with latest jar and update you.
and also
2) while iterating in a for loop we are adding one by one xml and where each xml is of 100kb on an average to collection.
199MB of multiple files( around 2000+ xml documents) to a single collection is taking 440 mb on harddisk.
//strAtr is a string array and contains file paths for (int j = 0; j < strAtr.length; j++) { new Add(strAtr[j]).execute(context); }
Please let me know if anything can be done, or anything needed for more clarification.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
On 10/18/2012 2:18 PM, Lukas Kircher wrote:
just a little addition:
Depending on the nature of the document you add and the kind of active indexes there will be some overhead in size and space.
400MB/3min for a 199MB document sounds a little special though ...
On Thu, Oct 18, 2012 at 10:33 AM, Lukas Kircher <lukaskircher1@gmail.com> <lukaskircher1@gmail.com>wrote:
Hi Riyaz,
1) do you use the [latest stable snapshot] or [latest release] of BaseX?
2) Could you equip us with some more facts about your case, like the exact query, the document (structure) or maybe even an SSCCE? You could also send it directly to one of us if you don't want to post it to the list.
Regards, Lukas
[latest stable snapshot] http://files.basex.org/releases/latest/ [latest release] http://files.basex.org/releases/7.3/
On Thu, Oct 18, 2012 at 6:54 AM, riyaz <riyaz.basha@intense.in> <riyaz.basha@intense.in> wrote:
Dear Team,
This is Riyaz from Hyderabad , India. I have been using X query since one and half year. I need help on the following query.
When I add xmls of 199 MB to create collection using " new Add(xmlfilepath).execute( context)" (JAVA), it is taking 440 MB on hard disk and 3 mins to complete the process. Is there any other way to reduce the size and time?
Thanks a ton in advance and would appreciate the earliest response. Please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
_______________________________________________ BaseX-Talk mailing listBaseX-Talk@mailman.uni-konstanz.dehttps://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Riyaz, I'm really busy today, but I'll try my best to help you out. 1) You can find a number of [Java examples] in our documentation. 2) There's also a specific code example on creating a [BXCollection] via XMLDB. Please note that the XMLDB section in the [Java examples] is flagged deprecated - but I'm not exactly sure what the actual state of the implementation is. Maybe someone else can fill you in on that. So it might be worth it considering a switch to our internal API ... Regards, don't hesitate to ask if you need help. Lukas [Java examples] http://docs.basex.org/wiki/Java_Examples [BXCollection] https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/base... On Fri, Oct 19, 2012 at 8:15 AM, riyaz <riyaz.basha@intense.in> wrote:
Dear lukas,
Would there be major code changes required to create collection if I migrate from basex671 to basex-7.3 jar. Because in the first step I am getting
new BXCollection(collection, true, objContext) - Can't find symbol constructor.
It is taking BxDatabase as third argument. I need to see any example codes on creating collection using basex7.3 , so that I can do it on my own.
Would you please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
On 10/18/2012 3:02 PM, Lukas Kircher wrote:
Hi Riyaz,
that's a good starting point - a huge load of things changed with BaseX since 671, so just have a look.
Are you using the full text index?
Regards, Lukas
On Thu, Oct 18, 2012 at 11:20 AM, riyaz <riyaz.basha@intense.in> <riyaz.basha@intense.in> wrote:
Dear Lukas,
Thanks for the quick response.
1) I am using Basex671.jar and basex-api.jar. Seems it is not latest one. I will check with latest jar and update you.
and also
2) while iterating in a for loop we are adding one by one xml and where each xml is of 100kb on an average to collection.
199MB of multiple files( around 2000+ xml documents) to a single collection is taking 440 mb on harddisk.
//strAtr is a string array and contains file paths for (int j = 0; j < strAtr.length; j++) { new Add(strAtr[j]).execute( context); }
Please let me know if anything can be done, or anything needed for more clarification.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
On 10/18/2012 2:18 PM, Lukas Kircher wrote:
just a little addition:
Depending on the nature of the document you add and the kind of active indexes there will be some overhead in size and space.
400MB/3min for a 199MB document sounds a little special though ...
On Thu, Oct 18, 2012 at 10:33 AM, Lukas Kircher <lukaskircher1@gmail.com> <lukaskircher1@gmail.com> <lukaskircher1@gmail.com> <lukaskircher1@gmail.com>wrote:
Hi Riyaz,
1) do you use the [latest stable snapshot] or [latest release] of BaseX?
2) Could you equip us with some more facts about your case, like the exact query, the document (structure) or maybe even an SSCCE? You could also send it directly to one of us if you don't want to post it to the list.
Regards, Lukas
[latest stable snapshot] http://files.basex.org/releases/latest/ [latest release] http://files.basex.org/releases/7.3/
On Thu, Oct 18, 2012 at 6:54 AM, riyaz <riyaz.basha@intense.in> <riyaz.basha@intense.in> <riyaz.basha@intense.in> <riyaz.basha@intense.in> wrote:
Dear Team,
This is Riyaz from Hyderabad , India. I have been using X query since one and half year. I need help on the following query.
When I add xmls of 199 MB to create collection using " new Add(xmlfilepath).execute( context)" (JAVA), it is taking 440 MB on hard disk and 3 mins to complete the process. Is there any other way to reduce the size and time?
Thanks a ton in advance and would appreciate the earliest response. Please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
_______________________________________________ BaseX-Talk mailing listBaseX-Talk@mailman.uni-konstanz.dehttps://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Dear Riyaz, as Lucas indicated, we have marked the XML:DB API as deprecated, as it's already many years old and not developed any further. Still, you are invited to have a look at our examples, which should be up-to-date, and see if they can be aligned with your scenario. Hope this helps, Christian ___________________________ On Fri, Oct 19, 2012 at 9:28 AM, Lukas Kircher <lukaskircher1@gmail.com> wrote:
Hi Riyaz,
I'm really busy today, but I'll try my best to help you out.
1) You can find a number of [Java examples] in our documentation. 2) There's also a specific code example on creating a [BXCollection] via XMLDB.
Please note that the XMLDB section in the [Java examples] is flagged deprecated - but I'm not exactly sure what the actual state of the implementation is. Maybe someone else can fill you in on that. So it might be worth it considering a switch to our internal API ...
Regards, don't hesitate to ask if you need help. Lukas
[Java examples] http://docs.basex.org/wiki/Java_Examples [BXCollection] https://github.com/BaseXdb/basex-examples/blob/master/src/main/java/org/base...
On Fri, Oct 19, 2012 at 8:15 AM, riyaz <riyaz.basha@intense.in> wrote:
Dear lukas,
Would there be major code changes required to create collection if I migrate from basex671 to basex-7.3 jar. Because in the first step I am getting
new BXCollection(collection, true, objContext) - Can't find symbol constructor.
It is taking BxDatabase as third argument. I need to see any example codes on creating collection using basex7.3 , so that I can do it on my own.
Would you please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
"What you risk reveals what you value."
On 10/18/2012 3:02 PM, Lukas Kircher wrote:
Hi Riyaz,
that's a good starting point - a huge load of things changed with BaseX since 671, so just have a look.
Are you using the full text index?
Regards, Lukas
On Thu, Oct 18, 2012 at 11:20 AM, riyaz <riyaz.basha@intense.in> wrote:
Dear Lukas,
Thanks for the quick response.
1) I am using Basex671.jar and basex-api.jar. Seems it is not latest one. I will check with latest jar and update you.
and also
2) while iterating in a for loop we are adding one by one xml and where each xml is of 100kb on an average to collection.
199MB of multiple files( around 2000+ xml documents) to a single collection is taking 440 mb on harddisk.
//strAtr is a string array and contains file paths for (int j = 0; j < strAtr.length; j++) { new Add(strAtr[j]).execute( context); }
Please let me know if anything can be done, or anything needed for more clarification.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
On 10/18/2012 2:18 PM, Lukas Kircher wrote:
just a little addition:
Depending on the nature of the document you add and the kind of active indexes there will be some overhead in size and space.
400MB/3min for a 199MB document sounds a little special though ...
On Thu, Oct 18, 2012 at 10:33 AM, Lukas Kircher <lukaskircher1@gmail.com> <lukaskircher1@gmail.com>wrote:
Hi Riyaz,
1) do you use the [latest stable snapshot] or [latest release] of BaseX?
2) Could you equip us with some more facts about your case, like the exact query, the document (structure) or maybe even an SSCCE? You could also send it directly to one of us if you don't want to post it to the list.
Regards, Lukas
[latest stable snapshot] http://files.basex.org/releases/latest/ [latest release] http://files.basex.org/releases/7.3/
On Thu, Oct 18, 2012 at 6:54 AM, riyaz <riyaz.basha@intense.in> <riyaz.basha@intense.in> wrote:
Dear Team,
This is Riyaz from Hyderabad , India. I have been using X query since one and half year. I need help on the following query.
When I add xmls of 199 MB to create collection using " new Add(xmlfilepath).execute( context)" (JAVA), it is taking 440 MB on hard disk and 3 mins to complete the process. Is there any other way to reduce the size and time?
Thanks a ton in advance and would appreciate the earliest response. Please help me on this.
-- Thanks & Regards, Riyaz Shaik, In10^s Technologies Ltd.
* "What you risk reveals what you value."*
_______________________________________________ BaseX-Talk mailing listBaseX-Talk@mailman.uni-konstanz.dehttps://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
participants (3)
-
Christian Grün -
Lukas Kircher -
riyaz