Hi, Emailed to announce by mistake.
The stacktrace as requested
xquery for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at org.basex.util.hash.IntSet.rehash(IntSet.java:125) at org.basex.util.hash.IntObjMap.rehash(IntObjMap.java:55) at org.basex.util.hash.ASet.checkSize(ASet.java:82) at org.basex.util.hash.IntSet.index(IntSet.java:109) at org.basex.util.hash.IntSet.put(IntSet.java:49) at org.basex.util.hash.IntObjMap.put(IntObjMap.java:29) at org.basex.query.up.DataUpdates.add(DataUpdates.java:77) at org.basex.query.up.ContextModifier.add(ContextModifier.java:56) at org.basex.query.up.DatabaseModifier.add(DatabaseModifier.java:40) at org.basex.query.up.Updates.add(Updates.java:96) at org.basex.query.up.expr.Delete.item(Delete.java:44) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:44) at org.basex.query.QueryContext.iter(QueryContext.java:409) at org.basex.query.expr.If.iter(If.java:126) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:100) at org.basex.query.MainModule.cache(MainModule.java:100) at org.basex.query.QueryContext.iter(QueryContext.java:345) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81) at org.basex.core.cmd.AQuery.query(AQuery.java:90) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) org.basex.core.BaseXException: Out of Main Memory. at org.basex.core.Command.execute(Command.java:101) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) Out of Main Memory.
________________________________ Från: Christian Grün christian.gruen@gmail.com Skickat: den 9 juli 2016 22:24 Till: Henning Phan Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
Runs out of memory.
Thanks for trying. Could you please pass me on the stack trace? If it doesn't show up immediately, you may need to switch on debugging (via SET DEBUG true, or -d on command line).
It seems to be the large size of independent delete operations that causes the OOM error. Maybe you should first delete all nodes in a single run:
1. delete //*[empty(@mark)] 2. db:optimize($db, true())
What will be the resulting database size after these two steps?
In general, you can always increase the amount of memory assigned to Java via -Xmx [1].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Start_Scripts
The stacktrace as requested
xquery for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at org.basex.util.hash.IntSet.rehash(IntSet.java:125) at org.basex.util.hash.IntObjMap.rehash(IntObjMap.java:55) at org.basex.util.hash.ASet.checkSize(ASet.java:82) at org.basex.util.hash.IntSet.index(IntSet.java:109) at org.basex.util.hash.IntSet.put(IntSet.java:49) at org.basex.util.hash.IntObjMap.put(IntObjMap.java:29) at org.basex.query.up.DataUpdates.add(DataUpdates.java:77) at org.basex.query.up.ContextModifier.add(ContextModifier.java:56) at org.basex.query.up.DatabaseModifier.add(DatabaseModifier.java:40) at org.basex.query.up.Updates.add(Updates.java:96) at org.basex.query.up.expr.Delete.item(Delete.java:44) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:44) at org.basex.query.QueryContext.iter(QueryContext.java:409) at org.basex.query.expr.If.iter(If.java:126) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:100) at org.basex.query.MainModule.cache(MainModule.java:100) at org.basex.query.QueryContext.iter(QueryContext.java:345) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81) at org.basex.core.cmd.AQuery.query(AQuery.java:90) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) org.basex.core.BaseXException: Out of Main Memory. at org.basex.core.Command.execute(Command.java:101) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) Out of Main Memory.
Från: Christian Grün christian.gruen@gmail.com Skickat: den 9 juli 2016 22:24 Till: Henning Phan Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
Runs out of memory.
Thanks for trying. Could you please pass me on the stack trace? If it doesn’t show up immediately, you may need to switch on debugging (via SET DEBUG true, or -d on command line).
Hi,
Im not interested at all in this approach, deleting the nodes. Though if you are interested I feel obligated to help you out as you have spared your time for me.
For some reason
1. delete //*[empty(@mark)] returns 0 items or hits
2 //*[empty(@mark)] returns 100 000
Therefore after updating the db stays the same size 2176 MB
________________________________ Från: Christian Grün christian.gruen@gmail.com Skickat: den 9 juli 2016 23:31 Till: Henning Phan Kopia: basex-talk@mailman.uni-konstanz.de Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
It seems to be the large size of independent delete operations that causes the OOM error. Maybe you should first delete all nodes in a single run:
1. delete //*[empty(@mark)] 2. db:optimize($db, true())
What will be the resulting database size after these two steps?
In general, you can always increase the amount of memory assigned to Java via -Xmx [1].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Start_Scripts Start Scripts - BaseX Documentationhttp://docs.basex.org/wiki/Start_Scripts docs.basex.org Main Package. The following scripts can be used to launch the standalone version of BaseX. Please replace the class name in org.basex.BaseX with either BaseXClient, ...
The stacktrace as requested
xquery for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at org.basex.util.hash.IntSet.rehash(IntSet.java:125) at org.basex.util.hash.IntObjMap.rehash(IntObjMap.java:55) at org.basex.util.hash.ASet.checkSize(ASet.java:82) at org.basex.util.hash.IntSet.index(IntSet.java:109) at org.basex.util.hash.IntSet.put(IntSet.java:49) at org.basex.util.hash.IntObjMap.put(IntObjMap.java:29) at org.basex.query.up.DataUpdates.add(DataUpdates.java:77) at org.basex.query.up.ContextModifier.add(ContextModifier.java:56) at org.basex.query.up.DatabaseModifier.add(DatabaseModifier.java:40) at org.basex.query.up.Updates.add(Updates.java:96) at org.basex.query.up.expr.Delete.item(Delete.java:44) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:44) at org.basex.query.QueryContext.iter(QueryContext.java:409) at org.basex.query.expr.If.iter(If.java:126) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:100) at org.basex.query.MainModule.cache(MainModule.java:100) at org.basex.query.QueryContext.iter(QueryContext.java:345) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81) at org.basex.core.cmd.AQuery.query(AQuery.java:90) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) org.basex.core.BaseXException: Out of Main Memory. at org.basex.core.Command.execute(Command.java:101) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42) Out of Main Memory.
Från: Christian Grün christian.gruen@gmail.com Skickat: den 9 juli 2016 22:24 Till: Henning Phan Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
Runs out of memory.
Thanks for trying. Could you please pass me on the stack trace? If it doesn't show up immediately, you may need to switch on debugging (via SET DEBUG true, or -d on command line).
Thanks for you suggestions,
I thought I would provide an update as we now have a working solution.
The final solution was to create a package-info.java, because the xml contained xmlns:= and not xmlns:xyz= we defined namespace prefix as "" in the package-info file.
@XmlSchema(namespace = "http://server.my.org/xyz", xmlns = { @XmlNs(namespaceURI = "http://server.my.org/xyz", prefix = "") }, elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED)
This enabled JAXB to successfully un-marshall the XML from BaseX without modifying the XML returned by the simple query
Hope this helps someone,
P.S. Now all we need is for someone to add BaseX support to JPA :-)
MikeE
On Sat, Jul 9, 2016 at 3:07 PM, Henning Phan henningphan@hotmail.com wrote:
Hi,
Im not interested at all in this approach, deleting the nodes. Though if you are interested I feel obligated to help you out as you have spared your time for me.
For some reason
- delete //*[empty(@mark)] returns 0 items or hits
2 //*[empty(@mark)] returns 100 000
Therefore after updating the db stays the same size 2176 MB
*Från:* Christian Grün christian.gruen@gmail.com *Skickat:* den 9 juli 2016 23:31 *Till:* Henning Phan *Kopia:* basex-talk@mailman.uni-konstanz.de *Ämne:* Re: [basex-talk] Removing the xmlns attribute and/or adding the prefix
It seems to be the large size of independent delete operations that causes the OOM error. Maybe you should first delete all nodes in a single run:
- delete //*[empty(@mark)]
- db:optimize($db, true())
What will be the resulting database size after these two steps?
In general, you can always increase the amount of memory assigned to Java via -Xmx [1].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Start_Scripts Start Scripts - BaseX Documentation http://docs.basex.org/wiki/Start_Scripts docs.basex.org Main Package. The following scripts can be used to launch the standalone version of BaseX. Please replace the class name in org.basex.BaseX with either BaseXClient, ...
The stacktrace as requested
xquery for $e in //* let $m := $e/@mark return if($m) then ( delete node $m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at org.basex.util.hash.IntSet.rehash(IntSet.java:125) at org.basex.util.hash.IntObjMap.rehash(IntObjMap.java:55) at org.basex.util.hash.ASet.checkSize(ASet.java:82) at org.basex.util.hash.IntSet.index(IntSet.java:109) at org.basex.util.hash.IntSet.put(IntSet.java:49) at org.basex.util.hash.IntObjMap.put(IntObjMap.java:29) at org.basex.query.up.DataUpdates.add(DataUpdates.java:77) at
org.basex.query.up.ContextModifier.add(ContextModifier.java:56)
atorg.basex.query.up.DatabaseModifier.add(DatabaseModifier.java:40)
at org.basex.query.up.Updates.add(Updates.java:96) at org.basex.query.up.expr.Delete.item(Delete.java:44) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:44) at org.basex.query.QueryContext.iter(QueryContext.java:409) at org.basex.query.expr.If.iter(If.java:126) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:100) at org.basex.query.MainModule.cache(MainModule.java:100) at org.basex.query.QueryContext.iter(QueryContext.java:345) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:81) at org.basex.core.cmd.AQuery.query(AQuery.java:90) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) atorg.basex.api.client.LocalSession.execute(LocalSession.java:132)
at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42)org.basex.core.BaseXException: Out of Main Memory. at org.basex.core.Command.execute(Command.java:101) at
org.basex.api.client.LocalSession.execute(LocalSession.java:132)
at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:103) at org.basex.core.CLI.execute(CLI.java:87) at org.basex.BaseX.console(BaseX.java:191) at org.basex.BaseX.<init>(BaseX.java:166) at org.basex.BaseX.main(BaseX.java:42)Out of Main Memory.
Från: Christian Grün christian.gruen@gmail.com Skickat: den 9 juli 2016 22:24 Till: Henning Phan Ämne: Re: [basex-talk] Removing the xmlns attribute and/or adding the
prefix
for $e in //* let $m := $e/@mark return if($m) then ( delete
node
$m, (: [.] is same as: where $d ne "" :) for $d in data($m)[.] return ( delete node $e/text() ) ) else ( delete node $e )
Runs out of memory.
Thanks for trying. Could you please pass me on the stack trace? If it doesn’t show up immediately, you may need to switch on debugging (via SET DEBUG true, or -d on command line).
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Henning Phan -
Mike Engledew