Xquery returns 'Out of Main Memory'
Hello, The query below returns 'Out of Main Memory': XQUERY collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() contains text "Nobel" ftand "prize"] I am using basexclient. I have a collection of xml files that have 14GB size. --------------------------------------------------------------------
LIST name Resources Size Input Path
bd_inex2009 730283 16263937902 --------------------------------------------------------------------- I dumped those files by the commands below: ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/; ---------------------------------------------------------------------------------------- The command INFO returns: INFO General Information Version: 7.7.2 Used Memory: 79 MB Main Options DBPATH: /home/arwen/BaseXData DEBUG: false EVENTPORT: 1985 GLOBALLOCK: false HOST: localhost HTTPLOCAL: false KEEPALIVE: 0 LANG: English LANGKEYS: false LOG: true LOGMSGMAXLEN: 1000 NONPROXYHOSTS: PARALLEL: 8 PASSWORD: PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: /home/arwen/BaseXRepo RESTXQPATH: SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 0 USER: WEBPATH: /home/arwen/BaseXWeb Options ADDARCHIVES: true ADDCACHE: false ADDRAW: false ATTRINDEX: true AUTOFLUSH: true BINDINGS: CACHEQUERY: false CASESENS: false CATFILE: CHECKSTRINGS: true CHOP: true COMPPLAN: true CREATEFILTER: *.xml CREATEONLY: false DIACRITICS: false DOTCOMPACT: false DOTDISPLAY: true DOTPLAN: false DOTTY: dotty DTD: false EXPORTER: FORCECREATE: false FTINDEX: false FTINDEXSPLITSIZE: 0 HTMLOPT: html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 INDEXSPLITSIZE: 0 INLINELIMIT: 0 INTPARSE: true LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXHITS: -1 MAXLEN: 96 MAXSTAT: 30 PARSER: xml PARSEROPT: encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma QUERYINFO: false QUERYPATH: RUNS: 1 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false STEMMING: false STOPWORDS: STRIPNS: false TAILCALLS: 256 TEXTINDEX: true UPDINDEX: false WRITEBACK: false XMLPLAN: false XQUERY3: true ----------------------------------------------------------- How can I run XQuery in that collection? Regards, Daiane
Hi Daiane, due to some nifty details in the XQFT spec., it is hardly possible to make "ftand" really efficent. But there is one way out: you can use the "all" keyword. Please find more details here: http://docs.basex.org/wiki/Full-Text#Combining_Results Hope this helps, Christian On Mon, Feb 3, 2014 at 1:59 PM, DAIANE ROBERTA CANDIDA <dairc_8@hotmail.com> wrote:
Hello,
The query below returns 'Out of Main Memory':
XQUERY collection("/home/arwen/daiane_oracle/loading_inex2009/xml_pages/")//article[.//text() contains text "Nobel" ftand "prize"]
I am using basexclient.
I have a collection of xml files that have 14GB size. --------------------------------------------------------------------
LIST name Resources Size Input Path
bd_inex2009 730283 16263937902 ---------------------------------------------------------------------
I dumped those files by the commands below:
ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles1/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles2/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles3/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles4/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles5/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles6/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles7/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles8/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles9/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles10/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles11/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles12/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles13/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles14/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles15/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles16/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles17/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles18/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles19/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles20/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles21/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles22/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles23/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles24/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles25/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles26/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles27/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles28/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles29/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles30/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles31/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles32/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles33/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles34/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles35/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles36/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles37/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles38/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles39/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles40/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles41/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles42/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles43/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles44/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles45/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles46/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles47/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles48/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles49/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles50/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles51/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles52/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles53/; ADD /home/arwen/daiane_oracle/loading_inex2009/xml_pages/xmlfiles54/;
----------------------------------------------------------------------------------------
The command INFO returns: INFO General Information Version: 7.7.2 Used Memory: 79 MB
Main Options DBPATH: /home/arwen/BaseXData DEBUG: false EVENTPORT: 1985 GLOBALLOCK: false HOST: localhost HTTPLOCAL: false KEEPALIVE: 0 LANG: English LANGKEYS: false LOG: true LOGMSGMAXLEN: 1000 NONPROXYHOSTS: PARALLEL: 8 PASSWORD: PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: /home/arwen/BaseXRepo RESTXQPATH: SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 0 USER: WEBPATH: /home/arwen/BaseXWeb
Options ADDARCHIVES: true ADDCACHE: false ADDRAW: false ATTRINDEX: true AUTOFLUSH: true BINDINGS: CACHEQUERY: false CASESENS: false CATFILE: CHECKSTRINGS: true CHOP: true COMPPLAN: true CREATEFILTER: *.xml CREATEONLY: false DIACRITICS: false DOTCOMPACT: false DOTDISPLAY: true DOTPLAN: false DOTTY: dotty DTD: false EXPORTER: FORCECREATE: false FTINDEX: false FTINDEXSPLITSIZE: 0 HTMLOPT: html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 INDEXSPLITSIZE: 0 INLINELIMIT: 0 INTPARSE: true LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXHITS: -1 MAXLEN: 96 MAXSTAT: 30 PARSER: xml PARSEROPT: encoding=UTF-8,flat=false,header=false,jsonml=false,lines=true,separator=comma QUERYINFO: false QUERYPATH: RUNS: 1 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false STEMMING: false STOPWORDS: STRIPNS: false TAILCALLS: 256 TEXTINDEX: true UPDINDEX: false WRITEBACK: false XMLPLAN: false XQUERY3: true
-----------------------------------------------------------
How can I run XQuery in that collection?
Regards,
Daiane
_______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
participants (2)
-
Christian Grün -
DAIANE ROBERTA CANDIDA