Re: [basex-talk] Simple xQuery functions do not work as expected
Hi Christian, the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings? Kind regards, Goetz -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 11:27 An: Goetz Heller Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
P.S.: Please make sure that our conversation appears at BaseX-Talk since I'm receiving mails from other members of the community who do appear to not be informed about the current status.
I continued to write directly to you, because your second e-mail was not directed to the list anymore. It makes perfectly sense to include the list as well.
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 09:38 An: Goetz Heller Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
However, I would not consider this issue as resolved.
Obviously yes.
For me it is a hint that under heavy load (memory), such misbehavior might occur, and therefore some investigations make sense in my opinion.
I was surprised to hear that a restart of your machine solved the problem, because the GUI is nothing else than a plain JVM instance with a low memory footprint (at least for queries like the one you sent to me).
As I cannot remember to have encountered a similar problem, we rely on your feedback. Feel free to send us some more examples if the problem should decide to haunt you.
Best, Christian
Kind regards,
Goetz
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Dienstag, 24. März 2015 17:12 An: Goetz Heller Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
I am using Windows as well.
Does the problem also happen with the latest snapshot? Could you try the zip distribution?
http://files.basex.org/releases/latest/
On Tue, Mar 24, 2015 at 5:07 PM, Goetz Heller <heller@hellerim.de> wrote:
Did you try it on the Windows platform already? Maybe there was a mistake when packaging.
Kind regards,
Goetz
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Dienstag, 24. März 2015 16:50 An: Goetz Heller Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
Strange. Do you possibly have an empty or out-dated database named "labels"? If yes, could you try to back up and delete it, and run the query again?
On Tue, Mar 24, 2015 at 4:45 PM, Goetz Heller <heller@hellerim.de> wrote:
Hi Christian,
here ist he complete output. Apparently, it's over-optimized:
Result:
<r>et | </r>
Compiling: - pre-evaluating doc("C:\test\labels.xml") - applying attribute index for $key_0 - applying attribute index for "_and" - inlining local:getLabel#2 - removing redundant $key_4 as xs:string* cast. - removing redundant $lang_5 as xs:string* cast. - inlining $key_4 - inlining $lang_5 - removing unknown element/attribute TERM - pre-evaluating db:attribute("Labels", "_and")/self::*:label/parent::*:LABEL/()/text() - pre-evaluating ((: xs:string*, true :) ()) - simplifying flwor expression - inlining $retVal1_2 - inlining $retVal2_3 - simplifying flwor expression
Query: declare variable $labels := doc('C:\test\labels.xml'); declare function local:getLabel( $key as xs:string*, $lang as xs:string* ) as xs:string* { $labels/LABELS/LABEL[@label = $key]/TERM[@LG = $lang]/text() }; let $retVal1 := $labels/LABELS/LABEL[@label = '_and']/TERM[@LG = 'FR']/text() let $retVal2 := local:getLabel('_and', 'FR') return <r>{$retVal1} | {$retVal2}</r>
Optimized Query: element r { (db:attribute("Labels", "_and")/self::*:label/parent::*:LABEL/*:TERM[(@*:LG = "FR")]/text(), " | ", ()) }
Result: - Hit(s): 1 Item - Updated: 0 Items - Printed: 12 Bytes - Read Locking: global - Write Locking: none
Timing: - Parsing: 1.32 ms - Compiling: 154.5 ms - Evaluating: 0.43 ms - Printing: 0.01 ms - Total Time: 156.26 ms
Query plan: <QueryPlan compiled="true"> <CElem> <QNm value="r" type="xs:QName"/> <IterPath> <ValueAccess data="Labels" type="ATTRIBUTE"> <Str value="_and" type="xs:string"/> </ValueAccess> <IterStep axis="self" test="*:label"/> <IterStep axis="parent" test="*:LABEL"/> <IterStep axis="child" test="*:TERM"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="FR" type="xs:string"/> </CmpG> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> <Str value=" | " type="xs:string"/> <Empty size="0"/> </CElem> </QueryPlan>
Kind regards,
Goetz
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on. You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
Hi Christian, this time, both the function you've seen already and a different query which tries to extract data from a larger Xml file fail with the behavior described. In both query execution plans you can see that there is some "optimization" which places an empty pair of parentheses into the optimized query where something different should be. It looks as if an shortage of some - memory? - resources throws an exception which is silently caught somewhere. I know from my own programming experience that it happens like that: a boring nuisance is caught in the middle of something nice being developed so you neutralize this effect temporarily - and then simply forget it. Is it possible that something similar happened in this case? The second query is too complicated to simply reduce its complexity. But I suppose you would be able to get the same behavior if you took a machine like mine - a laptop with 8 GB RAM and an Intel dual core processor running at 1,8 GHz and Win 8.1. Possibly reduce the RAM by 4 GB, or run a VMWare virtual machine on it (what I cannot do here because Microsoft's Hypervisor is installed), and then experiment using your tools at hand. I had a Microsoft VS 2013 Community Edition installed, several Eclipse versions, and Node.js (none of these running when the problem showed up). No Microsoft Office. Unfortunately, I cannot invest to much time investigating the issue since I have a lot of other things to do, but let me know anyway if I can give you further information. At first sight, I liked BaseX very much for its simple installation and ease of use, its small footprint and its well-thought GUI, but I will not be able to use it in a project if I cannot rely on the results it delivers. Kind regards, Goetz P.S. Here the query plan for the second query; you will see that the $ti variable is not resolved appropriately (line 37 of output). BEGIN_OUTPUT>>> Compiling: - inlining $retVal_5 - simplifying flwor expression - pre-evaluating doc("C:\test\Labels.xml") - rewriting (compare(@*:label, $key_0) = 0) - rewriting (compare(@*:LG, "DE") = 0) - inlining $retVal_1 - simplifying flwor expression - rewriting (compare(@*:label, $key_2) = 0) - rewriting (compare(@*:LG, $lang_3) = 0) - rewriting (tokenize($nodocojs_9, "-"))[position() = 2] - inlining local:slz#1 - inlining $arg_15 - simplifying flwor expression - rewriting (tokenize($nodocojs_9, "/"))[position() = 1] - removing unknown element/attribute text - pre-evaluating $ti_7/*:TI_TEXT/() - inlining local:getLabel#1 - inlining $key_16 - simplifying flwor expression - inlining local:getLabel2#2 - removing redundant $lang_18 as xs:string cast. - inlining $key_17 - inlining $lang_18 - removing unknown element/attribute TERM - pre-evaluating document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/()/text() - simplifying flwor expression - inlining $pd_8 - inlining $nd_10 - inlining $oj_11 - inlining $ds_12 - inlining $dt_13 - inlining $hd_14 Query: declare variable $lang as xs:string := 'DE'; declare variable $labels := doc('C:\test\Labels.xml'); declare function local:getLabel( $key ) { let $retVal := $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() return $retVal }; declare function local:getLabel2( $key, $lang as xs:string ) { $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() }; declare function local:slz( $arg ) { let $retVal := xs:string(xs:double($arg)) return $retVal }; for $n in(/TED_EXPORT) let $ti := $n/TRANSLATION_SECTION/ML_TITLES/ML_TI_DOC[@LG=$lang] let $pd := $n/CODED_DATA_SECTION/REF_OJS/DATE_PUB/text() let $nodocojs := $n/CODED_DATA_SECTION/NOTICE_DATA/NO_DOC_OJS let $nd := concat(local:slz(tokenize($nodocojs, '-')[2]), '-', tokenize($nodocojs, '/')[1]) let $oj := concat($n/CODED_DATA_SECTION/REF_OJS/NO_OJ, '/', substring($n/CODED_DATA_SECTION/REF_OJS/DATE_PUB, 1, 4)) let $ds := $n/CODED_DATA_SECTION/CODIF_DATA/DS_DATE_DISPATCH/text() let $dt := replace(replace($n/CODED_DATA_SECTION/CODIF_DATA/DT_DATE_FOR_SUBMISSION, ' ', ' '), ':', ' ') let $hd := $n/CODED_DATA_SECTION/CODIF_DATA/HEADING/text() return ( <line>1.0 </line>, <line>TI: {$ti/TI_CY/replace(replace(text(), 'die ', ''), 'das ', '')}-{$ti/TI_TOWN/text()}: {$ti/TI_TEXT/text}</line>, <line>PD: {$pd}</line>, <line>ND: {$nd}</line>, <line>OJ: {$oj}</line>, <line>DS: {$ds}</line>, <line>DT: {$dt}</line>, <line>HD: {$hd}</line>, <line>LB: {local:getLabel('_and')}</line>, <line>LB_FR: {local:getLabel2('_and', 'FR')}</line> ) Optimized Query: for $n_6 in (db:open-pre("20150324_058",0), ...)/*:TED_EXPORT let $ti_7 := $n_6/*:TRANSLATION_SECTION/*:ML_TITLES/*:ML_TI_DOC[(@*:LG = "DE")] let $nodocojs_9 := $n_6/*:CODED_DATA_SECTION/*:NOTICE_DATA/*:NO_DOC_OJS return (element line { ("1.0 ") }, element line { ("TI: ", $ti_7/*:TI_CY/replace(replace(text(), "die ", ""), "das ", ""), "-", $ti_7/*:TI_TOWN/text(), ": ", ()) }, element line { ("PD: ", $n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB/text()) }, element line { ("ND: ", concat(basex:item-at(tokenize($nodocojs_9, "-"), 2) cast as xs:double? cast as xs:string?, "-", basex:item-at(tokenize($nodocojs_9, "/"), 1))) }, element line { ("OJ: ", concat($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:NO_OJ, "/", substring($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB, 1, 4))) }, element line { ("DS: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DS_DATE_DISPATCH/text()) }, element line { ("DT: ", replace(replace($n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DT_DATE_FOR_SUBMISSION, " ", " "), ":", " ")) }, element line { ("HD: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:HEADING/text()) }, element line { ("LB: ", document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/*:TERM[compare(@*:LG, "DE") = 0.0]/text()) }, element line { ("LB_FR: ", ()) }) Result: - Hit(s): 17590 Items - Updated: 0 Items - Printed: 454 KB - Read Locking: global - Write Locking: none Timing: - Parsing: 14.83 ms - Compiling: 198.69 ms - Evaluating: 991.26 ms - Printing: 31.57 ms - Total Time: 1236.35 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$n" id="6"/> <IterPath> <DBNodeSeq size="1759"> <DBNode name="20150324_058" pre="0"/> <DBNode name="20150324_058" pre="4945"/> <DBNode name="20150324_058" pre="6786"/> <DBNode name="20150324_058" pre="9742"/> <DBNode name="20150324_058" pre="14138"/> </DBNodeSeq> <IterStep axis="child" test="*:TED_EXPORT"/> </IterPath> </For> <Let> <Var name="$ti" id="7"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:TRANSLATION_SECTION"/> <IterStep axis="child" test="*:ML_TITLES"/> <IterStep axis="child" test="*:ML_TI_DOC"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </CmpG> </IterStep> </IterPath> </Let> <Let> <Var name="$nodocojs" id="9"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:NOTICE_DATA"/> <IterStep axis="child" test="*:NO_DOC_OJS"/> </IterPath> </Let> <List> <CElem> <QNm value="line" type="xs:QName"/> <Str value="1.0 " type="xs:string"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="TI: " type="xs:string"/> <MixedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_CY"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <CachedPath> <IterStep axis="child" test="text()"/> </CachedPath> <Str value="die " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> <Str value="das " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> </MixedPath> <Str value="-" type="xs:string"/> <CachedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_TOWN"/> <IterStep axis="child" test="text()"/> </CachedPath> <Str value=": " type="xs:string"/> <Empty size="0"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="PD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="ND: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <Cast type="xs:string?"> <Cast type="xs:double?"> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="-" type="xs:string"/> </FnTokenize> <Int value="2" type="xs:integer"/> </BaseXItemAt> </Cast> </Cast> <Str value="-" type="xs:string"/> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="/" type="xs:string"/> </FnTokenize> <Int value="1" type="xs:integer"/> </BaseXItemAt> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="OJ: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:NO_OJ"/> </IterPath> <Str value="/" type="xs:string"/> <FnSubstring name="substring(string,start[,len])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> </IterPath> <Int value="1" type="xs:integer"/> <Int value="4" type="xs:integer"/> </FnSubstring> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DS: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DS_DATE_DISPATCH"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DT: " type="xs:string"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DT_DATE_FOR_SUBMISSION"/> </IterPath> <Str value=" " type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> <Str value=":" type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="HD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:HEADING"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB: " type="xs:string"/> <IterPath> <DBNode name="Labels" pre="0"/> <IterStep axis="child" test="*:LABELS"/> <IterStep axis="child" test="*:LABEL"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:label"/> </CachedPath> <Str value="_and" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="*:TERM"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB_FR: " type="xs:string"/> <Empty size="0"/> </CElem> </List> </GFLWOR> </QueryPlan><<<END_OUTPUT -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 14:00 An: Goetz Heller Cc: BaseX Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on. You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
Phew, it's really difficult to say anything on this.As long as we cannot reproduce this on at least a second machine, it's hardly possible to tell what's happening here. As indicated, I haven't experienced a similar behavior so far, neither on my own system, which is pretty similar to yours, nor on any server machines in our productive instances. But of course I couldn't seriously recommend to go on with your current setup as long as the problem persists. If you want to continue working on this, maybe some more questions: 1. Does this only happen in the GUI or also on command line? 2. Do you have any other JVM instance running beside the single BaseX GUI instance? 3. Do you get the same error when restarting the GUI? Christian On Wed, Mar 25, 2015 at 3:03 PM, Goetz Heller <heller@hellerim.de> wrote:
Hi Christian,
this time, both the function you've seen already and a different query which tries to extract data from a larger Xml file fail with the behavior described. In both query execution plans you can see that there is some "optimization" which places an empty pair of parentheses into the optimized query where something different should be. It looks as if an shortage of some - memory? - resources throws an exception which is silently caught somewhere. I know from my own programming experience that it happens like that: a boring nuisance is caught in the middle of something nice being developed so you neutralize this effect temporarily - and then simply forget it. Is it possible that something similar happened in this case?
The second query is too complicated to simply reduce its complexity. But I suppose you would be able to get the same behavior if you took a machine like mine - a laptop with 8 GB RAM and an Intel dual core processor running at 1,8 GHz and Win 8.1. Possibly reduce the RAM by 4 GB, or run a VMWare virtual machine on it (what I cannot do here because Microsoft's Hypervisor is installed), and then experiment using your tools at hand. I had a Microsoft VS 2013 Community Edition installed, several Eclipse versions, and Node.js (none of these running when the problem showed up). No Microsoft Office. Unfortunately, I cannot invest to much time investigating the issue since I have a lot of other things to do, but let me know anyway if I can give you further information. At first sight, I liked BaseX very much for its simple installation and ease of use, its small footprint and its well-thought GUI, but I will not be able to use it in a project if I cannot rely on the results it delivers.
Kind regards,
Goetz
P.S. Here the query plan for the second query; you will see that the $ti variable is not resolved appropriately (line 37 of output).
BEGIN_OUTPUT>>> Compiling: - inlining $retVal_5 - simplifying flwor expression - pre-evaluating doc("C:\test\Labels.xml") - rewriting (compare(@*:label, $key_0) = 0) - rewriting (compare(@*:LG, "DE") = 0) - inlining $retVal_1 - simplifying flwor expression - rewriting (compare(@*:label, $key_2) = 0) - rewriting (compare(@*:LG, $lang_3) = 0) - rewriting (tokenize($nodocojs_9, "-"))[position() = 2] - inlining local:slz#1 - inlining $arg_15 - simplifying flwor expression - rewriting (tokenize($nodocojs_9, "/"))[position() = 1] - removing unknown element/attribute text - pre-evaluating $ti_7/*:TI_TEXT/() - inlining local:getLabel#1 - inlining $key_16 - simplifying flwor expression - inlining local:getLabel2#2 - removing redundant $lang_18 as xs:string cast. - inlining $key_17 - inlining $lang_18 - removing unknown element/attribute TERM - pre-evaluating document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/()/text() - simplifying flwor expression - inlining $pd_8 - inlining $nd_10 - inlining $oj_11 - inlining $ds_12 - inlining $dt_13 - inlining $hd_14 Query: declare variable $lang as xs:string := 'DE'; declare variable $labels := doc('C:\test\Labels.xml'); declare function local:getLabel( $key ) { let $retVal := $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() return $retVal }; declare function local:getLabel2( $key, $lang as xs:string ) { $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() }; declare function local:slz( $arg ) { let $retVal := xs:string(xs:double($arg)) return $retVal }; for $n in(/TED_EXPORT) let $ti := $n/TRANSLATION_SECTION/ML_TITLES/ML_TI_DOC[@LG=$lang] let $pd := $n/CODED_DATA_SECTION/REF_OJS/DATE_PUB/text() let $nodocojs := $n/CODED_DATA_SECTION/NOTICE_DATA/NO_DOC_OJS let $nd := concat(local:slz(tokenize($nodocojs, '-')[2]), '-', tokenize($nodocojs, '/')[1]) let $oj := concat($n/CODED_DATA_SECTION/REF_OJS/NO_OJ, '/', substring($n/CODED_DATA_SECTION/REF_OJS/DATE_PUB, 1, 4)) let $ds := $n/CODED_DATA_SECTION/CODIF_DATA/DS_DATE_DISPATCH/text() let $dt := replace(replace($n/CODED_DATA_SECTION/CODIF_DATA/DT_DATE_FOR_SUBMISSION, ' ', ' '), ':', ' ') let $hd := $n/CODED_DATA_SECTION/CODIF_DATA/HEADING/text() return ( <line>1.0 </line>, <line>TI: {$ti/TI_CY/replace(replace(text(), 'die ', ''), 'das ', '')}-{$ti/TI_TOWN/text()}: {$ti/TI_TEXT/text}</line>, <line>PD: {$pd}</line>, <line>ND: {$nd}</line>, <line>OJ: {$oj}</line>, <line>DS: {$ds}</line>, <line>DT: {$dt}</line>, <line>HD: {$hd}</line>, <line>LB: {local:getLabel('_and')}</line>, <line>LB_FR: {local:getLabel2('_and', 'FR')}</line> ) Optimized Query: for $n_6 in (db:open-pre("20150324_058",0), ...)/*:TED_EXPORT let $ti_7 := $n_6/*:TRANSLATION_SECTION/*:ML_TITLES/*:ML_TI_DOC[(@*:LG = "DE")] let $nodocojs_9 := $n_6/*:CODED_DATA_SECTION/*:NOTICE_DATA/*:NO_DOC_OJS return (element line { ("1.0 ") }, element line { ("TI: ", $ti_7/*:TI_CY/replace(replace(text(), "die ", ""), "das ", ""), "-", $ti_7/*:TI_TOWN/text(), ": ", ()) }, element line { ("PD: ", $n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB/text()) }, element line { ("ND: ", concat(basex:item-at(tokenize($nodocojs_9, "-"), 2) cast as xs:double? cast as xs:string?, "-", basex:item-at(tokenize($nodocojs_9, "/"), 1))) }, element line { ("OJ: ", concat($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:NO_OJ, "/", substring($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB, 1, 4))) }, element line { ("DS: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DS_DATE_DISPATCH/text()) }, element line { ("DT: ", replace(replace($n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DT_DATE_FOR_SUBMISSION, " ", " "), ":", " ")) }, element line { ("HD: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:HEADING/text()) }, element line { ("LB: ", document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/*:TERM[compare(@*:LG, "DE") = 0.0]/text()) }, element line { ("LB_FR: ", ()) }) Result: - Hit(s): 17590 Items - Updated: 0 Items - Printed: 454 KB - Read Locking: global - Write Locking: none Timing: - Parsing: 14.83 ms - Compiling: 198.69 ms - Evaluating: 991.26 ms - Printing: 31.57 ms - Total Time: 1236.35 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$n" id="6"/> <IterPath> <DBNodeSeq size="1759"> <DBNode name="20150324_058" pre="0"/> <DBNode name="20150324_058" pre="4945"/> <DBNode name="20150324_058" pre="6786"/> <DBNode name="20150324_058" pre="9742"/> <DBNode name="20150324_058" pre="14138"/> </DBNodeSeq> <IterStep axis="child" test="*:TED_EXPORT"/> </IterPath> </For> <Let> <Var name="$ti" id="7"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:TRANSLATION_SECTION"/> <IterStep axis="child" test="*:ML_TITLES"/> <IterStep axis="child" test="*:ML_TI_DOC"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </CmpG> </IterStep> </IterPath> </Let> <Let> <Var name="$nodocojs" id="9"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:NOTICE_DATA"/> <IterStep axis="child" test="*:NO_DOC_OJS"/> </IterPath> </Let> <List> <CElem> <QNm value="line" type="xs:QName"/> <Str value="1.0 " type="xs:string"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="TI: " type="xs:string"/> <MixedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_CY"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <CachedPath> <IterStep axis="child" test="text()"/> </CachedPath> <Str value="die " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> <Str value="das " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> </MixedPath> <Str value="-" type="xs:string"/> <CachedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_TOWN"/> <IterStep axis="child" test="text()"/> </CachedPath> <Str value=": " type="xs:string"/> <Empty size="0"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="PD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="ND: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <Cast type="xs:string?"> <Cast type="xs:double?"> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="-" type="xs:string"/> </FnTokenize> <Int value="2" type="xs:integer"/> </BaseXItemAt> </Cast> </Cast> <Str value="-" type="xs:string"/> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="/" type="xs:string"/> </FnTokenize> <Int value="1" type="xs:integer"/> </BaseXItemAt> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="OJ: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:NO_OJ"/> </IterPath> <Str value="/" type="xs:string"/> <FnSubstring name="substring(string,start[,len])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> </IterPath> <Int value="1" type="xs:integer"/> <Int value="4" type="xs:integer"/> </FnSubstring> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DS: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DS_DATE_DISPATCH"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DT: " type="xs:string"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DT_DATE_FOR_SUBMISSION"/> </IterPath> <Str value=" " type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> <Str value=":" type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="HD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:HEADING"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB: " type="xs:string"/> <IterPath> <DBNode name="Labels" pre="0"/> <IterStep axis="child" test="*:LABELS"/> <IterStep axis="child" test="*:LABEL"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:label"/> </CachedPath> <Str value="_and" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="*:TERM"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB_FR: " type="xs:string"/> <Empty size="0"/> </CElem> </List> </GFLWOR> </QueryPlan><<<END_OUTPUT
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 14:00 An: Goetz Heller Cc: BaseX Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on.
You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
Note that I'm not using the GUI, I'm using the basexclient command via a bash script. I have the basexserver running as a "service" (meaning it's a background task under OS X and then I'm using git commit hooks to run bash scripts that call the basexclient command. Cheers, E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com On 3/25/15, 12:54 PM, "Christian Grün" <christian.gruen@gmail.com> wrote:
Phew, it's really difficult to say anything on this.As long as we cannot reproduce this on at least a second machine, it's hardly possible to tell what's happening here.
As indicated, I haven't experienced a similar behavior so far, neither on my own system, which is pretty similar to yours, nor on any server machines in our productive instances. But of course I couldn't seriously recommend to go on with your current setup as long as the problem persists.
If you want to continue working on this, maybe some more questions:
1. Does this only happen in the GUI or also on command line? 2. Do you have any other JVM instance running beside the single BaseX GUI instance? 3. Do you get the same error when restarting the GUI?
Christian
On Wed, Mar 25, 2015 at 3:03 PM, Goetz Heller <heller@hellerim.de> wrote:
Hi Christian,
this time, both the function you've seen already and a different query which tries to extract data from a larger Xml file fail with the behavior described. In both query execution plans you can see that there is some "optimization" which places an empty pair of parentheses into the optimized query where something different should be. It looks as if an shortage of some - memory? - resources throws an exception which is silently caught somewhere. I know from my own programming experience that it happens like that: a boring nuisance is caught in the middle of something nice being developed so you neutralize this effect temporarily - and then simply forget it. Is it possible that something similar happened in this case?
The second query is too complicated to simply reduce its complexity. But I suppose you would be able to get the same behavior if you took a machine like mine - a laptop with 8 GB RAM and an Intel dual core processor running at 1,8 GHz and Win 8.1. Possibly reduce the RAM by 4 GB, or run a VMWare virtual machine on it (what I cannot do here because Microsoft's Hypervisor is installed), and then experiment using your tools at hand. I had a Microsoft VS 2013 Community Edition installed, several Eclipse versions, and Node.js (none of these running when the problem showed up). No Microsoft Office. Unfortunately, I cannot invest to much time investigating the issue since I have a lot of other things to do, but let me know anyway if I can give you further information. At first sight, I liked BaseX very much for its simple installation and ease of use, its small footprint and its well-thought GUI, but I will not be able to use it in a project if I cannot rely on the results it delivers.
Kind regards,
Goetz
P.S. Here the query plan for the second query; you will see that the $ti variable is not resolved appropriately (line 37 of output).
BEGIN_OUTPUT>>> Compiling: - inlining $retVal_5 - simplifying flwor expression - pre-evaluating doc("C:\test\Labels.xml") - rewriting (compare(@*:label, $key_0) = 0) - rewriting (compare(@*:LG, "DE") = 0) - inlining $retVal_1 - simplifying flwor expression - rewriting (compare(@*:label, $key_2) = 0) - rewriting (compare(@*:LG, $lang_3) = 0) - rewriting (tokenize($nodocojs_9, "-"))[position() = 2] - inlining local:slz#1 - inlining $arg_15 - simplifying flwor expression - rewriting (tokenize($nodocojs_9, "/"))[position() = 1] - removing unknown element/attribute text - pre-evaluating $ti_7/*:TI_TEXT/() - inlining local:getLabel#1 - inlining $key_16 - simplifying flwor expression - inlining local:getLabel2#2 - removing redundant $lang_18 as xs:string cast. - inlining $key_17 - inlining $lang_18 - removing unknown element/attribute TERM - pre-evaluating document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/()/text() - simplifying flwor expression - inlining $pd_8 - inlining $nd_10 - inlining $oj_11 - inlining $ds_12 - inlining $dt_13 - inlining $hd_14 Query: declare variable $lang as xs:string := 'DE'; declare variable $labels := doc('C:\test\Labels.xml'); declare function local:getLabel( $key ) { let $retVal := $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() return $retVal }; declare function local:getLabel2( $key, $lang as xs:string ) { $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() }; declare function local:slz( $arg ) { let $retVal := xs:string(xs:double($arg)) return $retVal }; for $n in(/TED_EXPORT) let $ti := $n/TRANSLATION_SECTION/ML_TITLES/ML_TI_DOC[@LG=$lang] let $pd := $n/CODED_DATA_SECTION/REF_OJS/DATE_PUB/text() let $nodocojs := $n/CODED_DATA_SECTION/NOTICE_DATA/NO_DOC_OJS let $nd := concat(local:slz(tokenize($nodocojs, '-')[2]), '-', tokenize($nodocojs, '/')[1]) let $oj := concat($n/CODED_DATA_SECTION/REF_OJS/NO_OJ, '/', substring($n/CODED_DATA_SECTION/REF_OJS/DATE_PUB, 1, 4)) let $ds := $n/CODED_DATA_SECTION/CODIF_DATA/DS_DATE_DISPATCH/text() let $dt := replace(replace($n/CODED_DATA_SECTION/CODIF_DATA/DT_DATE_FOR_SUBMISSION, ' ', ' '), ':', ' ') let $hd := $n/CODED_DATA_SECTION/CODIF_DATA/HEADING/text() return ( <line>1.0 </line>, <line>TI: {$ti/TI_CY/replace(replace(text(), 'die ', ''), 'das ', '')}-{$ti/TI_TOWN/text()}: {$ti/TI_TEXT/text}</line>, <line>PD: {$pd}</line>, <line>ND: {$nd}</line>, <line>OJ: {$oj}</line>, <line>DS: {$ds}</line>, <line>DT: {$dt}</line>, <line>HD: {$hd}</line>, <line>LB: {local:getLabel('_and')}</line>, <line>LB_FR: {local:getLabel2('_and', 'FR')}</line> ) Optimized Query: for $n_6 in (db:open-pre("20150324_058",0), ...)/*:TED_EXPORT let $ti_7 := $n_6/*:TRANSLATION_SECTION/*:ML_TITLES/*:ML_TI_DOC[(@*:LG = "DE")] let $nodocojs_9 := $n_6/*:CODED_DATA_SECTION/*:NOTICE_DATA/*:NO_DOC_OJS return (element line { ("1.0 ") }, element line { ("TI: ", $ti_7/*:TI_CY/replace(replace(text(), "die ", ""), "das ", ""), "-", $ti_7/*:TI_TOWN/text(), ": ", ()) }, element line { ("PD: ", $n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB/text()) }, element line { ("ND: ", concat(basex:item-at(tokenize($nodocojs_9, "-"), 2) cast as xs:double? cast as xs:string?, "-", basex:item-at(tokenize($nodocojs_9, "/"), 1))) }, element line { ("OJ: ", concat($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:NO_OJ, "/", substring($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB, 1, 4))) }, element line { ("DS: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DS_DATE_DISPATCH/text()) }, element line { ("DT: ", replace(replace($n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DT_DATE_FOR_SUBM ISSION, " ", " "), ":", " ")) }, element line { ("HD: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:HEADING/text()) }, element line { ("LB: ", document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/*:TERM[compare(@*:LG, "DE") = 0.0]/text()) }, element line { ("LB_FR: ", ()) }) Result: - Hit(s): 17590 Items - Updated: 0 Items - Printed: 454 KB - Read Locking: global - Write Locking: none Timing: - Parsing: 14.83 ms - Compiling: 198.69 ms - Evaluating: 991.26 ms - Printing: 31.57 ms - Total Time: 1236.35 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$n" id="6"/> <IterPath> <DBNodeSeq size="1759"> <DBNode name="20150324_058" pre="0"/> <DBNode name="20150324_058" pre="4945"/> <DBNode name="20150324_058" pre="6786"/> <DBNode name="20150324_058" pre="9742"/> <DBNode name="20150324_058" pre="14138"/> </DBNodeSeq> <IterStep axis="child" test="*:TED_EXPORT"/> </IterPath> </For> <Let> <Var name="$ti" id="7"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:TRANSLATION_SECTION"/> <IterStep axis="child" test="*:ML_TITLES"/> <IterStep axis="child" test="*:ML_TI_DOC"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </CmpG> </IterStep> </IterPath> </Let> <Let> <Var name="$nodocojs" id="9"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:NOTICE_DATA"/> <IterStep axis="child" test="*:NO_DOC_OJS"/> </IterPath> </Let> <List> <CElem> <QNm value="line" type="xs:QName"/> <Str value="1.0 " type="xs:string"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="TI: " type="xs:string"/> <MixedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_CY"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <CachedPath> <IterStep axis="child" test="text()"/> </CachedPath> <Str value="die " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> <Str value="das " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> </MixedPath> <Str value="-" type="xs:string"/> <CachedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_TOWN"/> <IterStep axis="child" test="text()"/> </CachedPath> <Str value=": " type="xs:string"/> <Empty size="0"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="PD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="ND: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <Cast type="xs:string?"> <Cast type="xs:double?"> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="-" type="xs:string"/> </FnTokenize> <Int value="2" type="xs:integer"/> </BaseXItemAt> </Cast> </Cast> <Str value="-" type="xs:string"/> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="/" type="xs:string"/> </FnTokenize> <Int value="1" type="xs:integer"/> </BaseXItemAt> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="OJ: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:NO_OJ"/> </IterPath> <Str value="/" type="xs:string"/> <FnSubstring name="substring(string,start[,len])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> </IterPath> <Int value="1" type="xs:integer"/> <Int value="4" type="xs:integer"/> </FnSubstring> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DS: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DS_DATE_DISPATCH"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DT: " type="xs:string"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DT_DATE_FOR_SUBMISSION"/> </IterPath> <Str value=" " type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> <Str value=":" type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="HD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:HEADING"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB: " type="xs:string"/> <IterPath> <DBNode name="Labels" pre="0"/> <IterStep axis="child" test="*:LABELS"/> <IterStep axis="child" test="*:LABEL"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:label"/> </CachedPath> <Str value="_and" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="*:TERM"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB_FR: " type="xs:string"/> <Empty size="0"/> </CElem> </List> </GFLWOR> </QueryPlan><<<END_OUTPUT
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 14:00 An: Goetz Heller Cc: BaseX Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on.
You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
Just to avoid confusion.. Are both of you working on the same project? On Wed, Mar 25, 2015 at 7:19 PM, Eliot Kimber <ekimber@contrext.com> wrote:
Note that I'm not using the GUI, I'm using the basexclient command via a bash script.
I have the basexserver running as a "service" (meaning it's a background task under OS X and then I'm using git commit hooks to run bash scripts that call the basexclient command.
Cheers,
E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com
On 3/25/15, 12:54 PM, "Christian Grün" <christian.gruen@gmail.com> wrote:
Phew, it's really difficult to say anything on this.As long as we cannot reproduce this on at least a second machine, it's hardly possible to tell what's happening here.
As indicated, I haven't experienced a similar behavior so far, neither on my own system, which is pretty similar to yours, nor on any server machines in our productive instances. But of course I couldn't seriously recommend to go on with your current setup as long as the problem persists.
If you want to continue working on this, maybe some more questions:
1. Does this only happen in the GUI or also on command line? 2. Do you have any other JVM instance running beside the single BaseX GUI instance? 3. Do you get the same error when restarting the GUI?
Christian
On Wed, Mar 25, 2015 at 3:03 PM, Goetz Heller <heller@hellerim.de> wrote:
Hi Christian,
this time, both the function you've seen already and a different query which tries to extract data from a larger Xml file fail with the behavior described. In both query execution plans you can see that there is some "optimization" which places an empty pair of parentheses into the optimized query where something different should be. It looks as if an shortage of some - memory? - resources throws an exception which is silently caught somewhere. I know from my own programming experience that it happens like that: a boring nuisance is caught in the middle of something nice being developed so you neutralize this effect temporarily - and then simply forget it. Is it possible that something similar happened in this case?
The second query is too complicated to simply reduce its complexity. But I suppose you would be able to get the same behavior if you took a machine like mine - a laptop with 8 GB RAM and an Intel dual core processor running at 1,8 GHz and Win 8.1. Possibly reduce the RAM by 4 GB, or run a VMWare virtual machine on it (what I cannot do here because Microsoft's Hypervisor is installed), and then experiment using your tools at hand. I had a Microsoft VS 2013 Community Edition installed, several Eclipse versions, and Node.js (none of these running when the problem showed up). No Microsoft Office. Unfortunately, I cannot invest to much time investigating the issue since I have a lot of other things to do, but let me know anyway if I can give you further information. At first sight, I liked BaseX very much for its simple installation and ease of use, its small footprint and its well-thought GUI, but I will not be able to use it in a project if I cannot rely on the results it delivers.
Kind regards,
Goetz
P.S. Here the query plan for the second query; you will see that the $ti variable is not resolved appropriately (line 37 of output).
BEGIN_OUTPUT>>> Compiling: - inlining $retVal_5 - simplifying flwor expression - pre-evaluating doc("C:\test\Labels.xml") - rewriting (compare(@*:label, $key_0) = 0) - rewriting (compare(@*:LG, "DE") = 0) - inlining $retVal_1 - simplifying flwor expression - rewriting (compare(@*:label, $key_2) = 0) - rewriting (compare(@*:LG, $lang_3) = 0) - rewriting (tokenize($nodocojs_9, "-"))[position() = 2] - inlining local:slz#1 - inlining $arg_15 - simplifying flwor expression - rewriting (tokenize($nodocojs_9, "/"))[position() = 1] - removing unknown element/attribute text - pre-evaluating $ti_7/*:TI_TEXT/() - inlining local:getLabel#1 - inlining $key_16 - simplifying flwor expression - inlining local:getLabel2#2 - removing redundant $lang_18 as xs:string cast. - inlining $key_17 - inlining $lang_18 - removing unknown element/attribute TERM - pre-evaluating document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/()/text() - simplifying flwor expression - inlining $pd_8 - inlining $nd_10 - inlining $oj_11 - inlining $ds_12 - inlining $dt_13 - inlining $hd_14 Query: declare variable $lang as xs:string := 'DE'; declare variable $labels := doc('C:\test\Labels.xml'); declare function local:getLabel( $key ) { let $retVal := $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() return $retVal }; declare function local:getLabel2( $key, $lang as xs:string ) { $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() }; declare function local:slz( $arg ) { let $retVal := xs:string(xs:double($arg)) return $retVal }; for $n in(/TED_EXPORT) let $ti := $n/TRANSLATION_SECTION/ML_TITLES/ML_TI_DOC[@LG=$lang] let $pd := $n/CODED_DATA_SECTION/REF_OJS/DATE_PUB/text() let $nodocojs := $n/CODED_DATA_SECTION/NOTICE_DATA/NO_DOC_OJS let $nd := concat(local:slz(tokenize($nodocojs, '-')[2]), '-', tokenize($nodocojs, '/')[1]) let $oj := concat($n/CODED_DATA_SECTION/REF_OJS/NO_OJ, '/', substring($n/CODED_DATA_SECTION/REF_OJS/DATE_PUB, 1, 4)) let $ds := $n/CODED_DATA_SECTION/CODIF_DATA/DS_DATE_DISPATCH/text() let $dt := replace(replace($n/CODED_DATA_SECTION/CODIF_DATA/DT_DATE_FOR_SUBMISSION, ' ', ' '), ':', ' ') let $hd := $n/CODED_DATA_SECTION/CODIF_DATA/HEADING/text() return ( <line>1.0 </line>, <line>TI: {$ti/TI_CY/replace(replace(text(), 'die ', ''), 'das ', '')}-{$ti/TI_TOWN/text()}: {$ti/TI_TEXT/text}</line>, <line>PD: {$pd}</line>, <line>ND: {$nd}</line>, <line>OJ: {$oj}</line>, <line>DS: {$ds}</line>, <line>DT: {$dt}</line>, <line>HD: {$hd}</line>, <line>LB: {local:getLabel('_and')}</line>, <line>LB_FR: {local:getLabel2('_and', 'FR')}</line> ) Optimized Query: for $n_6 in (db:open-pre("20150324_058",0), ...)/*:TED_EXPORT let $ti_7 := $n_6/*:TRANSLATION_SECTION/*:ML_TITLES/*:ML_TI_DOC[(@*:LG = "DE")] let $nodocojs_9 := $n_6/*:CODED_DATA_SECTION/*:NOTICE_DATA/*:NO_DOC_OJS return (element line { ("1.0 ") }, element line { ("TI: ", $ti_7/*:TI_CY/replace(replace(text(), "die ", ""), "das ", ""), "-", $ti_7/*:TI_TOWN/text(), ": ", ()) }, element line { ("PD: ", $n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB/text()) }, element line { ("ND: ", concat(basex:item-at(tokenize($nodocojs_9, "-"), 2) cast as xs:double? cast as xs:string?, "-", basex:item-at(tokenize($nodocojs_9, "/"), 1))) }, element line { ("OJ: ", concat($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:NO_OJ, "/", substring($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB, 1, 4))) }, element line { ("DS: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DS_DATE_DISPATCH/text()) }, element line { ("DT: ", replace(replace($n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DT_DATE_FOR_SUBM ISSION, " ", " "), ":", " ")) }, element line { ("HD: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:HEADING/text()) }, element line { ("LB: ", document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/*:TERM[compare(@*:LG, "DE") = 0.0]/text()) }, element line { ("LB_FR: ", ()) }) Result: - Hit(s): 17590 Items - Updated: 0 Items - Printed: 454 KB - Read Locking: global - Write Locking: none Timing: - Parsing: 14.83 ms - Compiling: 198.69 ms - Evaluating: 991.26 ms - Printing: 31.57 ms - Total Time: 1236.35 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$n" id="6"/> <IterPath> <DBNodeSeq size="1759"> <DBNode name="20150324_058" pre="0"/> <DBNode name="20150324_058" pre="4945"/> <DBNode name="20150324_058" pre="6786"/> <DBNode name="20150324_058" pre="9742"/> <DBNode name="20150324_058" pre="14138"/> </DBNodeSeq> <IterStep axis="child" test="*:TED_EXPORT"/> </IterPath> </For> <Let> <Var name="$ti" id="7"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:TRANSLATION_SECTION"/> <IterStep axis="child" test="*:ML_TITLES"/> <IterStep axis="child" test="*:ML_TI_DOC"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </CmpG> </IterStep> </IterPath> </Let> <Let> <Var name="$nodocojs" id="9"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:NOTICE_DATA"/> <IterStep axis="child" test="*:NO_DOC_OJS"/> </IterPath> </Let> <List> <CElem> <QNm value="line" type="xs:QName"/> <Str value="1.0 " type="xs:string"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="TI: " type="xs:string"/> <MixedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_CY"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <CachedPath> <IterStep axis="child" test="text()"/> </CachedPath> <Str value="die " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> <Str value="das " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> </MixedPath> <Str value="-" type="xs:string"/> <CachedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_TOWN"/> <IterStep axis="child" test="text()"/> </CachedPath> <Str value=": " type="xs:string"/> <Empty size="0"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="PD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="ND: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <Cast type="xs:string?"> <Cast type="xs:double?"> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="-" type="xs:string"/> </FnTokenize> <Int value="2" type="xs:integer"/> </BaseXItemAt> </Cast> </Cast> <Str value="-" type="xs:string"/> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="/" type="xs:string"/> </FnTokenize> <Int value="1" type="xs:integer"/> </BaseXItemAt> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="OJ: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:NO_OJ"/> </IterPath> <Str value="/" type="xs:string"/> <FnSubstring name="substring(string,start[,len])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> </IterPath> <Int value="1" type="xs:integer"/> <Int value="4" type="xs:integer"/> </FnSubstring> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DS: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DS_DATE_DISPATCH"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DT: " type="xs:string"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DT_DATE_FOR_SUBMISSION"/> </IterPath> <Str value=" " type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> <Str value=":" type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="HD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:HEADING"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB: " type="xs:string"/> <IterPath> <DBNode name="Labels" pre="0"/> <IterStep axis="child" test="*:LABELS"/> <IterStep axis="child" test="*:LABEL"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:label"/> </CachedPath> <Str value="_and" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="*:TERM"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB_FR: " type="xs:string"/> <Empty size="0"/> </CElem> </List> </GFLWOR> </QueryPlan><<<END_OUTPUT
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 14:00 An: Goetz Heller Cc: BaseX Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on.
You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
Sorry--just realized I confused the threads. Ignore me. Cheers, E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com On 3/25/15, 1:19 PM, "Eliot Kimber" <ekimber@contrext.com> wrote:
Note that I'm not using the GUI, I'm using the basexclient command via a bash script.
I have the basexserver running as a "service" (meaning it's a background task under OS X and then I'm using git commit hooks to run bash scripts that call the basexclient command.
Cheers,
E. ————— Eliot Kimber, Owner Contrext, LLC http://contrext.com
On 3/25/15, 12:54 PM, "Christian Grün" <christian.gruen@gmail.com> wrote:
Phew, it's really difficult to say anything on this.As long as we cannot reproduce this on at least a second machine, it's hardly possible to tell what's happening here.
As indicated, I haven't experienced a similar behavior so far, neither on my own system, which is pretty similar to yours, nor on any server machines in our productive instances. But of course I couldn't seriously recommend to go on with your current setup as long as the problem persists.
If you want to continue working on this, maybe some more questions:
1. Does this only happen in the GUI or also on command line? 2. Do you have any other JVM instance running beside the single BaseX GUI instance? 3. Do you get the same error when restarting the GUI?
Christian
On Wed, Mar 25, 2015 at 3:03 PM, Goetz Heller <heller@hellerim.de> wrote:
Hi Christian,
this time, both the function you've seen already and a different query which tries to extract data from a larger Xml file fail with the behavior described. In both query execution plans you can see that there is some "optimization" which places an empty pair of parentheses into the optimized query where something different should be. It looks as if an shortage of some - memory? - resources throws an exception which is silently caught somewhere. I know from my own programming experience that it happens like that: a boring nuisance is caught in the middle of something nice being developed so you neutralize this effect temporarily - and then simply forget it. Is it possible that something similar happened in this case?
The second query is too complicated to simply reduce its complexity. But I suppose you would be able to get the same behavior if you took a machine like mine - a laptop with 8 GB RAM and an Intel dual core processor running at 1,8 GHz and Win 8.1. Possibly reduce the RAM by 4 GB, or run a VMWare virtual machine on it (what I cannot do here because Microsoft's Hypervisor is installed), and then experiment using your tools at hand. I had a Microsoft VS 2013 Community Edition installed, several Eclipse versions, and Node.js (none of these running when the problem showed up). No Microsoft Office. Unfortunately, I cannot invest to much time investigating the issue since I have a lot of other things to do, but let me know anyway if I can give you further information. At first sight, I liked BaseX very much for its simple installation and ease of use, its small footprint and its well-thought GUI, but I will not be able to use it in a project if I cannot rely on the results it delivers.
Kind regards,
Goetz
P.S. Here the query plan for the second query; you will see that the $ti variable is not resolved appropriately (line 37 of output).
BEGIN_OUTPUT>>> Compiling: - inlining $retVal_5 - simplifying flwor expression - pre-evaluating doc("C:\test\Labels.xml") - rewriting (compare(@*:label, $key_0) = 0) - rewriting (compare(@*:LG, "DE") = 0) - inlining $retVal_1 - simplifying flwor expression - rewriting (compare(@*:label, $key_2) = 0) - rewriting (compare(@*:LG, $lang_3) = 0) - rewriting (tokenize($nodocojs_9, "-"))[position() = 2] - inlining local:slz#1 - inlining $arg_15 - simplifying flwor expression - rewriting (tokenize($nodocojs_9, "/"))[position() = 1] - removing unknown element/attribute text - pre-evaluating $ti_7/*:TI_TEXT/() - inlining local:getLabel#1 - inlining $key_16 - simplifying flwor expression - inlining local:getLabel2#2 - removing redundant $lang_18 as xs:string cast. - inlining $key_17 - inlining $lang_18 - removing unknown element/attribute TERM - pre-evaluating document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/()/text() - simplifying flwor expression - inlining $pd_8 - inlining $nd_10 - inlining $oj_11 - inlining $ds_12 - inlining $dt_13 - inlining $hd_14 Query: declare variable $lang as xs:string := 'DE'; declare variable $labels := doc('C:\test\Labels.xml'); declare function local:getLabel( $key ) { let $retVal := $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() return $retVal }; declare function local:getLabel2( $key, $lang as xs:string ) { $labels/LABELS/LABEL[compare(@label, $key) = 0]/TERM[compare(@LG, $lang) = 0]/text() }; declare function local:slz( $arg ) { let $retVal := xs:string(xs:double($arg)) return $retVal }; for $n in(/TED_EXPORT) let $ti := $n/TRANSLATION_SECTION/ML_TITLES/ML_TI_DOC[@LG=$lang] let $pd := $n/CODED_DATA_SECTION/REF_OJS/DATE_PUB/text() let $nodocojs := $n/CODED_DATA_SECTION/NOTICE_DATA/NO_DOC_OJS let $nd := concat(local:slz(tokenize($nodocojs, '-')[2]), '-', tokenize($nodocojs, '/')[1]) let $oj := concat($n/CODED_DATA_SECTION/REF_OJS/NO_OJ, '/', substring($n/CODED_DATA_SECTION/REF_OJS/DATE_PUB, 1, 4)) let $ds := $n/CODED_DATA_SECTION/CODIF_DATA/DS_DATE_DISPATCH/text() let $dt := replace(replace($n/CODED_DATA_SECTION/CODIF_DATA/DT_DATE_FOR_SUBMISSION, ' ', ' '), ':', ' ') let $hd := $n/CODED_DATA_SECTION/CODIF_DATA/HEADING/text() return ( <line>1.0 </line>, <line>TI: {$ti/TI_CY/replace(replace(text(), 'die ', ''), 'das ', '')}-{$ti/TI_TOWN/text()}: {$ti/TI_TEXT/text}</line>, <line>PD: {$pd}</line>, <line>ND: {$nd}</line>, <line>OJ: {$oj}</line>, <line>DS: {$ds}</line>, <line>DT: {$dt}</line>, <line>HD: {$hd}</line>, <line>LB: {local:getLabel('_and')}</line>, <line>LB_FR: {local:getLabel2('_and', 'FR')}</line> ) Optimized Query: for $n_6 in (db:open-pre("20150324_058",0), ...)/*:TED_EXPORT let $ti_7 := $n_6/*:TRANSLATION_SECTION/*:ML_TITLES/*:ML_TI_DOC[(@*:LG = "DE")] let $nodocojs_9 := $n_6/*:CODED_DATA_SECTION/*:NOTICE_DATA/*:NO_DOC_OJS return (element line { ("1.0 ") }, element line { ("TI: ", $ti_7/*:TI_CY/replace(replace(text(), "die ", ""), "das ", ""), "-", $ti_7/*:TI_TOWN/text(), ": ", ()) }, element line { ("PD: ", $n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB/text()) }, element line { ("ND: ", concat(basex:item-at(tokenize($nodocojs_9, "-"), 2) cast as xs:double? cast as xs:string?, "-", basex:item-at(tokenize($nodocojs_9, "/"), 1))) }, element line { ("OJ: ", concat($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:NO_OJ, "/", substring($n_6/*:CODED_DATA_SECTION/*:REF_OJS/*:DATE_PUB, 1, 4))) }, element line { ("DS: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DS_DATE_DISPATCH/text()) }, element line { ("DT: ", replace(replace($n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:DT_DATE_FOR_SUB M ISSION, " ", " "), ":", " ")) }, element line { ("HD: ", $n_6/*:CODED_DATA_SECTION/*:CODIF_DATA/*:HEADING/text()) }, element line { ("LB: ", document-node {"Labels.xml"}/*:LABELS/*:LABEL[compare(@*:label, "_and") = 0.0]/*:TERM[compare(@*:LG, "DE") = 0.0]/text()) }, element line { ("LB_FR: ", ()) }) Result: - Hit(s): 17590 Items - Updated: 0 Items - Printed: 454 KB - Read Locking: global - Write Locking: none Timing: - Parsing: 14.83 ms - Compiling: 198.69 ms - Evaluating: 991.26 ms - Printing: 31.57 ms - Total Time: 1236.35 ms Query plan: <QueryPlan compiled="true"> <GFLWOR> <For> <Var name="$n" id="6"/> <IterPath> <DBNodeSeq size="1759"> <DBNode name="20150324_058" pre="0"/> <DBNode name="20150324_058" pre="4945"/> <DBNode name="20150324_058" pre="6786"/> <DBNode name="20150324_058" pre="9742"/> <DBNode name="20150324_058" pre="14138"/> </DBNodeSeq> <IterStep axis="child" test="*:TED_EXPORT"/> </IterPath> </For> <Let> <Var name="$ti" id="7"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:TRANSLATION_SECTION"/> <IterStep axis="child" test="*:ML_TITLES"/> <IterStep axis="child" test="*:ML_TI_DOC"> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </CmpG> </IterStep> </IterPath> </Let> <Let> <Var name="$nodocojs" id="9"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:NOTICE_DATA"/> <IterStep axis="child" test="*:NO_DOC_OJS"/> </IterPath> </Let> <List> <CElem> <QNm value="line" type="xs:QName"/> <Str value="1.0 " type="xs:string"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="TI: " type="xs:string"/> <MixedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_CY"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <CachedPath> <IterStep axis="child" test="text()"/> </CachedPath> <Str value="die " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> <Str value="das " type="xs:string"/> <Str value="" type="xs:string"/> </FnReplace> </MixedPath> <Str value="-" type="xs:string"/> <CachedPath> <VarRef> <Var name="$ti" id="7"/> </VarRef> <IterStep axis="child" test="*:TI_TOWN"/> <IterStep axis="child" test="text()"/> </CachedPath> <Str value=": " type="xs:string"/> <Empty size="0"/> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="PD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="ND: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <Cast type="xs:string?"> <Cast type="xs:double?"> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="-" type="xs:string"/> </FnTokenize> <Int value="2" type="xs:integer"/> </BaseXItemAt> </Cast> </Cast> <Str value="-" type="xs:string"/> <BaseXItemAt name="item-at(items,pos)"> <FnTokenize name="tokenize(string[,pattern[,mod]])"> <VarRef> <Var name="$nodocojs" id="9"/> </VarRef> <Str value="/" type="xs:string"/> </FnTokenize> <Int value="1" type="xs:integer"/> </BaseXItemAt> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="OJ: " type="xs:string"/> <FnConcat name="concat(atom1,atom2[,...])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:NO_OJ"/> </IterPath> <Str value="/" type="xs:string"/> <FnSubstring name="substring(string,start[,len])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:REF_OJS"/> <IterStep axis="child" test="*:DATE_PUB"/> </IterPath> <Int value="1" type="xs:integer"/> <Int value="4" type="xs:integer"/> </FnSubstring> </FnConcat> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DS: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DS_DATE_DISPATCH"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="DT: " type="xs:string"/> <FnReplace name="replace(string,pattern,replace[,mod])"> <FnReplace name="replace(string,pattern,replace[,mod])"> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:DT_DATE_FOR_SUBMISSION"/> </IterPath> <Str value=" " type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> <Str value=":" type="xs:string"/> <Str value=" " type="xs:string"/> </FnReplace> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="HD: " type="xs:string"/> <IterPath> <VarRef> <Var name="$n" id="6"/> </VarRef> <IterStep axis="child" test="*:CODED_DATA_SECTION"/> <IterStep axis="child" test="*:CODIF_DATA"/> <IterStep axis="child" test="*:HEADING"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB: " type="xs:string"/> <IterPath> <DBNode name="Labels" pre="0"/> <IterStep axis="child" test="*:LABELS"/> <IterStep axis="child" test="*:LABEL"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:label"/> </CachedPath> <Str value="_and" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="*:TERM"> <CmpR min="0.0" max="0.0"> <FnCompare name="compare(first,second[,collation])"> <CachedPath> <IterStep axis="attribute" test="*:LG"/> </CachedPath> <Str value="DE" type="xs:string"/> </FnCompare> </CmpR> </IterStep> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="line" type="xs:QName"/> <Str value="LB_FR: " type="xs:string"/> <Empty size="0"/> </CElem> </List> </GFLWOR> </QueryPlan><<<END_OUTPUT
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 25. März 2015 14:00 An: Goetz Heller Cc: BaseX Betreff: Re: [basex-talk] Simple xQuery functions do not work as expected
the issue showed up again, this time outside of a function. The computer was only moderately charged, but the change of behavior occurred after I loaded a 25MB text file into Notepad++. Should I change something in the JVM settings?
To be honest, I really have no clue yet what's going on here (I can't remember to have encountered a similar behavior before).. If you want, you could once again try to simplify the example again as much as possible and send it to us. Of course it would be great if you manage to reproduce it on any other machine you can get your hands on.
You are exclusively using the BaseX GUI, right? Does the problem also happen on command-line?
participants (3)
-
Christian Grün -
Eliot Kimber -
Goetz Heller