Hmm, it is surprisingly hard to get some useful logging information out of this. Here for example I fall foul of a XQuery syntax here: “all expressions must be updating or return an empty sequence. ” for $zip in file:list($hm) return try { db:add("myDB", $hm || '\' || $zip, "", map { 'intparse': true() }) } catch * { $err:code } Can anyone help me rephrase this so when BaseX halts on a bad zipfile, I at least know what the name of the zipfile is? C. From: Alexander Holupirek [mailto:alex@holupirek.de] Sent: 16 October 2015 15:23 To: Hondros, Constantine (ELS-AMS) Cc: BaseX Subject: Re: [basex-talk] Which zip did BaseX choke on? Have you tried using try/catch? for zip in zips return try { unzip() } catch ... Am 16.10.2015 um 15:16 schrieb Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com<mailto:C.Hondros@elsevier.com>>: I love BaseX for the simplicity it brings to XML handling. But this is a problem I have not encountered before. I am creating a DB from about 17,000 small zipfiles, each containing a directory structure and somewhere within each, some XML. BaseX chokes on one of these files giving the error: “invalid entry size (expected 0 but got 11083 bytes)”. So clearly one or more of the zips is invalid - but which one(s)? Is there any way that BaseX can echo to me the file that is causing the error? DEBUG is set to TRUE, but I don’t get any morer verbose output. I am using the GUI, and running a BSX script too. Thanks in advance, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

db:output certainly looked promising, but I get no output from a try/catch clause, alas: for $zip in file:list($hm) return try { db:add("myDB", $hm || '\' || $zip, "", map { 'intparse': true()}) } catch * { db:output($zip) } Any other thoughts, BaseX users? C. From: Tim Thompson [mailto:timathom@gmail.com] Sent: 16 October 2015 16:40 To: Hondros, Constantine (ELS-AMS) Cc: Alexander Holupirek; BaseX Subject: Re: [basex-talk] Which zip did BaseX choke on? I may be wrong, but I think you need to wrap the $err:code in db:output(): see http://docs.basex.org/wiki/XQuery_Update#Returning_Results Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library On Fri, Oct 16, 2015 at 10:28 AM, Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com<mailto:C.Hondros@elsevier.com>> wrote: Hmm, it is surprisingly hard to get some useful logging information out of this. Here for example I fall foul of a XQuery syntax here: “all expressions must be updating or return an empty sequence. ” for $zip in file:list($hm) return try { db:add("myDB", $hm || '\' || $zip, "", map { 'intparse': true() }) } catch * { $err:code } Can anyone help me rephrase this so when BaseX halts on a bad zipfile, I at least know what the name of the zipfile is? C. From: Alexander Holupirek [mailto:alex@holupirek.de<mailto:alex@holupirek.de>] Sent: 16 October 2015 15:23 To: Hondros, Constantine (ELS-AMS) Cc: BaseX Subject: Re: [basex-talk] Which zip did BaseX choke on? Have you tried using try/catch? for zip in zips return try { unzip() } catch ... Am 16.10.2015 um 15:16 schrieb Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com<mailto:C.Hondros@elsevier.com>>: I love BaseX for the simplicity it brings to XML handling. But this is a problem I have not encountered before. I am creating a DB from about 17,000 small zipfiles, each containing a directory structure and somewhere within each, some XML. BaseX chokes on one of these files giving the error: “invalid entry size (expected 0 but got 11083 bytes)”. So clearly one or more of the zips is invalid - but which one(s)? Is there any way that BaseX can echo to me the file that is causing the error? DEBUG is set to TRUE, but I don’t get any morer verbose output. I am using the GUI, and running a BSX script too. Thanks in advance, C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

Hi Christian, I was able to work around this in the following way (as usual, with BaseX, there is a way): (: TEST ZIP FILES FOR VALIDITY:) for $zip in file:list($hm) return try { if (zip:entries($hm || '\' || $zip)/@href ) then () else $zip } catch * { $zip, $err:code, $err:description } It turned out there was indeed something wrong with the zip header of one of the files: experr:ZIP0003 Operation failed: invalid CEN header (bad signature). Kind regards, Constantine -----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: 19 October 2015 13:58 To: Hondros, Constantine (ELS-AMS) Cc: Tim Thompson; BaseX Subject: Re: [basex-talk] Which zip did BaseX choke on? Hi Constantine, I guess that try/catch won't help you here, because the files will only be unzipped in the update phase (after actual query evaluation). I am not sure when the error is generated. Could you send me the full stack trace (triggered via -d)? Thanks, Christian Fri, Oct 16, 2015 at 4:55 PM, Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com> wrote:

db:output certainly looked promising, but I get no output from a try/catch clause, alas:

for $zip in file:list($hm)

return

try {

db:add("myDB", $hm || '\' || $zip, "", map { 'intparse': true()})

} catch * {

db:output($zip)

}

Any other thoughts, BaseX users?

C.

From: Tim Thompson [mailto:timathom@gmail.com] Sent: 16 October 2015 16:40 To: Hondros, Constantine (ELS-AMS) Cc: Alexander Holupirek; BaseX

Subject: Re: [basex-talk] Which zip did BaseX choke on?

I may be wrong, but I think you need to wrap the $err:code in db:output(): see http://docs.basex.org/wiki/XQuery_Update#Returning_Results

Tim

-- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

On Fri, Oct 16, 2015 at 10:28 AM, Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com> wrote:

Hmm, it is surprisingly hard to get some useful logging information out of this.

Here for example I fall foul of a XQuery syntax here: “all expressions must be updating or return an empty sequence. ”

for $zip in file:list($hm)

return try {

db:add("myDB", $hm || '\' || $zip, "", map { 'intparse': true() })

} catch * {

$err:code

}

Can anyone help me rephrase this so when BaseX halts on a bad zipfile, I at least know what the name of the zipfile is?

C.

From: Alexander Holupirek [mailto:alex@holupirek.de] Sent: 16 October 2015 15:23 To: Hondros, Constantine (ELS-AMS) Cc: BaseX Subject: Re: [basex-talk] Which zip did BaseX choke on?

Have you tried using try/catch?

for zip in zips return try { unzip() } catch ...

Am 16.10.2015 um 15:16 schrieb Hondros, Constantine (ELS-AMS) <C.Hondros@elsevier.com>:

I love BaseX for the simplicity it brings to XML handling. But this is a problem I have not encountered before.

I am creating a DB from about 17,000 small zipfiles, each containing a directory structure and somewhere within each, some XML. BaseX chokes on one of these files giving the error: “invalid entry size (expected 0 but got 11083 bytes)”.

So clearly one or more of the zips is invalid - but which one(s)?

Is there any way that BaseX can echo to me the file that is causing the error? DEBUG is set to TRUE, but I don’t get any morer verbose output. I am using the GUI, and running a BSX script too.

Thanks in advance,

C.

________________________________

Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

________________________________

Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

________________________________

Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.