BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

Dagger et al. being replaced by char refs in returned XML
by Chuck Bearden 19 Apr '18

19 Apr '18

Hi all, I have loaded a large XML document (a dictionary of New Testament Greek) into its own database in BaseX 9.0. I'm using OpenJDK 1.8.0_162 under Ubuntu 16.04.4. I used the default Java parser, and I enabled the token & full text indices, but otherwise the database settings were the defaults. When I execute a particular query, I'm getting a sequence of the character reference for the carriage return ('') instead of some characters like the single & double daggers. Here is some typical output:  <entry n="ἄγε|G33"> <note type="occurrencesNT">2</note> <form> <orth>ἄγε</orth>,</form> <seg type="derivation">prop. imperat. of<ref> <foreign xml:lang="grc">ἄγω</foreign> </ref>,</seg> <sense> <gloss>come!</gloss>used as<gramGrp> <pos>adv.</pos> </gramGrp>and addressed, like<ref> <foreign xml:lang="grc">φέρε</foreign> </ref>, to one or more persons:<ref osisRef="Jas.4.13">Ja 4:13</ref>,<ref osisRef="Jas.5.1">5:1</ref> </sense> </entry> The sequence of '' char refs replace a dagger '†' (U+2020). I wonder if I'm doing something wrong, or if I've happened on a bug. Here is my XQuery: declare namespace tei = "http://www.crosswire.org/2013/TEIOSIS/namespace"; declare namespace xsi = "http://www.w3.org/2001/XMLSchema-instance"; let $coll := collection('abbott-smith.tei'), $elems := $coll//tei:seg[parent::tei:entry] return <div xmlns="http://www.crosswire.org/2013/TEIOSIS/namespace" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">{ for $elem in $elems let $entry := $elem/parent::tei:entry, $entry_lemma := $entry/@n/data(), $page_num := $elem/preceding::tei:pb[1]/@n/data(), $comment_text := concat('Page: ', $page_num, ' ; Entry: ', $entry_lemma, ' ') return ( comment {$comment_text}, $entry ) }</div> The source XML document is here: https://github.com/translatable-exegetical-tools/Abbott-Smith/blob/master/a… Thanks in advance for your guidance. All the best, Charles Bearden

3 4

Validation Module: validate:xsd-report( ) improvement
by Yitzhak Khabinsky 19 Apr '18

19 Apr '18

Hello, I am successfully using BaseX Validation Module. Along the following lines: let $xml := 'd:\Temp\CDW\HOME\id4879_BO201801_HomeSubscriberMovementFact.xml' let $xsd := 'd:\Temp\CDW\HOME\HomeSubscriberMovementFact.xsd' return validate:xsd-report($xml, $xsd, '1.1') My XML files have multi-megabyte size and lots of validation errors. In tens or hundreds of thousands of errors. Behind the scenes, Saxon validator 9.8.0.11 is running. Unfortunately, the output structure contains a repeating url attribute. The BaseX output pane cannot present all the errors. It says: "(Chopped) Results". <report> <status>invalid</status> <message level="Error" line="10" column="26" url="file:///D:/Temp/CDW/HOME/id4879_BO201801_HomeSubscriberMovementFact.xml">The content "N/A" of element <CommercialServiceCode> does not match the required simple type. Value "N/A" contravenes the enumeration facet "R60080-X00162, R60080-X00163, ..." of the type Q{http://www.millicom.com}CommercialServiceCodeType</message> <message level="Error" line="19" column="23" url="file:///D:/Temp/CDW/HOME/id4879_BO201801_HomeSubscriberMovementFact.xml">The content "TBD" of element <MovementTechnology> does not match the required simple type. Value "TBD" contravenes the enumeration facet "N/A, HFC, GPON, MMDS, FIBER, C..." of the type Q{http://www.millicom.com}MovementTechnologyType</message> <message level="Error" line="24" column="18" url="file:///D:/Temp/CDW/HOME/id4879_BO201801_HomeSubscriberMovementFact.xml">The content "-1.0000" of element <DownloadSpeed> does not match the required simple type. Value "-1" contravenes the minExclusive facet "0" of the type Q{http://www.millicom.com}DownloadSpeedType</message> <message level="Error" line="26" column="6" url="file:///D:/Temp/CDW/HOME/id4879_BO201801_HomeSubscriberMovementFact.xml">The 7th field in constraint {PK} has no value</message> ... </report> My proposal is to eliminate the repeated url attribute from the each message and elevate it to its own element just once under the root report tag. Along the following output structure: <report> <status>invalid</status> <url>file:///D:/Temp/CDW/HOME/id4879_BO201801_HomeSubscriberMovementFact.xml</url> <message level="Error" line="10" column="26">The content "N/A" of element <CommercialServiceCode> does not match the required simple type. Value "N/A" contravenes the enumeration facet "R60080-X00162, R60080-X00163, ..." of the type Q{http://www.millicom.com}CommercialServiceCodeType</message> <message level="Error" line="19" column="23">The content "TBD" of element <MovementTechnology> does not match the required simple type. Value "TBD" contravenes the enumeration facet "N/A, HFC, GPON, MMDS, FIBER, C..." of the type Q{http://www.millicom.com}MovementTechnologyType</message> <message level="Error" line="24" column="18">The content "-1.0000" of element <DownloadSpeed> does not match the required simple type. Value "-1" contravenes the minExclusive facet "0" of the type Q{http://www.millicom.com}DownloadSpeedType</message> <message level="Error" line="26" column="6">The 7th field in constraint {PK} has no value</message> ... </report> This way the output of the validation is much more readable and hopefully will fit in its entirety to the output pane. Regards, Yitzhak Khabinsky Technical Services Lead Millicom International Services LLC 396 Alhambra Circle, Suite 1100 Coral Gables, FL 33134 Skype4B: +1 (305) 445-4172 Tel: (954) 684-8673 yitzhak.khabinsky(a)millicom.com<mailto:.khabinsky@millicom.com> www.millicom.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.millico…>

3 3

Re: [basex-talk] Validation Module: validate:xsd-report( ) improvement
by Yitzhak Khabinsky 19 Apr '18

19 Apr '18

Hi Marco, The proposed solution is way too slow. let $xml := 'd:\Temp\CDW\HOME\id4879_BO201801_HomeSubscriberMovementFact.xml' let $xsd := 'd:\Temp\CDW\HOME\HomeSubscriberMovementFact.xsd' let $validate := validate:xsd-report($xml, $xsd, '1.1') return file:write("output.xml", copy $newvalidate := $validate modify (delete node $newvalidate//@url) return $newvalidate ) I guess delete node... is too heavy. It runs for 143 seconds. Without it just 11 seconds. The input XML file size is about 40MB. The output.xml file has about the same size. That's why I was proposing to change the default output format of the validation. Regards, Yitzhak Khabinsky Technical Services Lead Millicom International Services LLC 396 Alhambra Circle, Suite 1100 Coral Gables, FL 33134 Skype4B: +1 (305) 445-4172 Tel: (954) 684-8673 yitzhak.khabinsky(a)millicom.com<mailto:.khabinsky@millicom.com> www.millicom.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.millico…> From: Yitzhak Khabinsky Sent: Thursday, April 19, 2018 2:51 PM To: m.lettere(a)gmail.com Subject: Re: [basex-talk] Validation Module: validate:xsd-report( ) improvement Hi Marco, Thanks for the proposed solution. It works. But I was referring to the default behavior. The url attribute is redundant for every message element. Regards, Yitzhak Khabinsky Technical Services Lead Millicom International Services LLC 396 Alhambra Circle, Suite 1100 Coral Gables, FL 33134 Skype4B: +1 (305) 445-4172 Tel: (954) 684-8673 yitzhak.khabinsky(a)millicom.com<mailto:.khabinsky@millicom.com> www.millicom.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.millico…>

1 0

Increase/Decrease font size not possible with MacOS
by Kirsten, Dirk 19 Apr '18

19 Apr '18

Hi Christian, Minor issue, but using the latest snapshot I am unable to increase/decrease the editor font size in MacOS (using Command +/-), whereas it works flawlessly using Windows (with Ctrl +/-). Cheers Dirk Dirk Kirsten ______________________________ Senacor Technologies AG D?sseldorfer Stra?e 13 65760 Eschborn M +49 (151) 4435 1967 Dirk.Kirsten(a)senacor.com www.senacor.com Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Gr?zinger Diese E-Mail inklusive Anlagen enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten, informieren Sie bitte den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail including any attachments may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the materials in this e-mail is strictly forbidden.

2 3

Unicode problem with database
by Giuseppe Celano 18 Apr '18

18 Apr '18

Hi, I tried to create a database containing the attached file and I get some Unicode character converted into "carriage return" characters. Any hint? Thanks! Ciao, Giuseppe Universität Leipzig Institute of Computer Science Augustusplatz 10 04109 Leipzig Deutschland E-mail: celano(a)informatik.uni-leipzig.de E-mail: giuseppegacelano(a)gmail.com Web site 1: http://www.dh.uni-leipzig.de/wo/team/ Web site 2: https://sites.google.com/site/giuseppegacelano/

2 2

RFC, Client for R
by Ben Engbers 18 Apr '18

18 Apr '18

Hi, Last month I have been working on a R client. Results from my work are attached. The 'add, replace and store'-commands should be working but haven't been tested yet since I don't have any good example-commands at hand. I'm looking forward for comments! Ben Engbers

2 1

Collection function
by Giuseppe Celano 18 Apr '18

18 Apr '18

Hi, It seems there is an error with the collection function. Something like this: collection("directory")[5] does not return anything in 9.0 but it does in 8.6.7 Best, Giuseppe Universität Leipzig Institute of Computer Science, Digital Humanities Augustusplatz 10 04109 Leipzig Deutschland E-mail: celano(a)informatik.uni-leipzig.de E-mail: giuseppegacelano(a)gmail.com Web site 1: http://www.dh.uni-leipzig.de/wo/team/ Web site 2: https://sites.google.com/site/giuseppegacelano/

2 1

GUI
by Giuseppe Celano 16 Apr '18

16 Apr '18

I see that in the 9.0 version the "+ button" to add a new tab is missing. I think it was very useful: can it be re-introduced in the following releases? Best, Giuseppe Universität Leipzig Institute of Computer Science Augustusplatz 10 04109 Leipzig Deutschland E-mail: celano(a)informatik.uni-leipzig.de E-mail: giuseppegacelano(a)gmail.com Web site 1: http://www.dh.uni-leipzig.de/wo/team/ Web site 2: https://sites.google.com/site/giuseppegacelano/

2 2

Multipart and charset
by Marco Lettere 16 Apr '18

16 Apr '18

Hi everyone, when parsing a multipart response we are getting from a server we need to integrate with basex raises the exception [1]. By forcing the mediatype (with override-media-type attribute) to text/plain we've been able to see that parts look something like [2]. Our suspect was that the multipart parser does not tolerate the quotes around UTF-8 but exploring the jungles of RFC they seem to be allowed as per an example of https://tools.ietf.org/html/rfc2045 that states: Note that the value of a quoted string parameter does not include the quotes. That is, the quotation marks in a quoted-string are not a part of the value of the parameter, but are merely used to delimit that parameter value. In addition, comments are allowed in accordance with RFC 822 rules for structured header fields. Thus the following two forms Content-type: text/plain; charset=us-ascii (Plain text) Content-type: text/plain; charset="us-ascii" are completely equivalent. Could we kindly have some feedback on this in order to avoid having to parse the whole multipart on our own? Thank you very much. Marco. [1] Unsupported encoding: java.nio.charset.IllegalCharsetNameException: "UTF-8" at org.basex.core.Command.execute(Command.java:94) at org.basex.gui.GUI.exec(GUI.java:427) at org.basex.gui.GUI.lambda$4(GUI.java:370) at org.basex.gui.GUI$$Lambda$80/502071500.run(Unknown Source) at java.lang.Thread.run(Thread.java:745) Caused by: org.basex.query.QueryException: Unsupported encoding: java.nio.charset.IllegalCharsetNameException: "UTF-8" at org.basex.query.QueryError.get(QueryError.java:1392) [...] [2] ----boundary44723329.705882352941176-- Content-Type: application/xop+xml; type="application/soap+xml"; charset="UTF-8" Content-Transfer-Encoding: 8bit Content-Id: <0.1808ACE4.38B8.11E8.8E46.50569966F200>

2 2

Replace command long execution time
by Andrey Fedosenko 14 Apr '18

14 Apr '18

Hello, The size of my database is about 23GB and about 650,000 records. The execution of the REPLACE command can take about 10 minutes. What could be the reason?

2 5

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk