Re: [basex-talk] Parsing entities

28 Jan 2011


      Actually, no. Your previous email was pure genius and worked 
beautifully. Thank you very much!
My base file for starting up the db now looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE content [
  <!ENTITY % HTMLlat1 SYSTEM 
"file:///var/www/xxx/content/entities/xhtml-lat1.ent"> %HTMLlat1;
  <!ENTITY % HTMLsymbol SYSTEM 
"file:///var/www/xxx/content/entities/xhtml-symbol.ent"> %HTMLsymbol;
  <!ENTITY % HTMLspecial SYSTEM 
"file:///var/www/xxx/content/entities/xhtml-special.ent"> %HTMLspecial;
]>
<data/>
I create the DB thus:
session.execute("CREATE DATABASE data /var/www/xxx/data.xml")
session.execute("CREATE INDEX TEXT")
session.execute("CREATE INDEX FULLTEXT")
session.execute("CREATE INDEX ATTRIBUTE")
I open it like this:
session.execute("SET CHOP off")
session.execute("SET INTPARSE on")
session.execute("SET PATHINDEX on")
session.execute("SET TEXTINDEX on")
session.execute("SET ATTRINDEX on")
session.execute("SET FTINDEX on")
session.execute("SET WILDCARDS on")
session.execute("SET DIACRITICS on")
Now I find that I can insert the output from TinyMCE like this:
insert node <remark id='1'>
   <name>My Remark</name>
   <content>
     <!-- This is from TinyMCE -->
     <p>Does &nbsp;&mdash;&nbsp; this &nbsp;&mdash;&nbsp; work?</p>
   </content>
</remark> into /data/remarks
And it works perfectly! Thanks! Thanks! Thanks!
Chas.
P.S. For anyone who wants them, here is the content of the three entity 
files, stripped of all useless junk:
xhtml-lat1.ent:
<!ENTITY nbsp   "&#160;">
<!ENTITY iexcl  "&#161;">
<!ENTITY cent   "&#162;">
<!ENTITY pound  "&#163;">
<!ENTITY curren "&#164;">
<!ENTITY yen    "&#165;">
<!ENTITY brvbar "&#166;">
<!ENTITY sect   "&#167;">
<!ENTITY uml    "&#168;">
<!ENTITY copy   "&#169;">
<!ENTITY ordf   "&#170;">
<!ENTITY laquo  "&#171;">
<!ENTITY not    "&#172;">
<!ENTITY shy    "&#173;">
<!ENTITY reg    "&#174;">
<!ENTITY macr   "&#175;">
<!ENTITY deg    "&#176;">
<!ENTITY plusmn "&#177;">
<!ENTITY sup2   "&#178;">
<!ENTITY sup3   "&#179;">
<!ENTITY acute  "&#180;">
<!ENTITY micro  "&#181;">
<!ENTITY para   "&#182;">
<!ENTITY middot "&#183;">
<!ENTITY cedil  "&#184;">
<!ENTITY sup1   "&#185;">
<!ENTITY ordm   "&#186;">
<!ENTITY raquo  "&#187;">
<!ENTITY frac14 "&#188;">
<!ENTITY frac12 "&#189;">
<!ENTITY frac34 "&#190;">
<!ENTITY iquest "&#191;">
<!ENTITY Agrave "&#192;">
<!ENTITY Aacute "&#193;">
<!ENTITY Acirc  "&#194;">
<!ENTITY Atilde "&#195;">
<!ENTITY Auml   "&#196;">
<!ENTITY Aring  "&#197;">
<!ENTITY AElig  "&#198;">
<!ENTITY Ccedil "&#199;">
<!ENTITY Egrave "&#200;">
<!ENTITY Eacute "&#201;">
<!ENTITY Ecirc  "&#202;">
<!ENTITY Euml   "&#203;">
<!ENTITY Igrave "&#204;">
<!ENTITY Iacute "&#205;">
<!ENTITY Icirc  "&#206;">
<!ENTITY Iuml   "&#207;">
<!ENTITY ETH    "&#208;">
<!ENTITY Ntilde "&#209;">
<!ENTITY Ograve "&#210;">
<!ENTITY Oacute "&#211;">
<!ENTITY Ocirc  "&#212;">
<!ENTITY Otilde "&#213;">
<!ENTITY Ouml   "&#214;">
<!ENTITY times  "&#215;">
<!ENTITY Oslash "&#216;">
<!ENTITY Ugrave "&#217;">
<!ENTITY Uacute "&#218;">
<!ENTITY Ucirc  "&#219;">
<!ENTITY Uuml   "&#220;">
<!ENTITY Yacute "&#221;">
<!ENTITY THORN  "&#222;">
<!ENTITY szlig  "&#223;">
<!ENTITY agrave "&#224;">
<!ENTITY aacute "&#225;">
<!ENTITY acirc  "&#226;">
<!ENTITY atilde "&#227;">
<!ENTITY auml   "&#228;">
<!ENTITY aring  "&#229;">
<!ENTITY aelig  "&#230;">
<!ENTITY ccedil "&#231;">
<!ENTITY egrave "&#232;">
<!ENTITY eacute "&#233;">
<!ENTITY ecirc  "&#234;">
<!ENTITY euml   "&#235;">
<!ENTITY igrave "&#236;">
<!ENTITY iacute "&#237;">
<!ENTITY icirc  "&#238;">
<!ENTITY iuml   "&#239;">
<!ENTITY eth    "&#240;">
<!ENTITY ntilde "&#241;">
<!ENTITY ograve "&#242;">
<!ENTITY oacute "&#243;">
<!ENTITY ocirc  "&#244;">
<!ENTITY otilde "&#245;">
<!ENTITY ouml   "&#246;">
<!ENTITY divide "&#247;">
<!ENTITY oslash "&#248;">
<!ENTITY ugrave "&#249;">
<!ENTITY uacute "&#250;">
<!ENTITY ucirc  "&#251;">
<!ENTITY uuml   "&#252;">
<!ENTITY yacute "&#253;">
<!ENTITY thorn  "&#254;">
<!ENTITY yuml   "&#255;">
xhtml-symbol.ent:
<!ENTITY fnof     "&#402;">
<!ENTITY Alpha    "&#913;">
<!ENTITY Beta     "&#914;">
<!ENTITY Gamma    "&#915;">
<!ENTITY Delta    "&#916;">
<!ENTITY Epsilon  "&#917;">
<!ENTITY Zeta     "&#918;">
<!ENTITY Eta      "&#919;">
<!ENTITY Theta    "&#920;">
<!ENTITY Iota     "&#921;">
<!ENTITY Kappa    "&#922;">
<!ENTITY Lambda   "&#923;">
<!ENTITY Mu       "&#924;">
<!ENTITY Nu       "&#925;">
<!ENTITY Xi       "&#926;">
<!ENTITY Omicron  "&#927;">
<!ENTITY Pi       "&#928;">
<!ENTITY Rho      "&#929;">
<!ENTITY Sigma    "&#931;">
<!ENTITY Tau      "&#932;">
<!ENTITY Upsilon  "&#933;">
<!ENTITY Phi      "&#934;">
<!ENTITY Chi      "&#935;">
<!ENTITY Psi      "&#936;">
<!ENTITY Omega    "&#937;">
<!ENTITY alpha    "&#945;">
<!ENTITY beta     "&#946;">
<!ENTITY gamma    "&#947;">
<!ENTITY delta    "&#948;">
<!ENTITY epsilon  "&#949;">
<!ENTITY zeta     "&#950;">
<!ENTITY eta      "&#951;">
<!ENTITY theta    "&#952;">
<!ENTITY iota     "&#953;">
<!ENTITY kappa    "&#954;">
<!ENTITY lambda   "&#955;">
<!ENTITY mu       "&#956;">
<!ENTITY nu       "&#957;">
<!ENTITY xi       "&#958;">
<!ENTITY omicron  "&#959;">
<!ENTITY pi       "&#960;">
<!ENTITY rho      "&#961;">
<!ENTITY sigmaf   "&#962;">
<!ENTITY sigma    "&#963;">
<!ENTITY tau      "&#964;">
<!ENTITY upsilon  "&#965;">
<!ENTITY phi      "&#966;">
<!ENTITY chi      "&#967;">
<!ENTITY psi      "&#968;">
<!ENTITY omega    "&#969;">
<!ENTITY thetasym "&#977;">
<!ENTITY upsih    "&#978;">
<!ENTITY piv      "&#982;">
<!ENTITY bull     "&#8226;">
<!ENTITY hellip   "&#8230;">
<!ENTITY prime    "&#8242;">
<!ENTITY Prime    "&#8243;">
<!ENTITY oline    "&#8254;">
<!ENTITY frasl    "&#8260;">
<!ENTITY weierp   "&#8472;">
<!ENTITY image    "&#8465;">
<!ENTITY real     "&#8476;">
<!ENTITY trade    "&#8482;">
<!ENTITY alefsym  "&#8501;">
<!ENTITY larr     "&#8592;">
<!ENTITY uarr     "&#8593;">
<!ENTITY rarr     "&#8594;">
<!ENTITY darr     "&#8595;">
<!ENTITY harr     "&#8596;">
<!ENTITY crarr    "&#8629;">
<!ENTITY lArr     "&#8656;">
<!ENTITY uArr     "&#8657;">
<!ENTITY rArr     "&#8658;">
<!ENTITY dArr     "&#8659;">
<!ENTITY hArr     "&#8660;">
<!ENTITY forall   "&#8704;">
<!ENTITY part     "&#8706;">
<!ENTITY exist    "&#8707;">
<!ENTITY empty    "&#8709;">
<!ENTITY nabla    "&#8711;">
<!ENTITY isin     "&#8712;">
<!ENTITY notin    "&#8713;">
<!ENTITY ni       "&#8715;">
<!ENTITY prod     "&#8719;">
<!ENTITY sum      "&#8721;">
<!ENTITY minus    "&#8722;">
<!ENTITY lowast   "&#8727;">
<!ENTITY radic    "&#8730;">
<!ENTITY prop     "&#8733;">
<!ENTITY infin    "&#8734;">
<!ENTITY ang      "&#8736;">
<!ENTITY and      "&#8743;">
<!ENTITY or       "&#8744;">
<!ENTITY cap      "&#8745;">
<!ENTITY cup      "&#8746;">
<!ENTITY int      "&#8747;">
<!ENTITY there4   "&#8756;">
<!ENTITY sim      "&#8764;">
<!ENTITY cong     "&#8773;">
<!ENTITY asymp    "&#8776;">
<!ENTITY ne       "&#8800;">
<!ENTITY equiv    "&#8801;">
<!ENTITY le       "&#8804;">
<!ENTITY ge       "&#8805;">
<!ENTITY sub      "&#8834;">
<!ENTITY sup      "&#8835;">
<!ENTITY nsub     "&#8836;">
<!ENTITY sube     "&#8838;">
<!ENTITY supe     "&#8839;">
<!ENTITY oplus    "&#8853;">
<!ENTITY otimes   "&#8855;">
<!ENTITY perp     "&#8869;">
<!ENTITY sdot     "&#8901;">
<!ENTITY lceil    "&#8968;">
<!ENTITY rceil    "&#8969;">
<!ENTITY lfloor   "&#8970;">
<!ENTITY rfloor   "&#8971;">
<!ENTITY lang     "&#9001;">
<!ENTITY rang     "&#9002;">
<!ENTITY loz      "&#9674;">
<!ENTITY spades   "&#9824;">
<!ENTITY clubs    "&#9827;">
<!ENTITY hearts   "&#9829;">
<!ENTITY diams    "&#9830;">
xhtml-special.ent:
<!ENTITY quot     "&#34;">
<!ENTITY amp      "&#38;#38;">
<!ENTITY lt       "&#38;#60;">
<!ENTITY gt       "&#62;">
<!ENTITY apos	    "&#39;">
<!ENTITY OElig    "&#338;">
<!ENTITY oelig    "&#339;">
<!ENTITY Scaron   "&#352;">
<!ENTITY scaron   "&#353;">
<!ENTITY Yuml     "&#376;">
<!ENTITY circ     "&#710;">
<!ENTITY tilde    "&#732;">
<!ENTITY ensp     "&#8194;">
<!ENTITY emsp     "&#8195;">
<!ENTITY thinsp   "&#8201;">
<!ENTITY zwnj     "&#8204;">
<!ENTITY zwj      "&#8205;">
<!ENTITY lrm      "&#8206;">
<!ENTITY rlm      "&#8207;">
<!ENTITY ndash    "&#8211;">
<!ENTITY mdash    "&#8212;">
<!ENTITY lsquo    "&#8216;">
<!ENTITY rsquo    "&#8217;">
<!ENTITY sbquo    "&#8218;">
<!ENTITY ldquo    "&#8220;">
<!ENTITY rdquo    "&#8221;">
<!ENTITY bdquo    "&#8222;">
<!ENTITY dagger   "&#8224;">
<!ENTITY Dagger   "&#8225;">
<!ENTITY permil   "&#8240;">
<!ENTITY lsaquo   "&#8249;">
<!ENTITY rsaquo   "&#8250;">
<!ENTITY euro     "&#8364;">
On 01/28/2011 6:35 PM, Imsieke, Gerrit, le-tex wrote:
...
While I was writing my lengthy reply to Patrick and your original post,
I simply forgot about the TinyMCE issue. But anyway, this internal
subset stuff is interesting for the young people on this list who grew
up in unprecedented wealth, but without DOCTYPEs and catalogs. Given the
new site search facility, this knowledge will be discoverable in
eternity. Amen.
Patching TinyMCE's output with a DOCTYPE declaration with an internal
subset might be a challenge, too.
There must be a configuration option that will tell TinyMCE to serialize
numerical entities.
It turns out that they actually put some effort in creating the named
entities: see the list on [1].
Indeed, there is such an option: [2]
So you'll have to find out where tinyMCE.init() is called (if nowhere
then maybe just add this call in a script tag on the page that invokes
TinyMCE? Don't know exactly.) and add the appropriate entity_encoding
name/value pair.
-Gerrit
[1] http://drupal.org/node/373542
[2] http://tinymce.moxiecode.com/wiki.php/Configuration:entity_encoding
On 28.01.2011 22:17, Charles F. Munat wrote:
...
In response to the question below, yes, I could do a replace on the
string and exchange &#160; for &nbsp;, but then what would I do about
the hundred-plus *other* character entity references? Like &rdquo; or
&mdash;? TinyMCE uses the HTML entity references and there are lots of
them. I can't rewrite TinyMCE, and searching and replacing for all of
these is going to be a bear.
There must be some way to get the parser to accept the HTML entities.
I intend to create a DTD for my data anyway, and I can simply include
the HTML entities in that. How do I set the database to test against a
DTD (as the default namespace)?
Thanks!
Chas.
Charles:
Can you replace &nbsp; with &#160; ?
*P
On 1/28/11 9:07 AM, Charles F. Munat wrote:
...
I'm having trouble using XHTML generated by TinyMCE in a BaseX
database.
...
...
...
I get the error: The entity "nbsp" was referenced, but not declared.
OK, I understand that it doesn't parse entities automatically. I need
to add a DTD.
So I have a setup like this:
In a file called catalog.xml:
<?xml version="1.0"?>
<catalog prefer="system"
xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
<rewriteSystem
systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/"
rewritePrefix="file:///var/www/xxx/content/dtds/" />
</catalog>
In /var/www/xxx/content/dtds/ I have a file called xhtml1-strict.dtd
which is exactly what it says it is (downloaded from the W3C).
Now I need to add the DOCTYPE to my database. But how?
I create this database via a web interface using Vaadin (a bunch of
forms, tables, etc. Then I just do XQuery inserts, deletes, etc. There
are no XML documents. It's all done piecemeal.
My root node is <data>. Some of my nodes have a <content> node that
contains XHTML:
<data>
<remarks>
<remark id='1'>
<name>Test Remark</name>
<content>
<p>This contains a &nbsp; non-breaking space!</p>
</content>
</remark>
</remarks>
</data>
How do I add the DTD? How do I indicate that it should be applied to
the contents of the <content> node only? How can I do that without
adding namespaces to all the XHTML tags?
Can anyone provide a brief example?
By the way, when I connect to the database I do this:
val session = new ClientSession("localhost", 1984, "admin", "x")
session.setOutputStream(System.out)
session.execute("SET CATFILE /var/www/xxx/catalog.xml")
session.execute("SET CHOP off")
session.execute("SET INTPARSE on")
session.execute("SET ENTITY on")
session.execute("SET PATHINDEX on")
session.execute("SET TEXTINDEX on")
session.execute("SET ATTRINDEX on")
session.execute("SET FTINDEX on")
session.execute("SET WILDCARDS on")
session.execute("SET DIACRITICS on")
session.execute("INFO")
Thanks!
Chas.
_______________________________________________
BaseX-Talk mailing list
BaseX-Talk at mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Re: [basex-talk] Parsing entities