- BaseX-Talk - mailman.uni-konstanz.de

Including the "doc" information in the RETURN clause
by David leigh 02 Feb '11

02 Feb '11

I may be doing this completely wrong to begin with, so try not to be too shocked at my examples! ;c) My BaseX database consists of songs. They were loaded from individual XML files that are stored in a directory structure. I'm searching for songs that contain given lyrics. So far so good. I'm NOT returning XML format data, but tabular data. I'm using the "return" clause with the concat() function of XQuery to formulate each results row. For instance "song title", "author", "lyrics", etc. One row per song that is found from my search. What I'd like to be able to do is to have that results row also contain the information in the "doc" node so that in addition to selected contents of the XML file, the results row would also contain the file name and the relative directory path (which I see are in the "doc" node when I use the BaseX GUI). Here's a real-life example of a query that works well for me to return rows of elements found WITHIN the XML file: for $song in collection("OpenSongSearch") where $song/song/lyrics[ . contains text "baloche"] or $song/song/title[ . contains text "baloche"] or $song/song/author[ . contains text "baloche"] order by $song/song/title return concat($song/song/title,string('|'),$song/song/author,string('|'),replace(re place($song/song/lyrics,'\n','^line^'),'\[\w+\]',''),string('
')) The program that invokes the BaseX call parses these results to put into a listbox in a dialog. The only thing that I knew to try to add the "doc" node was: return concat($song/doc,string('|'), $song/song/title,string('|'),$song/song/author,string('|'),replace(replace($ song/song/lyrics,'\n','^line^'),'\[\w+\]',''),string('
')) This did not give me an error but simply added the additional vertical bar at the beginning and no "doc" node info. I know this is more of a XQuery question than a BaseX question per se, but any hints for novice would be greatly appreciated. Thanks! <mailto:david@leighweb.com> DavidEmailSig

1 0

Force default file type to XML during "add to" command?
by Chad Lancour 02 Feb '11

02 Feb '11

Hello- Just want to start out with KUDOS on the project, contributors, and technology. Amazing. We are importing 300K+ xml & binary files from a file system with 30k in nested hierarchical folder structures into basex. The auto-magical capabilities of basex to suck in this data (and the speed at which is does it) is kick-a%$. Question: All of the 300k+ xml files do NOT have a file extension. Is there a way to have BaseX default to xml if there is no file type? Is there perhaps some other way to handle this other than recursively renaming all the files? Fyi -Here is the simple batch file I am using to import the data: java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "drop db metadata" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "create db metadata; alter user admin xxxxxxx" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open metadata; SET INTPARSE ON; add to \set1 %devbase%\code\basex\import\webcat" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open metadata; SET INTPARSE ON; add to \set2 %devbase%\code\basex\import\rpd" java -Xmx1024m -cp %devbase%\basex65\BaseX.jar org.basex.BaseX -Vc "open metadata; create index text; create index attribute; create index fulltext; create index path" Regards, Chad chad(a)helmetcameracentral.com

3 5

Questions concerning backups
by Fabrice Etanchaud 31 Jan '11

31 Jan '11

Dear all at DBIS, the SHOW BACKUPS command does not seem to work. Is there a way to choose which backup to restore ? best regards, Fabrice Etanchaud Senior Software Engineer CT Corsearch / Edital, a Wolters Kluwer business Berkenlaan 1 1831 Diegem Belgium <skype:fabrice.etanchaud?call> skype:fabrice.etanchaud Get Skype <http://www.skype.com/go/download> and call me for free. fetanchaud(a)edital.com <mailto:fetanchaud@edital.com> <http://www.edital.com/> www.edital.com (website) <http://www.ctcorsearch.com/> www.ctcorsearch.com (website) Confidentiality Notice: This email and its attachments (if any) contain confidential information of the sender. The information is intended only for the use by the direct addressees of the original sender of this email. If you are not an intended recipient of the original sender (or responsible for delivering the message to such person), you are hereby notified that any review, disclosure, copying, distribution or the taking of any action in reliance of the contents of and attachments to this email is strictly prohibited. If you have received this email in error, please immediately notify the sender at the address shown herein and permanently delete any copies of this email (digital or paper) in your possession.

2 1

BaseX 6.5.1 - BaseXGUI - error in Manage/Restore when database name contains hyphens
by Fabrice Etanchaud 31 Jan '11

31 Jan '11

Dear all, It seems there is an error in the BaseXGUI/Manage/Restore menu when restoring a dabase whose name contains hyphens. Changing hyphens to underscores solves the problem. Maybe because hyphen was chosen as information separator in the backup filename (mydbname-2011-01-31-11-54-20.zip). best regards, Fabrice Etanchaud Senior Software Engineer CT Corsearch / Edital, a Wolters Kluwer business Berkenlaan 1 1831 Diegem Belgium <skype:fabrice.etanchaud?call> skype:fabrice.etanchaud Get Skype <http://www.skype.com/go/download> and call me for free. fetanchaud(a)edital.com <mailto:fetanchaud@edital.com> <http://www.edital.com/> www.edital.com (website) <http://www.ctcorsearch.com/> www.ctcorsearch.com (website) Confidentiality Notice: This email and its attachments (if any) contain confidential information of the sender. The information is intended only for the use by the direct addressees of the original sender of this email. If you are not an intended recipient of the original sender (or responsible for delivering the message to such person), you are hereby notified that any review, disclosure, copying, distribution or the taking of any action in reliance of the contents of and attachments to this email is strictly prohibited. If you have received this email in error, please immediately notify the sender at the address shown herein and permanently delete any copies of this email (digital or paper) in your possession.

1 0

Problems using Python API and resolution (RuntimeError: maximum recursion depth exceeded)
by Jan Vlčinský (CAD) 31 Jan '11

31 Jan '11

Hi I was trying to use new Python API, looking forward to use binding variables, but all my calls were failing File "D:\var\projects\BaseXFilter\TomTom\BaseXClient.py", line 96, in receive return self.receive() File "D:\var\projects\BaseXFilter\TomTom\BaseXClient.py", line 96, in receive return self.receive() File "D:\var\projects\BaseXFilter\TomTom\BaseXClient.py", line 96, in receive return self.receive() File "D:\var\projects\BaseXFilter\TomTom\BaseXClient.py", line 96, in receive return self.receive() File "D:\var\projects\BaseXFilter\TomTom\BaseXClient.py", line 95, in receive self.init() RuntimeError: maximum recursion depth exceeded I compared old and new version of BaseXClient.py and noticed, that old calls readString() were changed to receive() And on line 96 is call to self.receive() inside of class method receive() so never ending call is here. Renaming receive call to readString as it used to be resolved the problem. Attached is corrected Python file. If you want to call it as usual, you shall rename it to BaseXClient.py Jan -- *Ing. Jan Vlčinský* CAD programy Slunečnicová 338/3, 734 01 Karviná Ráj, Czech Republic tel: +420-597 602 024; mob: +420-608 979 040 skype: janvlcinsky; GoogleTalk: jan.vlcinsky(a)gmail.com http://cz.linkedin.com/in/vlcinsky

2 1

Re: [basex-talk] DB comuni
by Christian Grün 30 Jan '11

30 Jan '11

Hi Pablo (cc to the mailing list), sorry for the long delay; here's finally my detailed answer. I spent some time in revising the "AnyAllOptions" (any word, all words, phrase, ...), and I noticed that not all combinations delivered the expected scoring results. The issue should now be solved; feel free to tell me if you need a jar build of the latest build; or check out the latest version from the repository. > Hello Christian, thanks por you prompt response. Enclosed I > send you the Comuni file for Italy, that we use to perform the math of > others 32000, with the final purpose of validate its. > […] Thanks for reporting on your usage scenario. One way to speed up your queries might be to create a full-text index (note, however, that the AnyAllOptions are not supported by the index). The improved performance gain after the first call, which you observed, might be due to the caching of results. To get better performance, I'd recommend to switch to the client/server architecture of BaseX. Apart from that, it's difficult to give you general guidelines on how to handle your amounts of data. Some questions might have to be answered first, such as… – do you plan to store the data and perform repeated queries, or will you just process it once/on the fly? – what's the total size of your XML documents? how many documents/XML snippets will you look at? Hope this helps, Christian

1 0

Re: [basex-talk] Parsing entities
by Charles F. Munat 29 Jan '11

29 Jan '11

Actually, no. Your previous email was pure genius and worked beautifully. Thank you very much! My base file for starting up the db now looks like this: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE content [ <!ENTITY % HTMLlat1 SYSTEM "file:///var/www/xxx/content/entities/xhtml-lat1.ent"> %HTMLlat1; <!ENTITY % HTMLsymbol SYSTEM "file:///var/www/xxx/content/entities/xhtml-symbol.ent"> %HTMLsymbol; <!ENTITY % HTMLspecial SYSTEM "file:///var/www/xxx/content/entities/xhtml-special.ent"> %HTMLspecial; ]> <data/> I create the DB thus: session.execute("CREATE DATABASE data /var/www/xxx/data.xml") session.execute("CREATE INDEX TEXT") session.execute("CREATE INDEX FULLTEXT") session.execute("CREATE INDEX ATTRIBUTE") I open it like this: session.execute("SET CHOP off") session.execute("SET INTPARSE on") session.execute("SET PATHINDEX on") session.execute("SET TEXTINDEX on") session.execute("SET ATTRINDEX on") session.execute("SET FTINDEX on") session.execute("SET WILDCARDS on") session.execute("SET DIACRITICS on") Now I find that I can insert the output from TinyMCE like this: insert node <remark id='1'> <name>My Remark</name> <content>  Does  —  this  —  work? </content> </remark> into /data/remarks And it works perfectly! Thanks! Thanks! Thanks! Chas. P.S. For anyone who wants them, here is the content of the three entity files, stripped of all useless junk: xhtml-lat1.ent: <!ENTITY nbsp " "> <!ENTITY iexcl "¡"> <!ENTITY cent "¢"> <!ENTITY pound "£"> <!ENTITY curren "¤"> <!ENTITY yen "¥"> <!ENTITY brvbar "¦"> <!ENTITY sect "§"> <!ENTITY uml "¨"> <!ENTITY copy "©"> <!ENTITY ordf "ª"> <!ENTITY laquo "«"> <!ENTITY not "¬"> <!ENTITY shy ""> <!ENTITY reg "®"> <!ENTITY macr "¯"> <!ENTITY deg "°"> <!ENTITY plusmn "±"> <!ENTITY sup2 "²"> <!ENTITY sup3 "³"> <!ENTITY acute "´"> <!ENTITY micro "µ"> <!ENTITY para "¶"> <!ENTITY middot "·"> <!ENTITY cedil "¸"> <!ENTITY sup1 "¹"> <!ENTITY ordm "º"> <!ENTITY raquo "»"> <!ENTITY frac14 "¼"> <!ENTITY frac12 "½"> <!ENTITY frac34 "¾"> <!ENTITY iquest "¿"> <!ENTITY Agrave "À"> <!ENTITY Aacute "Á"> <!ENTITY Acirc "Â"> <!ENTITY Atilde "Ã"> <!ENTITY Auml "Ä"> <!ENTITY Aring "Å"> <!ENTITY AElig "Æ"> <!ENTITY Ccedil "Ç"> <!ENTITY Egrave "È"> <!ENTITY Eacute "É"> <!ENTITY Ecirc "Ê"> <!ENTITY Euml "Ë"> <!ENTITY Igrave "Ì"> <!ENTITY Iacute "Í"> <!ENTITY Icirc "Î"> <!ENTITY Iuml "Ï"> <!ENTITY ETH "Ð"> <!ENTITY Ntilde "Ñ"> <!ENTITY Ograve "Ò"> <!ENTITY Oacute "Ó"> <!ENTITY Ocirc "Ô"> <!ENTITY Otilde "Õ"> <!ENTITY Ouml "Ö"> <!ENTITY times "×"> <!ENTITY Oslash "Ø"> <!ENTITY Ugrave "Ù"> <!ENTITY Uacute "Ú"> <!ENTITY Ucirc "Û"> <!ENTITY Uuml "Ü"> <!ENTITY Yacute "Ý"> <!ENTITY THORN "Þ"> <!ENTITY szlig "ß"> <!ENTITY agrave "à"> <!ENTITY aacute "á"> <!ENTITY acirc "â"> <!ENTITY atilde "ã"> <!ENTITY auml "ä"> <!ENTITY aring "å"> <!ENTITY aelig "æ"> <!ENTITY ccedil "ç"> <!ENTITY egrave "è"> <!ENTITY eacute "é"> <!ENTITY ecirc "ê"> <!ENTITY euml "ë"> <!ENTITY igrave "ì"> <!ENTITY iacute "í"> <!ENTITY icirc "î"> <!ENTITY iuml "ï"> <!ENTITY eth "ð"> <!ENTITY ntilde "ñ"> <!ENTITY ograve "ò"> <!ENTITY oacute "ó"> <!ENTITY ocirc "ô"> <!ENTITY otilde "õ"> <!ENTITY ouml "ö"> <!ENTITY divide "÷"> <!ENTITY oslash "ø"> <!ENTITY ugrave "ù"> <!ENTITY uacute "ú"> <!ENTITY ucirc "û"> <!ENTITY uuml "ü"> <!ENTITY yacute "ý"> <!ENTITY thorn "þ"> <!ENTITY yuml "ÿ"> xhtml-symbol.ent: <!ENTITY fnof "ƒ"> <!ENTITY Alpha "Α"> <!ENTITY Beta "Β"> <!ENTITY Gamma "Γ"> <!ENTITY Delta "Δ"> <!ENTITY Epsilon "Ε"> <!ENTITY Zeta "Ζ"> <!ENTITY Eta "Η"> <!ENTITY Theta "Θ"> <!ENTITY Iota "Ι"> <!ENTITY Kappa "Κ"> <!ENTITY Lambda "Λ"> <!ENTITY Mu "Μ"> <!ENTITY Nu "Ν"> <!ENTITY Xi "Ξ"> <!ENTITY Omicron "Ο"> <!ENTITY Pi "Π"> <!ENTITY Rho "Ρ"> <!ENTITY Sigma "Σ"> <!ENTITY Tau "Τ"> <!ENTITY Upsilon "Υ"> <!ENTITY Phi "Φ"> <!ENTITY Chi "Χ"> <!ENTITY Psi "Ψ"> <!ENTITY Omega "Ω"> <!ENTITY alpha "α"> <!ENTITY beta "β"> <!ENTITY gamma "γ"> <!ENTITY delta "δ"> <!ENTITY epsilon "ε"> <!ENTITY zeta "ζ"> <!ENTITY eta "η"> <!ENTITY theta "θ"> <!ENTITY iota "ι"> <!ENTITY kappa "κ"> <!ENTITY lambda "λ"> <!ENTITY mu "μ"> <!ENTITY nu "ν"> <!ENTITY xi "ξ"> <!ENTITY omicron "ο"> <!ENTITY pi "π"> <!ENTITY rho "ρ"> <!ENTITY sigmaf "ς"> <!ENTITY sigma "σ"> <!ENTITY tau "τ"> <!ENTITY upsilon "υ"> <!ENTITY phi "φ"> <!ENTITY chi "χ"> <!ENTITY psi "ψ"> <!ENTITY omega "ω"> <!ENTITY thetasym "ϑ"> <!ENTITY upsih "ϒ"> <!ENTITY piv "ϖ"> <!ENTITY bull "•"> <!ENTITY hellip "…"> <!ENTITY prime "′"> <!ENTITY Prime "″"> <!ENTITY oline "‾"> <!ENTITY frasl "⁄"> <!ENTITY weierp "℘"> <!ENTITY image "ℑ"> <!ENTITY real "ℜ"> <!ENTITY trade "™"> <!ENTITY alefsym "ℵ"> <!ENTITY larr "←"> <!ENTITY uarr "↑"> <!ENTITY rarr "→"> <!ENTITY darr "↓"> <!ENTITY harr "↔"> <!ENTITY crarr "↵"> <!ENTITY lArr "⇐"> <!ENTITY uArr "⇑"> <!ENTITY rArr "⇒"> <!ENTITY dArr "⇓"> <!ENTITY hArr "⇔"> <!ENTITY forall "∀"> <!ENTITY part "∂"> <!ENTITY exist "∃"> <!ENTITY empty "∅"> <!ENTITY nabla "∇"> <!ENTITY isin "∈"> <!ENTITY notin "∉"> <!ENTITY ni "∋"> <!ENTITY prod "∏"> <!ENTITY sum "∑"> <!ENTITY minus "−"> <!ENTITY lowast "∗"> <!ENTITY radic "√"> <!ENTITY prop "∝"> <!ENTITY infin "∞"> <!ENTITY ang "∠"> <!ENTITY and "∧"> <!ENTITY or "∨"> <!ENTITY cap "∩"> <!ENTITY cup "∪"> <!ENTITY int "∫"> <!ENTITY there4 "∴"> <!ENTITY sim "∼"> <!ENTITY cong "≅"> <!ENTITY asymp "≈"> <!ENTITY ne "≠"> <!ENTITY equiv "≡"> <!ENTITY le "≤"> <!ENTITY ge "≥"> <!ENTITY sub "⊂"> <!ENTITY sup "⊃"> <!ENTITY nsub "⊄"> <!ENTITY sube "⊆"> <!ENTITY supe "⊇"> <!ENTITY oplus "⊕"> <!ENTITY otimes "⊗"> <!ENTITY perp "⊥"> <!ENTITY sdot "⋅"> <!ENTITY lceil "⌈"> <!ENTITY rceil "⌉"> <!ENTITY lfloor "⌊"> <!ENTITY rfloor "⌋"> <!ENTITY lang "〈"> <!ENTITY rang "〉"> <!ENTITY loz "◊"> <!ENTITY spades "♠"> <!ENTITY clubs "♣"> <!ENTITY hearts "♥"> <!ENTITY diams "♦"> xhtml-special.ent: <!ENTITY quot """> <!ENTITY amp "&#38;"> <!ENTITY lt "&#60;"> <!ENTITY gt ">"> <!ENTITY apos "'"> <!ENTITY OElig "Œ"> <!ENTITY oelig "œ"> <!ENTITY Scaron "Š"> <!ENTITY scaron "š"> <!ENTITY Yuml "Ÿ"> <!ENTITY circ "ˆ"> <!ENTITY tilde "˜"> <!ENTITY ensp " "> <!ENTITY emsp " "> <!ENTITY thinsp " "> <!ENTITY zwnj "‌"> <!ENTITY zwj "‍"> <!ENTITY lrm "‎"> <!ENTITY rlm "‏"> <!ENTITY ndash "–"> <!ENTITY mdash "—"> <!ENTITY lsquo "‘"> <!ENTITY rsquo "’"> <!ENTITY sbquo "‚"> <!ENTITY ldquo "“"> <!ENTITY rdquo "”"> <!ENTITY bdquo "„"> <!ENTITY dagger "†"> <!ENTITY Dagger "‡"> <!ENTITY permil "‰"> <!ENTITY lsaquo "‹"> <!ENTITY rsaquo "›"> <!ENTITY euro "€"> On 01/28/2011 6:35 PM, Imsieke, Gerrit, le-tex wrote: > While I was writing my lengthy reply to Patrick and your original post, > I simply forgot about the TinyMCE issue. But anyway, this internal > subset stuff is interesting for the young people on this list who grew > up in unprecedented wealth, but without DOCTYPEs and catalogs. Given the > new site search facility, this knowledge will be discoverable in > eternity. Amen. > > Patching TinyMCE's output with a DOCTYPE declaration with an internal > subset might be a challenge, too. > > There must be a configuration option that will tell TinyMCE to serialize > numerical entities. > > It turns out that they actually put some effort in creating the named > entities: see the list on [1]. > > Indeed, there is such an option: [2] > So you'll have to find out where tinyMCE.init() is called (if nowhere > then maybe just add this call in a script tag on the page that invokes > TinyMCE? Don't know exactly.) and add the appropriate entity_encoding > name/value pair. > > -Gerrit > > [1] http://drupal.org/node/373542 > [2] http://tinymce.moxiecode.com/wiki.php/Configuration:entity_encoding > > > On 28.01.2011 22:17, Charles F. Munat wrote: >> In response to the question below, yes, I could do a replace on the >> string and exchange   for  , but then what would I do about >> the hundred-plus *other* character entity references? Like ” or >> —? TinyMCE uses the HTML entity references and there are lots of >> them. I can't rewrite TinyMCE, and searching and replacing for all of >> these is going to be a bear. >> >> There must be some way to get the parser to accept the HTML entities. >> >> I intend to create a DTD for my data anyway, and I can simply include >> the HTML entities in that. How do I set the database to test against a >> DTD (as the default namespace)? >> >> Thanks! >> >> Chas. >> >> >> >> Charles: >> Can you replace   with   ? >> *P >> >> On 1/28/11 9:07 AM, Charles F. Munat wrote: >>> I'm having trouble using XHTML generated by TinyMCE in a BaseX database. >>> >>> I get the error: The entity "nbsp" was referenced, but not declared. >>> >>> OK, I understand that it doesn't parse entities automatically. I need >>> to add a DTD. >>> >>> So I have a setup like this: >>> >>> In a file called catalog.xml: >>> >>> <?xml version="1.0"?> >>> <catalog prefer="system" >>> xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> >>> <rewriteSystem >>> systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/" >>> rewritePrefix="file:///var/www/xxx/content/dtds/" /> >>> </catalog> >>> >>> In /var/www/xxx/content/dtds/ I have a file called xhtml1-strict.dtd >>> which is exactly what it says it is (downloaded from the W3C). >>> >>> Now I need to add the DOCTYPE to my database. But how? >>> >>> I create this database via a web interface using Vaadin (a bunch of >>> forms, tables, etc. Then I just do XQuery inserts, deletes, etc. There >>> are no XML documents. It's all done piecemeal. >>> >>> My root node is <data>. Some of my nodes have a <content> node that >>> contains XHTML: >>> >>> <data> >>> <remarks> >>> <remark id='1'> >>> <name>Test Remark</name> >>> <content> >>> This contains a   non-breaking space! >>> </content> >>> </remark> >>> </remarks> >>> </data> >>> >>> How do I add the DTD? How do I indicate that it should be applied to >>> the contents of the <content> node only? How can I do that without >>> adding namespaces to all the XHTML tags? >>> >>> Can anyone provide a brief example? >>> >>> By the way, when I connect to the database I do this: >>> >>> val session = new ClientSession("localhost", 1984, "admin", "x") >>> >>> session.setOutputStream(System.out) >>> >>> session.execute("SET CATFILE /var/www/xxx/catalog.xml") >>> session.execute("SET CHOP off") >>> session.execute("SET INTPARSE on") >>> session.execute("SET ENTITY on") >>> session.execute("SET PATHINDEX on") >>> session.execute("SET TEXTINDEX on") >>> session.execute("SET ATTRINDEX on") >>> session.execute("SET FTINDEX on") >>> session.execute("SET WILDCARDS on") >>> session.execute("SET DIACRITICS on") >>> session.execute("INFO") >>> >>> Thanks! >>> >>> Chas. >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk at mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

2 2

Fwd: Re: Parsing entities
by Imsieke, Gerrit, le-tex 28 Jan '11

28 Jan '11

Oops, just noticed that replies don't go to the list by default. -------- Original Message -------- Subject: Re: [basex-talk] Parsing entities Date: Fri, 28 Jan 2011 22:35:24 +0100 From: Imsieke, Gerrit, le-tex <gerrit.imsieke(a)le-tex.de> Organization: le-tex publishing services GmbH To: chas(a)munat.com While I was writing my lengthy reply to Patrick and your original post, I simply forgot about the TinyMCE issue. But anyway, this internal subset stuff is interesting for the young people on this list who grew up in unprecedented wealth, but without DOCTYPEs and catalogs. Given the new site search facility, this knowledge will be discoverable in eternity. Amen. Patching TinyMCE's output with a DOCTYPE declaration with an internal subset might be a challenge, too. There must be a configuration option that will tell TinyMCE to serialize numerical entities. It turns out that they actually put some effort in creating the named entities: see the list on [1]. Indeed, there is such an option: [2] So you'll have to find out where tinyMCE.init() is called (if nowhere then maybe just add this call in a script tag on the page that invokes TinyMCE? Don't know exactly.) and add the appropriate entity_encoding name/value pair. -Gerrit [1] http://drupal.org/node/373542 [2] http://tinymce.moxiecode.com/wiki.php/Configuration:entity_encoding On 28.01.2011 22:17, Charles F. Munat wrote: > In response to the question below, yes, I could do a replace on the > string and exchange   for  , but then what would I do about > the hundred-plus *other* character entity references? Like ” or > —? TinyMCE uses the HTML entity references and there are lots of > them. I can't rewrite TinyMCE, and searching and replacing for all of > these is going to be a bear. > > There must be some way to get the parser to accept the HTML entities. > > I intend to create a DTD for my data anyway, and I can simply include > the HTML entities in that. How do I set the database to test against a > DTD (as the default namespace)? > > Thanks! > > Chas. > > > > Charles: > Can you replace   with   ? > *P > > On 1/28/11 9:07 AM, Charles F. Munat wrote: >> I'm having trouble using XHTML generated by TinyMCE in a BaseX database. >> >> I get the error: The entity "nbsp" was referenced, but not declared. >> >> OK, I understand that it doesn't parse entities automatically. I need >> to add a DTD. >> >> So I have a setup like this: >> >> In a file called catalog.xml: >> >> <?xml version="1.0"?> >> <catalog prefer="system" >> xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> >> <rewriteSystem >> systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/" >> rewritePrefix="file:///var/www/xxx/content/dtds/" /> >> </catalog> >> >> In /var/www/xxx/content/dtds/ I have a file called xhtml1-strict.dtd >> which is exactly what it says it is (downloaded from the W3C). >> >> Now I need to add the DOCTYPE to my database. But how? >> >> I create this database via a web interface using Vaadin (a bunch of >> forms, tables, etc. Then I just do XQuery inserts, deletes, etc. There >> are no XML documents. It's all done piecemeal. >> >> My root node is <data>. Some of my nodes have a <content> node that >> contains XHTML: >> >> <data> >> <remarks> >> <remark id='1'> >> <name>Test Remark</name> >> <content> >> This contains a   non-breaking space! >> </content> >> </remark> >> </remarks> >> </data> >> >> How do I add the DTD? How do I indicate that it should be applied to >> the contents of the <content> node only? How can I do that without >> adding namespaces to all the XHTML tags? >> >> Can anyone provide a brief example? >> >> By the way, when I connect to the database I do this: >> >> val session = new ClientSession("localhost", 1984, "admin", "x") >> >> session.setOutputStream(System.out) >> >> session.execute("SET CATFILE /var/www/xxx/catalog.xml") >> session.execute("SET CHOP off") >> session.execute("SET INTPARSE on") >> session.execute("SET ENTITY on") >> session.execute("SET PATHINDEX on") >> session.execute("SET TEXTINDEX on") >> session.execute("SET ATTRINDEX on") >> session.execute("SET FTINDEX on") >> session.execute("SET WILDCARDS on") >> session.execute("SET DIACRITICS on") >> session.execute("INFO") >> >> Thanks! >> >> Chas. >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk -- Gerrit Imsieke Geschäftsführer / Managing Director le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 110, Fax +49 341 355356 510 gerrit.imsieke(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard Vöckler

1 0

Re: [basex-talk] Parsing entities
by Charles F. Munat 28 Jan '11

28 Jan '11

In response to the question below, yes, I could do a replace on the string and exchange   for  , but then what would I do about the hundred-plus *other* character entity references? Like ” or —? TinyMCE uses the HTML entity references and there are lots of them. I can't rewrite TinyMCE, and searching and replacing for all of these is going to be a bear. There must be some way to get the parser to accept the HTML entities. I intend to create a DTD for my data anyway, and I can simply include the HTML entities in that. How do I set the database to test against a DTD (as the default namespace)? Thanks! Chas. Charles: Can you replace   with   ? *P On 1/28/11 9:07 AM, Charles F. Munat wrote: > I'm having trouble using XHTML generated by TinyMCE in a BaseX database. > > I get the error: The entity "nbsp" was referenced, but not declared. > > OK, I understand that it doesn't parse entities automatically. I need > to add a DTD. > > So I have a setup like this: > > In a file called catalog.xml: > > <?xml version="1.0"?> > <catalog prefer="system" > xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> > <rewriteSystem > systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/" > rewritePrefix="file:///var/www/xxx/content/dtds/" /> > </catalog> > > In /var/www/xxx/content/dtds/ I have a file called xhtml1-strict.dtd > which is exactly what it says it is (downloaded from the W3C). > > Now I need to add the DOCTYPE to my database. But how? > > I create this database via a web interface using Vaadin (a bunch of > forms, tables, etc. Then I just do XQuery inserts, deletes, etc. There > are no XML documents. It's all done piecemeal. > > My root node is <data>. Some of my nodes have a <content> node that > contains XHTML: > > <data> > <remarks> > <remark id='1'> > <name>Test Remark</name> > <content> > This contains a   non-breaking space! > </content> > </remark> > </remarks> > </data> > > How do I add the DTD? How do I indicate that it should be applied to > the contents of the <content> node only? How can I do that without > adding namespaces to all the XHTML tags? > > Can anyone provide a brief example? > > By the way, when I connect to the database I do this: > > val session = new ClientSession("localhost", 1984, "admin", "x") > > session.setOutputStream(System.out) > > session.execute("SET CATFILE /var/www/xxx/catalog.xml") > session.execute("SET CHOP off") > session.execute("SET INTPARSE on") > session.execute("SET ENTITY on") > session.execute("SET PATHINDEX on") > session.execute("SET TEXTINDEX on") > session.execute("SET ATTRINDEX on") > session.execute("SET FTINDEX on") > session.execute("SET WILDCARDS on") > session.execute("SET DIACRITICS on") > session.execute("INFO") > > Thanks! > > Chas. > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

1 0

Parsing entities
by Charles F. Munat 28 Jan '11

28 Jan '11

I'm having trouble using XHTML generated by TinyMCE in a BaseX database. I get the error: The entity "nbsp" was referenced, but not declared. OK, I understand that it doesn't parse entities automatically. I need to add a DTD. So I have a setup like this: In a file called catalog.xml: <?xml version="1.0"?> <catalog prefer="system" xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> <rewriteSystem systemIdStartString="http://www.w3.org/TR/xhtml1/DTD/" rewritePrefix="file:///var/www/xxx/content/dtds/" /> </catalog> In /var/www/xxx/content/dtds/ I have a file called xhtml1-strict.dtd which is exactly what it says it is (downloaded from the W3C). Now I need to add the DOCTYPE to my database. But how? I create this database via a web interface using Vaadin (a bunch of forms, tables, etc. Then I just do XQuery inserts, deletes, etc. There are no XML documents. It's all done piecemeal. My root node is <data>. Some of my nodes have a <content> node that contains XHTML: <data> <remarks> <remark id='1'> <name>Test Remark</name> <content> This contains a   non-breaking space! </content> </remark> </remarks> </data> How do I add the DTD? How do I indicate that it should be applied to the contents of the <content> node only? How can I do that without adding namespaces to all the XHTML tags? Can anyone provide a brief example? By the way, when I connect to the database I do this: val session = new ClientSession("localhost", 1984, "admin", "x") session.setOutputStream(System.out) session.execute("SET CATFILE /var/www/xxx/catalog.xml") session.execute("SET CHOP off") session.execute("SET INTPARSE on") session.execute("SET ENTITY on") session.execute("SET PATHINDEX on") session.execute("SET TEXTINDEX on") session.execute("SET ATTRINDEX on") session.execute("SET FTINDEX on") session.execute("SET WILDCARDS on") session.execute("SET DIACRITICS on") session.execute("INFO") Thanks! Chas.

3 2