- BaseX-Talk - mailman.uni-konstanz.de

Spatial XQuery
by David Stuebe 21 Dec '12

21 Dec '12

Hi Basex-ers I am new to the list but have been working with basex for a few weeks now. We are using basex to search ISO 19115 geospatial metadata. The full text search is simple and works better than any of the existing purpose built catalog tools we have worked with so far. I am very interested in the pervious thread on Spatial Xquery. https://mailman.uni-konstanz.de/pipermail/basex-talk/2012-January/002484.ht… I am looking over the Geo Module geospatial API<http://expath.org/spec/geo> today, and also at Zorba's Geo Module<http://www.zorba-xquery.com/html/modules/expath/geo> implementation. Has the Basex community made an further progress? The API is the first issue and so far the Geo Module definition looks great. If linear search using the existing attribute indexes is all that is available now, that is fine. It would be fantastic to write a geo spatial index to support that API in basex in the future. What type of indexes have you considered? I would think an Rtree would probably be best for supporting that API. This email is essentially a status query – as I work on this over the next week or two we can start new threads for specific topics. David Stuebe Scientist & Software Engineer – RPS ASA 55 Village Square Drive South Kingstown, RI 02879-8248 Tel: +1 (401) 789-6224 Email: David.Stuebe(a)rpsgroup.com<mailto:David.Stuebe@rpsgroup.com> www: asascience.com<http://www.asascience.com/> | rpsgroup.com<http://www.rpsgroup.com/> A member of the RPS Group plc

3 2

Restxq updating functions running indefinitely.
by France Baril 20 Dec '12

20 Dec '12

Hi, I just upgraded to 7.5 from 7.4. Everything seems to work except the Updating functions db:replace and db:store (I use db:add elsewhere and it works as it did before). There is very little information regarding what the problem could be: - I am starting the httpserver with these .basex settings: USER = admin and PASS = admin. So I do not expect the problem to be security. - Symtom: the query runs and never times out. Browser stays on empty page. I have to stop the server with the httpstop script. - Error messages in the terminal window: None. - When I start the server I get this message before the server starts: Null identity service, trying login service: null Finding identity service: null I tried two different 7.5 snapshots including today's RC1. Any clue is welcome. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 2

DTD validation
by France Baril 18 Dec '12

18 Dec '12

Hi, We are having challenges with validation. We have built a Documentation Controller for authors that create XML content. On a regular basis, we ask them to check that all documents are valid. We are running the process on hundreds of file. the ideal situation would be to collect all errors and display them of the webpage for authors to find and correct. Ex. of dream code: *let **$db* := *app:open-db*(*$lang*) *let* *$topic-validation* := <ul>{*for **$x* *in **$db*/*[* name*()='topic' or *name*()='media'] *return if* (*contains*(* validate:dtd*(*$x*, *$app:dtd-topic*), 'BXVA0001')) *then *<li>File {*$x*/* base-uri*()} doesn't validate against the DTD.</li> *else *() } </ul> *let* *$menu-validation* := <ul>{*for **$x* *in **$db*/*[* name*()='menu'] *return if* (*contains*(* validate:dtd*(*$x*, *$app:dtd-menu*), 'BXVA0001')) *then *<li>File {*$x*/* base-uri*()} doesn't validate against the DTD.</li> *else *() }</ul> *return if* (*count*(*$topic-validation*/*li*) + *count*(* $menu-validation*/*li*[*normalize-space*(.) != '']) != 0) *then *<div>{*$topic-validation*}{*$menu-validation*} </div> *else *<ul><li *style*=*"color:green"*>All menus and topics validate against the DTDs.</li></ul> Instead, as soon as validate:dtd finds one single file with an error, it crashes the web page with an error message like this: Stopped at line 270, column 77 in /Applications/basex75/webapp/restxq/How2_Admin.xqm: [BXVA0001] Validation failed: 1355790115420.xml, 3:22: Element type "headin" must be declared. On top of this, the name of the file that is referenced in the error message doesn't exist. I've been querying the DB to find errors. Ex. Search for all files with 'headin' in this case. However, we'd really like the writer to be able to find their own faulty files. Are we misusing the function? Is this a bug? What can we do about it? Thanks in advance for you help! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

2 3

Binding a sequence to an external variable?
by Johannes Krampf 18 Dec '12

18 Dec '12

Hello, I would like to bind a sequence of xs:string to an external variable. I've tried various ways and after trying to read the source and the spec I fear that it might not be possible. First of all the code: Inside the xquery: declare variable $mysequence as xs:string* external; On the Java side: ClientQuery clientQuery = clientSession.query(query); clientQuery.bind(parameter, values.get(parameter)); I tried to give the type "xs:string*" as a third parameter to clientQuery.bind, but I got the error message "org.basex.core.BaseXException: [XPST0003] Unknown type 'xs:string*'", so this is not the way to do it. I furthermore tried to pass a string array as a parameter, but the variable got the value of the output of the toString method. When I tried with a list of Strings, I got again the output of the toString method. What I want is something like the following: let $mysequence := ("Hello", "world") but using sequence values transmitted from the java application using an external variable. Is there any way to achieve this? The goal is to test for membership in the sequence, i.e. allowing the parametrize the query using a variable number of possible values. Thanks for your assistance, Johannes

2 2

Full Text Index giving error for ..7.5 GB XML Data
by John Best 18 Dec '12

18 Dec '12

Dear XML Team, I have a XML DB of size 7.5GB. When creating an Full-Text Index I get an error message - * Out of Main Memory. You can try to: - increase Java's heap size with the flag -Xmx<size> - deactivate the text and attribute indexes.* I tried this with 7.3 and thru GUI... To make searching fast I need FT Index.. :( Another interesting thing I would like to highlight about this error.... In the data folder the FT Index files (ftxXX and ftyXX is the format of the files) are created... The first time when this happened, I checked the data folder and found that the above ftxXX and ftyXX files where up to ftx23 and fty23. Then I tried Full-Text index again with no success. Again I checked the same files. This time they were up to ftx36 and fty36... This might help you that is why I am mentioning..... -- Have a nice day JBest

2 1

xmas feature wishes ;-)
by Maximilian Gärber 16 Dec '12

16 Dec '12

Hi, will there be a JSONiq package under the tree (or anytime soon)? Regards, Max -- Maximilian Gärber axxepta solutions GmbH Postfach 51 02 38 13362 Berlin Tel +49 (0)30 499 147 66 Fax +49 (0)30 499 147 67 Mail gaerber(a)axxepta.de

2 2

setting multiple bindings in the GUI
by Randy reed 16 Dec '12

16 Dec '12

Hi All, I've got three external variables that I want to test in the GUI. I put Set Bindings $dname="/home/randy/..." and that works fine, but when I try to do more I get a syntax error. I've tried separating the entries with ; or , or spaces but none of that seems to work. if I do the full command separated by ; (i.e. set bindings $dname=...; set bindings $inputf=....; set bindings $inputn=....) only the last one takes. Anyone had success doing this? Thanks Randy -- Dr. Randall Reed Assistant Professor, Dept. of Philosophy and Religion Appalachian State University

3 4

Java bindings error
by Randy reed 14 Dec '12

14 Dec '12

Hi all, I switched from 6.8 to 7.5 and I'm generating an error I didn't get in 6.8 when using Java Bindings There error is :[FORG0006] Invalid call of Java function: TimeZone.inDaylightTime(java(), java()) It appears that its problem is using the output of one java function in another function. Is there some new way to do this in 7.5? Thanks Randy The code is: declare namespace tz = "java:java.util.TimeZone"; declare namespace dt = "java:java.util.Date"; declare namespace df = "java:java.text.DateFormat"; declare namespace cal = "java:java.util.Calendar"; declare namespace sdf = "java:java.text.SimpleDateFormat"; let $odate := xs:long(1345665817) let $edate := xs:long($odate)*1000 let $fdate := dt:new(xs:long($edate)) let $tmz := tz:getDefault() let $tbool := tz:inDaylightTime($tmz, $fdate) return ( <results> <jdate>{$fdate}</jdate> <Timezone>{$tmz}</Timezone> <Daylight>{$tbool}</Daylight> </results> ) -- Dr. Randall Reed Assistant Professor, Dept. of Philosophy and Religion Appalachian State University

2 3

Upcoming: XMLPrague 2013
by Christian Grün 14 Dec '12

14 Dec '12

Hi all! The XMLPrague Conference is the most important European event for XML specialists, developers and users. Next year, it will take place from Feb 8-10: http://www.xmlprague.cz/ Once again, our team will be present there, and it will be a pleasure for us to see you and talk to you live! What may be even more interesting: we are planning yet another BaseX User Meeting on the first day (Friday). We have just started to compile our agenda. As many of you are using BaseX for both exciting projects and inn productive environments, it would be great to have some of you talk about your experiences with BaseX. Would you be interested to contribute? We are looking forward to your feedback! Feel free to ask for more details, Christian

2 2

strange behavior of ft:mark() and ft:extract()
by Francis Avila 14 Dec '12

14 Dec '12

ft:mark() and ft:extract() cannot be used with any intermediate looping construct, at least in BaseX 7.3. For example: > create db ftex "<r><a>text example string</a></r>" > xquery ft:mark(db:open('ftex')/descendant::text()[. contains text 'example']) text example string > xquery db:open('ftex')/descendant::text()[. contains text 'example'] ! ft:mark(.) text example string Notice that the use of a loop means that ft:mark() no longer works. This doesn't work with ft:search() either: > create index fulltext Index 'FULLTEXT' created in 7.63 ms. > xquery for $n in ft:search('ftex','example') return ft:mark($n) text example string However this works if there's no intermediate variable binding (implicit copying) even if we do some xpath navigation. > xquery ft:mark(ft:search('ftex','example')/..) <a>text example string</a> Query executed in 0.96 ms. > xquery ft:mark(ft:search('ftex','example')/../..) <r> <a>text example string</a> </r> However (this is a separate bug), the document-node() parent/child ordering is messed up: > xquery ft:mark(ft:search('ftex','example')/../../..) <r> <a>text </a> </r> example string > xquery ft:mark(ft:search('ftex','example')/ancestor::document-node()) <r> <a>text </a> </r> example string It seems to me that probably there is some extra hidden metadata attached to the text node which is not preserved by any implicit copying, such as by loops. (Although I'm not sure what's going on with the document-node() example.) One example of where this is a big headache is when we want to know in which document was a the text match. Here is a natural implementation: QUERY: for $n in ft:search('ftex','example') return <r doc="{document-uri($n/ancestor::document-node())}">{ft:mark($n)}</r> RESULT: <r doc="ftex/ftex.xml">text example string</r> However in this case the text will not be marked. We can't even do this because the "marked" document does not have a reference to the document-node() (which is understandable, as there's probably a copy/modify/return transform under there): QUERY: xquery for $n in ft:mark(ft:search('ftex','example')/..) return <r doc="{document-uri($n/ancestor::document-node())}">{$n/(* | text())}</r> RESULT: <r doc="">text example string</r> The workaround I used is this: QUERY: let $matches := ft:search('ftex','example') for $n at $i in ft:mark(ft:search('ftex','example')/..) return <r doc="{document-uri($matches[$i]/ancestor::document-node())}">{$n/(* | text())}</r> This is extremely ugly, and shares with the last example the downside that I need to mark the *parent* node of the matched text node and then carefully unbox it in the return clause. Quite a "gotcha"! In addition to being ugly, I'm also worried that it may be incorrect because I don't know for sure that the order of search will always be exactly the same (although it *seems* to be). Even if the order is stable, what if the fulltext index is updated by another session between the first call to ft:search() and the second? I don't know what the right answer to all of this is, but the way things are seems very not-good. At the very least the problems with ft:mark() and ft:extract() should be documented with big red text! Perhaps a better method is to have a function with a data structure that contains the text matched text node (as a reference, so that node references are retained) *and* matching substrings explicitly and separately. E.g. a ft:search-with-info('ftex','string example') could return: (map { 'text' := text('test example string'), 'substrings' := (5,8, 14,6) }, ...) or maybe this: (map { 'text' := text('test example string'), 'substrings' := (<s start="5" length="8"/>, <s start="14" length="6"/> ) }, ...) or maybe just a sequence of alternating items: (text('test example string'), <matchinfo><s start="5" length="8"/><s start="14" length="6"/></matchinfo>, ...) Unfortunately I don't see how we can return a simple sequence of elements since the text node result would have to be a copy. With the results separated we could easily make a family of functions which accepts the same data structure and does mark() and extract()-like things with it. The pairs can be processed either with a tumbling window or a for loop with a "where $i mod 2" in it. -- Francis Avila

2 2