- BaseX-Talk - mailman.uni-konstanz.de

Urgent Help, concurrently update db cause db error, how???
by easy 02 Jan '14

02 Jan '14

Hi,all, I have a restxq web service to do insert or update xml doc , which is called by many clients to provide xml data to db, there will be 500000 files to provide to basex server. I have found some problem few times : 1) often, after insert tens of thousands xml file, the db "list" will cause error,cause db corrupt, I don't know why, How, please help me. I don't know how to control the concurrently update db. 2) I often found there will automatically startup a basexhttp service in the server even the manually started service still run. ???, maybe the "basexhttp stop" command can't stop the service though it say stopped. the restxq service code as fellow: ---------------------------------------------------------- declare %output:method("xml") %output:omit-xml-declaration("no") %output:media-type("application/xml") %updating %rest:path("/add_ehr_doc") %rest:POST("{$body}") %rest:header-param("pid","{$pid}") %rest:header-param("doc_id","{$doc_id}") %rest:header-param("op","{$op}","APND") function ehr:add_ehr_doc( $body,$pid as xs:string,$doc_id as xs:string,$op as xs:string ) { let $dbname: ='myehr' let $fp:=substring($pid,1,6)||'/'||substring($pid,7,8)||'/'||substring($pid,15)||'.xml' (: pid is a patient id code as: area_code-section_code-serialnumber :) return try{ if (db:exists($dbname,$fp)) then ( insert node ($body//*:DataSet) into (doc($dbname||"/"||$fp)//*:BasicDataset), db:output(<msg><Ack>200:append</Ack><Url>{$fp}</Url></msg>) ) else ( db:add($dbname, $body,$fp),db:output(<msg><Ack>200 new add</Ack><Url>{$fp}</Url></msg>) ) } catch *{ db:output(<msg><Ack>201 error</Ack></msg>) } }; -- 此致林晓东莫愁前路无知己，天下谁人不识君。

1 0

Re: [basex-talk] Again need help on ft:search() :)
by Dirk Kirsten 31 Dec '13

31 Dec '13

Hello Anand, please make sure to send all help requests to the mailing list - this way all users can benefit from it. At the moment I don't see much potential to improve your query (but I am no expert on full-text search, so this is merely my opinion). The "all words" keywords are unnecessary, as you only use one word. The most important consideration is if the Full-Text Index is actually applied to your query. You can check by analyzing the query info, something like "Applying full-text index" should be found. You can find more information at https://docs.basex.org/wiki/Full-Text#Index_Processing So, have you created a Full-Text Index? This should definitely be your first step, as this will certainly boost your performance. You should also check the query info as suggested and also take a look at the optimized query plan given by the compiler, as this can point you to some bottlenecks in your query. Cheers, Dirk On 28/12/13 10:25, Anand Chiney wrote: > Dear Dirk, > > How are you ? > > Need some help on using ft:search() in following condition, if applicable. > > Following is the structure of XML file. Around 6 GB of articles in 3 XML DBs > > --- > > > <Docs> > <Doc> > <Title name="Speed of Light" author="John Best" /> > <SDesc> > <Summary> > <SSum> > <Keyword w="Speed of Light"/> > Short Summary Text here. > </SSum> > <LSum> > Long Summary. > </LSum> > </Summary> > </SDesc> > <LDesc> > Complete Description of the Article. > </LDesc> > </Doc> > <Doc> > <Title name="Vibration Analysis" author="John Best" /> > <SDesc> > <Summary> > <SSum> > <Keyword w="Vibration"/> > Short Summary Text here. > </SSum> > <LSum> > <P>Long Summary Para 1</P> > <P>Long Summary Para 2</P> > </LSum> > </Summary> > </SDesc> > <LDesc> > Complete Description of the Article. > </LDesc> > </Doc> > </Docs> > > Execution of below query is taking around 1 min !!! > > for $db in("Doc1","Doc2", "Doc3") > for $x in doc($db)/Docs/Doc > where $x/SDesc/Summary/SSum/Keyword/lower-case(@w) contains text > "vibration" and ($x/SDesc/Summary/LSum/P/text() contains text "spring" all > words or SDesc/Summary/SSum/text() contains text "spring" all words) > return $x/Title/@name > > Can this be queried using ft:search() ? I tried but failed. > > > -- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

1 0

:)
by Montante Tuggles 30 Dec '13

30 Dec '13

Happy new year !

1 0

BaseX for noobs : how to add user module ?
by jean-marc Mercier 30 Dec '13

30 Dec '13

Hello, I am trying to add some external features to BaseX, and would like to use them into a BaseX client XQUERY module. I can't see any simple way to do it, is there any ? Here is the scenario : suppose that you want to add a super-mega map module to BaseX, let us call it supermegamap, that is a collection of functions : supermegamap:new, supermegamap:contains, supermegamap:get. I would like to use my supermegamap through BaseX via a xquery module : let $my_supermegamp := supermegamap:new(). I see only two quite not very handy ways to do this : 1) The first one is to use JAVA bindings. Ok, it will work, but is not really elegant. 2) The second one is to fork BaseX code : adding the JAVA code to BaseX, modifying the file Function.JAVA. That might work, should be efficient, but it is quite heavy, and overall, will be most probably a dead-code the next time I will have to upgrade BaseX. Is there any alternative way ? Thx Jean-Marc

3 3

Re: [basex-talk] BaseX-Talk Digest, Vol 48, Issue 60
by Anand Chiney 30 Dec '13

30 Dec '13

Hi, The number (1, 2, 3,....) are the various searches done.. The huge amount (its not $$$$ ;) ) is the memory usage in K as indicated by Windows Task Manager. Query's, rather XQuery's are of various kinds, including XQuery functions like disitnct-values, casting functions, and many more.. some using joins, some using ft:search()... Well, I think, the reason would be in JAVA code itself, not related to BaseX.. On Mon, Dec 30, 2013 at 4:30 PM, <basex-talk-request(a)mailman.uni-konstanz.de > wrote: > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. Memory usage !! (John Best) > 2. Re: Memory usage !! (jean-marc Mercier) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 30 Dec 2013 16:04:40 +0530 > From: John Best <johnbest5673(a)gmail.com> > To: basex-talk(a)mailman.uni-konstanz.de > Subject: [basex-talk] Memory usage !! > Message-ID: > <CAO14Co4C85Y+VA6fURygkb-jUbeqX4qJ0Hm= > biduq8vMFDnzKg(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Dear Team, > > I have developed and application using Eclipse RCP. I am using BaseX 7.6 to > store XML data. > > Yesterday, when I was testing the application with various kind of > searches, the response was quite slow as I went ahead with more and more > searches. > > To my surprise, I notices the in Windows Task Manager, the javaw.exe was > consuming more memory... Then I restarted the laptop and did this procedure > again > > Following is the memory usage - > > 1 - 84920 > 2 - 101584 > 3 - 108224 > 4 - 109284 > 5 - 111768 > 6 - 128468 > 7 - 160616 > 8 - 180212 > 9 - 198313 > 10 - 212592 > > > Why this is so ? and how can I reduce this memory usage by javaw.exe ? > > > > -- > Have a nice day > JBest >

1 0

Memory usage !!
by John Best 30 Dec '13

30 Dec '13

Dear Team, I have developed and application using Eclipse RCP. I am using BaseX 7.6 to store XML data. Yesterday, when I was testing the application with various kind of searches, the response was quite slow as I went ahead with more and more searches. To my surprise, I notices the in Windows Task Manager, the javaw.exe was consuming more memory... Then I restarted the laptop and did this procedure again Following is the memory usage - 1 - 84920 2 - 101584 3 - 108224 4 - 109284 5 - 111768 6 - 128468 7 - 160616 8 - 180212 9 - 198313 10 - 212592 Why this is so ? and how can I reduce this memory usage by javaw.exe ? -- Have a nice day JBest

2 1

[ANN] Higher-Order XQuery Modules
by Leonard Wörteler 30 Dec '13

30 Dec '13

Hello everyone, I want to announce the small collection of XQuery 3.0 modules [1] that I've written over the last few weeks. I use them mostly for testing the BaseX optimizer, but there should be no BaseX-specific XQuery code in the repository. I have implemented some purely functional data structures including maps based on AVL and Red-Black Trees that both support arbitrary values as keys and efficient deletion. Some documentation can be found under each module's namespace URI or under [2]. Constructive feedback of any kind (suggestions, bug reports, pull requests) would be much appreciated. Hope you like it, cheers, Leo [1] https://github.com/LeoWoerteler/xq-modules/ [2] http://www.woerteler.de/xquery/modules/

2 3

INLINER : bug ?
by jean-marc Mercier 30 Dec '13

30 Dec '13

Hi Everybody @Leo, there might be a bug in the BaseX INLINER (I am currently profiling your module https://github.com/LeoWoerteler/xq-modules/) To reproduce it : - use the 3 files attached. - Edit baseX config file .baseX, set the following Local options : # Local Options INLINELIMIT = 5 - launch the XQUERY file MapProfiler.xquery. With these settings, the profile function works finely. Then - Edit baseX config file .baseX, set the following Local options : # Local Options INLINELIMIT = 6 With these settings, I am raising a [XPTY0004] Single item expected error. Cheers, Jean-Marc

2 2

what's bad basex ? slow insert doc after continuesly add/insert, lost db access..
by easy 27 Dec '13

27 Dec '13

Hi,all, after continuesly insert/add doc to basex db up to 80000 file with restxq web service, there are some nonsens result: very slow for insert/add doc to db. some time can't list db in basex cmd line. after restart ,it's ok. and this will cause the xquery can't work like collection('db/path')//.., say something like: can't found resource ..../webapp/db . who can give some reason or what 's wrong with me? Thanks. -- 此致 easy 莫愁前路无知己，天下谁人不识君。

1 0

restxq with html form get
by e-letter 27 Dec '13

27 Dec '13

Readers, A new file is trying to be created, with a html form using the element 'select'. The objective is to select from a menu and return a new html page after processing of the xquery function. >From the web page about http requests (http://docs.basex.org/wiki/RESTXQ#HTTP_Methods) where is the content type to be written in the xqm file? (:~ : This module contains some basic examples for RESTXQ annotations : @author BaseX Team :) module namespace page = 'http://basex.org/modules/web-page'; (:~ : This function generates the welcome page. : @return HTML page :) declare %rest:path("") %output:method("xhtml") %output:omit-xml-declaration("no") %output:doctype-public("-//W3C//DTD XHTML Basic 1.1//EN") %output:doctype-system("http://www.w3.org/TR/xhtml-basic/xhtml-basic11.dtd") function page:start() as element(Q{http://www.w3.org/1999/xhtml}html) { <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>BaseX HTTP Services</title> <link rel="stylesheet" type="text/css" href="static/style.css"/> </head> <body> <h3>Example 2</h3> <p>The next example presents how form data is processed via RESTXQ and the POST method:</p> <form method='get' name='databasequeryform' > <p> <select name='booktitle' > <option> Everyday Italian </option> <option> Harry Potter </option> </select> <input type="submit" value='send' /> </p> </form> </body> </html> }; declare %rest:path("/form") %rest:GET function page:post() { let $x := fn:doc("xmldatabasefile.xml")/bookstore/book/title[contains(text(),'Everyday Italian')]/parent::* return $x }; declare %rest:path("/put") %rest:PUT("{$body}") function page:put($body) { || $body }; Extract of xml file: <?xml version="1.0" encoding="utf-8"?>  <?xml-stylesheet type='text/xsl" href="xstylesheetexample.xslt" ?> <bookstore> <book category="COOKING"> <title lang="en" > Everyday Italian </title> <author>Giada De Laurentiis</author> <year>2005</year> <price>30.00</price> </book> ...

1 0