November 2015 - BaseX-Talk - mailman.uni-konstanz.de

BaseX, Zorba, e-mail module
by Tim Thompson 05 Feb '16

05 Feb '16

Hello, I would like to be able to send e-mail from a Web app I am developing in BaseX and was wondering whether anyone else had implemented this functionality (or whether a BaseX e-mail module might be in the works?). I see that Zorba has an e-mail module with support for SMTP and IMAP (not to mention many other useful modules), which leads me to wonder whether anyone has worked on a BaseX connector for Zorba. In the meantime, I decided to try using the BaseX process module to call Zorba, like so: proc:system( 'zorba', ( '-q', convert:binary-to-string( db:retrieve( 'out', 'email.xq' ) ) ) ) I stored the XQuery code for Zorba as a raw file in the BaseX database, since storing it in the webapp directory caused an error, due to the unrecognized Zorba module declarations. This approach seems to work adequately, but does anyone have any suggestions for improvement, or a different approach? Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

3 4

GUI feature request "Add as import"
by Andy Bunce 03 Jan '16

03 Jan '16

Hi, I often find myself adding module imports in the GUI and I wonder if the following suggestion might speed up that process. Add a new menu item to the filelist right click context menu "Add as import" when the file selection is a XQuery module. The action would be to add a line to the current edit window source with import module namespace {$prefix} = '{$namespace}' at {$path}; Where $prefix and $namespace} are read from the source of the selected file, and $path is calculated from the relative path from the edit file source to the selected filelist source. The line would be inserted at a suitable place in the prologue or at the current location if that is too tricky. /Andy

2 4

AUTOOPTIMIZE - was true now false
by James Ball 02 Dec '15

02 Dec '15

Hello, I’m trying out AUTOOPTIMIZE for the first time and I’m not getting what I expect. Running: BaseX 8.3 beta ec9b627 A new database is created with a single document which has TEXTINDEX, ATTRINDEX, UPDINDEX and AUTOOPTIMISE all set to true. This is what INFO DATABASE shows (as I expect): Indexes Up-to-date: true TEXTINDEX: true ATTRINDEX: true FTINDEX: false TEXTINCLUDE: ATTRINCLUDE: FTINCLUDE: LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: true AUTOOPTIMIZE: true MAXCATS: 100 MAXLEN: 96 Now I add a new document (or more than one) to the database. I expect that Up-to-date will still be true and that AUTOOPTIMIZE will still be true. But what I see is both are now false: Indexes Up-to-date: false TEXTINDEX: true ATTRINDEX: true FTINDEX: false TEXTINCLUDE: ATTRINCLUDE: FTINCLUDE: LANGUAGE: English STEMMING: false CASESENS: false DIACRITICS: false STOPWORDS: UPDINDEX: true AUTOOPTIMIZE: false MAXCATS: 100 MAXLEN: 96 Perhaps this is the expected behaviour and I’ve misunderstood AUTOOPTIMIZE? Or is there a bug? Many thanks, James

3 4

Running BaseX under Tomcat on a commercial Java hosting provider
by C. M. Sperberg-McQueen 01 Dec '15

01 Dec '15

In case there are other users of XQuery who are looking for a way to get an XQuery engine on the Web but are not sure how to go about it, I have described my adventures setting up one approach (running BaseX 7.9 under Tomcat 6 on a commercial Java hosting provider) in a blog post today, hoping that it might enable someone else to do it with less trouble and confusion than I had: http://cmsmcq.com/mib/?p=1368 When time allows, I expect to upgrade my Tomcat to Tomcat 7 so that I can install a current BaseX; I'll update the post then with information for BaseX 8.*. if those who understand Java development and deployment better than I do (or wish to do) find errors in the exposition or simpler ways to get things set up, I would be glad to have them pointed out. I'd also be interested in speculations as to why setting the default user in web.xml seems not to have worked, but at this point I doubt that it's worth significant effort to solve that problem. -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

3 2

RESTXQ : existing annotation for response headers ?
by Etanchaud Fabrice 30 Nov '15

30 Nov '15

Dear all, When I want to set response headers (for example to force a "save as..."), I currently have to write a specific response before the response body : <rest:response> <http:response> <http:header name="Content-Type" value="text/comma-separated-values; charset=utf-8"/> <http:header name="Content-Disposition" value="attachment; filename={concat($base, '_Parthenay.csv')}"/> </http:response> </rest:response> Is there an annotation to set header values directly in the function declaration ? Best regards, Fabrice

3 3

Geo Module Not found
by E. Wray Johnson 27 Nov '15

27 Nov '15

I am getting the following error: [XQST0059] Module 'http://expath.org/ns/geo' not found. I used the Windows exe setup. What am I missing?

2 1

Location of users.xml
by Tom Rauchenwald 27 Nov '15

27 Nov '15

Hi, is it possible to override the expected location of users.xml? >From the wiki (http://docs.basex.org/wiki/User_Management) > The permission file has been moved from the home directory to the > database directory. It was renamed from .basexperm to users.xml We're using the war distribution, and would prefer to either specify the location of users.xml via a parameter or have it reside in the war. Is this currently possible? Thanks, -tom

2 2

db:create, namespaces not stripped when input is variable?
by Hondros, Constantine (ELS-AMS) 27 Nov '15

27 Nov '15

Hello all, Any particular reason behind this observed behaviour? (Basex 8.3). .............................................................. [1] .............................................................. declare variable $test := <test xmlns:ce="http://www.elsevier.com/xml/ani/common" xmlns="http://www.elsevier.com/xml/ani/ani"> <ce:initials>M.S.</ce:initials> </test>; db:create('test', $test, 'blah', map {'stripns':true()}) .............................................................. [2] .............................................................. db:open('test')/* .............................................................. [3] .............................................................. <test xmlns="http://www.elsevier.com/xml/ani/ani" xmlns:ce="http://www.elsevier.com/xml/ani/common"> <ce:initials>M.S.</ce:initials> </test> I would have expected the namespaces to be shredded ... Cheers, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

3 2

Cant add raw files explicitly or text index ms office docs
by E. Wray Johnson 26 Nov '15

26 Nov '15

Here is what I want to do: For a given folder and all its subfolders on my physical dive, mirror its contents including the contents of archives, parsing xml, json,html, text, etc. using their respective parser skipping invalids, and adding all other files as raw. I want archive files (*.zip, *.doxc) to be added as raw, however I want the text inside archive files like docx (ms-word) to be indexed and any files in the archives files that match a filter to be indexed. Note: It would be nice if there was a single db:add method that allowed me to specify a map of filters to parsers with options, where all files that do not match a filter (or are invalid) will be optionally added as raw.

3 3

file:read-binary performance
by Jérôme Chauveau 25 Nov '15

25 Nov '15

Hi, My restxq app is very slow to deliver raw files (css, images, js). For example, the restxq method takes around 600ms to load a png file in my browser when it takes only 2ms in a "direct" or "static" way. --------------------------------------- declare %rest:path("/editions/static/{$project}/{$file=.+}") function max.file:file( $project as xs:string, $file as xs:string ) as item()+ { let $path := file:parent(file:base-dir()) || 'editions/' || $project || '/ui/' || $file return ( web:response-header(map { 'media-type': web:content-type($path) }), file:read-binary($path) ) }; --------------------------------------- Why such a difference? How can I improve it? I am using BaseX 8.3 (war version). Regards, Jérôme -- Jérôme Chauveau - CERTIC-DSI - Campus I Université de Caen Normandie

2 2

Assigning incrementing number to 100ks of elements
by Chuck Bearden 24 Nov '15

24 Nov '15

BaseX 8.3.1, installed on Linux today from zip archive. I have about 137,000 target elements, each having an average of 550 descendant elements, and each taking about 45k of space on disk. The target elements each have an @uuid value, and I'd like to extract each @uuid value and assign it a unique number from an incrementing sequence of integers. I'd like to be able to start the sequence at an arbitrary value. I tried a recursive function I borrowed and modified from a StackOverflow post: declare function local:loop($seq, $count) { if(empty($seq)) then () else let $prod := $seq[1], $count := $count + 1 return ( <result> <id>{$count}</id> <uuid>{data($prod)}</uuid> </result>, local:loop(tail($seq), $count) ) }; and calling it like so: return <results>{ local:loop($pubs, 0) }</results> where $prod is a sequence of all the @uuid attributes (not the values, and not the whole elements). My query fails with the error: [bxerr:BASX0005] Stack Overflow: Try tail recursion? I'm using -Xmx6g and -Xss4m as flags for Java in the basexgui script. Is there some way to get BaseX to use tail recursion? Or is there another approach to doing what I want? Many thanks, Chuck

2 1

issue with stack trace in query-info window
by Rob Stapper 24 Nov '15

24 Nov '15

Hi Christian, I found that in case of a runtime error the links in the stack trace in the Query info window to the different locations in the program don't work anymore. I use of course the latest snapshot: BaseX 8.4 beta 3b6f6e0 Regards, Rob --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 1

basex performance for high request number
by Filippo Gaudenzi 23 Nov '15

23 Nov '15

Hi all, I’m running some scripts to solve a NP-problem in order to check trees compatibility and i’m using Basex as a database for my hierarchy structure, indeed for each path/each node i run a query on the basexserver. This brings to have for a tree with 7 paths a 7! permutations and therefore at least 7!*4=21.000 queries (the query below) in a lapse of 5minutes. If i use the db in my localhost environment i see that basexserver goes over 150% of cpu usage. I was wondering how to increase the performance without causing the server crash. basex is running on a 2,6 GHz Intel Core i5 with 16gb of Ram, OS X 10.10.5 basex version is 8.2 Here the code i use: public class BaseXOntologyManager{ private final int PORT; private final String PASSWORD; private final String USERNAME; private final String ADDRESS; private final String DBNAME; /** * Constructor of the Basex ontology Manager * @param address The address of the DB * @param port The port to use to connect to the DB * @param username The username to use to connect to the DB * @param password The password to use to connect to the DB * @param dbName The db Name */ public BaseXOntologyManager(String address, int port, String username, String password, String dbName){//,boolean connected) { this.ADDRESS = address; this.PORT = port; this.USERNAME = username; this.PASSWORD = password; this.DBNAME = dbName; } private BaseXClient getOpenConnection() { try { BaseXClient session = new BaseXClient(this.ADDRESS, this.PORT, this.USERNAME, this.PASSWORD); return session; } catch (IOException e) { return null; } } /** * Return the subClasses of the element based on the ontology in the DB, * if the direct is true returns only the parents otherwise also the descendants * @param keyName The name of the Ontology to use * @param element the element to search in the param * @param direct true if you want only the parents * @return A list of subClasses of the ontology * @throws Exception when element doesn't belong to Ontology */ public ArrayList<String> getSubClasses(String keyName, String element, boolean direct) throws Exception { BaseXClient client = this.getOpenConnection(); if (client != null) { if(!checkExistence(element)) throw new Exception("notFound"+element); ArrayList<String> result = new ArrayList<String>(); String input = null; if (direct == true) { input = "for $n in db:open('" + this.DBNAME +"', '/" + keyName + "')//*[parent::" + element + "] return $n/name()"; } else { input = "for $n in db:open('" + this.DBNAME +"', '/" + keyName + "')//*[ancestor::" + element + "] return $n/name()"; } try { final BaseXClient.Query query = client.query(input); while (query.more()) { result.add(query.next()); } query.close(); result.add(element); return result; } catch (IOException e) { return null; } finally { try { client.close(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } } }else{ System.out.println("CLIENT NULL"); } return null; } Thanks, Filippo

2 2

issue with the latest snapshot
by Rob Stapper 23 Nov '15

23 Nov '15

Gute Morgen, After installing the latest Basex snaphot (8.4) this morning, the new version couldn't resolved function calls to function that are located in imported modules while compiling. I get an error messages: "Unknown function '...'; similar: '...' ." After installing the previous version (8.3) everything worked fine again. I'm using windows 7 and basex is installed on my L:-drive. Hope this is enough information. Rob Stapper --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 12

Curious "Stack Overflow: Try tail recursion?" error message
by Andy Bunce 20 Nov '15

20 Nov '15

Hi, Using a fresh install on 8.3.1 and running basexgui.bat on windows. I install this expath package https://github.com/expkg-zone58/ex-xparse/releases/download/v0.0.5/ex-xpars… Then, paste the following code into an editor window.. ------------------------ import module namespace xp="http://expath.org/ns/xparse"; "http://www.xqueryfunctions.com/xq/functx-1.0-doc-2007-01.xq" !unparsed-text(.) !xp:parse(.,map{"lang":"xquery"}) ----------------------- The result is the error... Stack Overflow: Try tail recursion? The curious part is.. if I keep retrying by clicking the execute button - after around 10 tries it starts working and seems to continue to work every time after that within that GUI instance. I have reproduced this on another pc. /Andy

2 2

Re: [basex-talk] Counting Occurences
by Christian Grün 20 Nov '15

20 Nov '15

Hi Ana (cc to the list), > I've tried using the "group by" clause, however, it only returns to me the > list of the present provinces, without the frequence, or, if I use the > "count" function, in only returns 1 (Since every curriculum on the > collection only contains a single occurence the province) Did you combine group by and count (as shown in the Wiki)? How does your current query look like? Christian > > Do you have any idea of how could I list the frequence of each province? > > Thank you for your help. > > 2015-11-20 15:17 GMT-02:00 Christian Grün <christian.gruen(a)gmail.com>: >> >> Hi Ana, >> >> The "group by" clause will help you here [1]. Hope this helps; feel >> free to ask if you need more examples. >> >> Christian >> >> [1] http://docs.basex.org/wiki/XQuery_3.0#group_by >> >> >> On Fri, Nov 20, 2015 at 6:06 PM, Ana Victória Ladeira >> <ladeira.ana(a)gmail.com> wrote: >> > Good afternoon! >> > >> > I have a collection of XML files in which all files contain the >> > attribute >> > //ENDERECO-PROFISISONAL/@UF >> > >> > In which UF is the province in which the person to whom the file makes >> > reference to lives. >> > >> > I want to develop a query that returns to me the number of occurences of >> > each province in the collection. I've tried using count and >> > distinct-values, >> > but it always return the value "1" for me. >> > >> > The query I'm trying to use is the following: >> > for $x in >> > >> > collection('Teste500')/CURRICULO-VITAE/DADOS-GERAIS/ENDERECO/ENDERECO-PROFISSIONAL >> > let $y := count(distinct-values($x/@UF)) >> > return $y >> > >> > The desired result would be, in a collection of documents in which 5 >> > people >> > live in SP, 3 in DF and 9 in AM would be the following: >> > >> > SP: 5 >> > DF: 3 >> > AM:9 >> > >> > Can somebody please help me with that? I'm still struggling with xQuery. >> > >> > > >

2 1

Counting Occurences
by Ana Victória Ladeira 20 Nov '15

20 Nov '15

Good afternoon! I have a collection of XML files in which all files contain the attribute //ENDERECO-PROFISISONAL/@UF In which UF is the province in which the person to whom the file makes reference to lives. I want to develop a query that returns to me the number of occurences of each province in the collection. I've tried using count and distinct-values, but it always return the value "1" for me. The query I'm trying to use is the following: for $x in collection('Teste500')/CURRICULO-VITAE/DADOS-GERAIS/ENDERECO/ENDERECO-PROFISSIONAL let $y := count(distinct-values($x/@UF)) return $y The desired result would be, in a collection of documents in which 5 people live in SP, 3 in DF and 9 in AM would be the following: SP: 5 DF: 3 AM:9 Can somebody please help me with that? I'm still struggling with xQuery.

2 1

Applicative Cache
by Marco Lettere 20 Nov '15

20 Nov '15

Hi all, I went though the emails talking about mainmemory databases and related but I've not been able to find an answer to this loud thinking ... Would it be possible to create an in memory database with a RESTXQ call and keep it accessible for other RESTXQ invocations as if it was a disk db throughout the lifecycle of a basexhttp server? Application level cacheing of (possibly distributed) dbs or management of transient volatile data items like tokens are the scenarios I have in mind. Just wondering whether this makes any sense. Whether it's feasible or whether someone already did this with other means. Thanks a lot! Marco.

3 6

Bash command
by meumapple 19 Nov '15

19 Nov '15

Hi all, I am using proc:system for a bash command but I cannot run it. It looks like opennlp postagger model.bin < file.txt. I suspect that the problem is the less-than sign, which is converted into < Is there any way to keep the less-than sign and pass it to the bash properly? Thanks. Joseph

4 8

How to improve performance of join query
by Chuck Bearden 18 Nov '15

18 Nov '15

Hi all, I'm joining elements from two sets of documents in a single database. There are many elements (six figures) in each set. The query doesn't finish after several minutes. My suspicion is that the indices aren't being used, except to limit the fingerprint 'content' elements to those with "Person" as the value of 'contentFamily'. Perhaps my data and query demand some form of optimization that hasn't been implemented yet, but perhaps there is something I could do to fix the query. I'm using BaseX 8.3.0. The query: ------------------------- xquery version "3.0"; declare namespace fingerprint="http://atira.dk/schemas/pure4/wsdl/template/fingerprint/current"; declare namespace concept="http://atira.dk/schemas/pure4/wsdl/template/concept/current"; declare namespace core="http://atira.dk/schemas/pure4/model/core/current"; declare namespace xsi="http://www.w3.org/2001/XMLSchema-instance"; declare namespace fincur="http://atira.dk/schemas/pure4/model/template/fingerprint/current"; declare namespace concur="http://atira.dk/schemas/pure4/model/template/concept/current"; declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method "csv"; declare option output:csv "header=yes, separator=comma"; let $coll := collection('pure_20151109') let $concept_contents := $coll/concept:GetConceptResponse/core:result/core:content let $fingerprint_contents := $coll/fingerprint:GetFingerprintResponse/core:result/core:content for $fin_con in $fingerprint_contents[fincur:contentFamily/text()="Person"] /fincur:rankedConcepts/fincur:rankedConcept let $concept_uuid := $fin_con/fincur:conceptUuid let $rank := $fin_con/fincur:rank let $weighted_rank := $fin_con/fincur:weightedRank let $person_uuid := $fin_con/ancestor::core:content[1]/@uuid let $concept_name := $concept_contents[@uuid=$concept_uuid]/concur:name/core:localizedString return <csv> <record> <person_uuid>{data($person_uuid)}</person_uuid> <concept_uuid>{$concept_uuid/text()}</concept_uuid> <rank>{$rank/text()}</rank> <weighted_rank>{$weighted_rank/text()}</weighted_rank> <concept_name>{$concept_name/text()}</concept_name> </record> </csv> ------------------------- The query info from the query, which I stopped after 3 minutes: ------------------------- Error: Interrupted. Compiling: - pre-evaluating collection("pure_20151109") - inlining $coll_0 - inlining $fingerprint_contents_2 - applying text index for "Person" - inlining $rank_5 - inlining $weighted_rank_6 - inlining $person_uuid_7 - inlining $concept_name_8 Query: xquery version "3.0"; declare namespace fingerprint="http://atira.dk/schemas/pure4/wsdl/template/fingerprint/current"; declare namespace concept="http://atira.dk/schemas/pure4/wsdl/template/concept/current"; declare namespace core="http://atira.dk/schemas/pure4/model/core/current"; declare namespace xsi="http://www.w3.org/2001/XMLSchema-instance"; declare namespace fincur="http://atira.dk/schemas/pure4/model/template/fingerprint/current"; declare namespace concur="http://atira.dk/schemas/pure4/model/template/concept/current"; declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method "csv"; declare option output:csv "header=yes, separator=comma"; let $coll := collection('pure_20151109') let $concept_contents := $coll/concept:GetConceptResponse/core:result/core:content let $fingerprint_contents := $coll/fingerprint:GetFingerprintResponse/core:result/core:content for $fin_con in $fingerprint_contents[fincur:contentFamily/text()="Person"] /fincur:rankedConcepts/fincur:rankedConcept let $concept_uuid := $fin_con/fincur:conceptUuid let $rank := $fin_con/fincur:rank let $weighted_rank := $fin_con/fincur:weightedRank let $person_uuid := $fin_con/ancestor::core:content[1]/@uuid let $concept_name := $concept_contents[@uuid=$concept_uuid]/concur:name/core:localizedString return <csv> <record> <person_uuid>{data($person_uuid)}</person_uuid> <concept_uuid>{$concept_uuid/text()}</concept_uuid> <rank>{$rank/text()}</rank> <weighted_rank>{$weighted_rank/text()}</weighted_rank> <concept_name>{$concept_name/text()}</concept_name> </record> </csv> Optimized Query: let $concept_contents_1 := (db:open-pre("pure_20151109",0), ...)/concept:GetConceptResponse/core:result/core:content for $fin_con_3 in db:text("pure_20151109", "Person")/parent::fincur:contentFamily/parent::core:content[parent::core:result/parent::fingerprint:GetFingerprintResponse/parent::document-node()]/fincur:rankedConcepts/fincur:rankedConcept let $concept_uuid_4 := $fin_con_3/fincur:conceptUuid return element csv { (element record { (element person_uuid { (data($fin_con_3/ancestor::core:content[position() = 1]/@uuid)) }, element concept_uuid { ($concept_uuid_4/text()) }, element rank { ($fin_con_3/fincur:rank/text()) }, element weighted_rank { ($fin_con_3/fincur:weightedRank/text()) }, element concept_name { (($concept_contents_1)[(@uuid = $concept_uuid_4)]/concur:name/core:localizedString/text()) }) }) } Query plan: <QueryPlan compiled="true"> <GFLWOR> <Let> <Var name="$concept_contents" id="1"/> <IterPath> <DBNodeSeq size="51261"> <DBNode name="pure_20151109" pre="0"/> <DBNode name="pure_20151109" pre="1603"/> <DBNode name="pure_20151109" pre="2781"/> <DBNode name="pure_20151109" pre="5227"/> <DBNode name="pure_20151109" pre="7212"/> </DBNodeSeq> <IterStep axis="child" test="concept:GetConceptResponse"/> <IterStep axis="child" test="core:result"/> <IterStep axis="child" test="core:content"/> </IterPath> </Let> <For> <Var name="$fin_con" id="3"/> <CachedPath> <ValueAccess data="pure_20151109" type="TEXT"> <Str value="Person" type="xs:string"/> </ValueAccess> <IterStep axis="parent" test="fincur:contentFamily"/> <IterStep axis="parent" test="core:content"> <CachedPath> <IterStep axis="parent" test="core:result"/> <IterStep axis="parent" test="fingerprint:GetFingerprintResponse"/> <IterStep axis="parent" test="document-node()"/> </CachedPath> </IterStep> <IterStep axis="child" test="fincur:rankedConcepts"/> <IterStep axis="child" test="fincur:rankedConcept"/> </CachedPath> </For> <Let> <Var name="$concept_uuid" id="4"/> <IterPath> <VarRef> <Var name="$fin_con" id="3"/> </VarRef> <IterStep axis="child" test="fincur:conceptUuid"/> </IterPath> </Let> <CElem> <QNm value="csv" type="xs:QName"/> <CElem> <QNm value="record" type="xs:QName"/> <CElem> <QNm value="person_uuid" type="xs:QName"/> <FnData name="data([items])"> <CachedPath> <VarRef> <Var name="$fin_con" id="3"/> </VarRef> <IterPosStep axis="ancestor" test="core:content"> <Pos min="1" max="1"/> </IterPosStep> <IterStep axis="attribute" test="uuid"/> </CachedPath> </FnData> </CElem> <CElem> <QNm value="concept_uuid" type="xs:QName"/> <CachedPath> <VarRef> <Var name="$concept_uuid" id="4"/> </VarRef> <IterStep axis="child" test="text()"/> </CachedPath> </CElem> <CElem> <QNm value="rank" type="xs:QName"/> <IterPath> <VarRef> <Var name="$fin_con" id="3"/> </VarRef> <IterStep axis="child" test="fincur:rank"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="weighted_rank" type="xs:QName"/> <IterPath> <VarRef> <Var name="$fin_con" id="3"/> </VarRef> <IterStep axis="child" test="fincur:weightedRank"/> <IterStep axis="child" test="text()"/> </IterPath> </CElem> <CElem> <QNm value="concept_name" type="xs:QName"/> <CachedPath> <IterFilter> <VarRef> <Var name="$concept_contents" id="1"/> </VarRef> <CmpG op="="> <CachedPath> <IterStep axis="attribute" test="uuid"/> </CachedPath> <VarRef> <Var name="$concept_uuid" id="4"/> </VarRef> </CmpG> </IterFilter> <IterStep axis="child" test="concur:name"/> <IterStep axis="child" test="core:localizedString"/> <IterStep axis="child" test="text()"/> </CachedPath> </CElem> </CElem> </CElem> </GFLWOR> </QueryPlan> ------------------------- Thanks in advance for any advice! Chuck

2 2

BaseX request from Saxon XSL transformation
by cmarchand＠oxiane.com 18 Nov '15

18 Nov '15

Hello, I have to query a BaseX database from a XSL transformation. Does someone has ever done this ? I have no idea where to look in... I use SaxonEE, so I can write an extension function, if needed... Best regards, Christophe

3 3

Upcoming: XMLPrague 2016
by Christian Grün 18 Nov '15

18 Nov '15

Dear all, In around three months, it will be the 5th time that we'd like to organize user meeting in Prague. This time, the XML Prague conference takes place from Thursday to Saturday, Feb 11-13, and our meeting will be onThursday. As you all do so many exciting things with BaseX, we would be more than interested to have some of you as speakers! We are very thankful to those of you who have already talked on one of our previous meetings; it has been your contributions that made the past event so awesome! Please don't hesitate and tell us if you are interested in presenting your work to the members of the community. If you have presented before, you are of course invited as well. More to come, Christian

1 0

zip:update-entries - unexpected results
by Hondros, Constantine (ELS-AMS) 18 Nov '15

18 Nov '15

Hi all, I'm attempting to create a modified copy a zipfile using zip:update-entries, but my output is a zip structure with zero-byte files. Am I missing a step? I've followed the example in: http://docs.basex.org/wiki/ZIP_Module#zip:update-entries Code snippet as follows: declare function util:clean-zip($zip as xs:string, $out as xs:string) { copy $doc := zip:entries($zip) modify delete node $doc//zip:entry[ends-with(lower-case(@name), '.pdf')] return zip:update-entries($doc, $out) }; Thanks for any pointers ;-) C. ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 3

sending variables to xquery module via client api (python)
by Cholfin, Bryan 17 Nov '15

17 Nov '15

Hi- Hoping someone can help me with this. I'm trying to write a python script that calls an xquery module in the database. It seems to be pretty easy to use a line such as: session.execute("run gettext.xqm") and get back the response I want. What I want to do is pass a bound variable to the module. The client api has a provision to bind a variable to a query object, but I'm not making a query object. I'm simply executing the stored query. I don't find anything in the documentation to suggest that a variable can be added to the run command this way. Bryan Cholfin Associate Editor for Digital Projects and Metadata Harvard University Press 617.496.9116 79 Garden Street | Cambridge, MA 02138-1400 www.hup.harvard.edu<http://www.hup.harvard.edu/>

3 8

Restxq
by meumapple 16 Nov '15

16 Nov '15

Hi All, I am using restxq. I wrote a function which takes one variable (this should be a chunk of text, not a single word). The output is xml. I can have the function work if I pass the content of the variable with curl -X POST --data "variable=my text is long" http://address How can I get the same result of the curl command with an xquery function? More in general, is this a good way to write a function in restxq when the variable is not a single word but the content of a file? (I am not even sure if a file can be passed as such in restxq) Thanks, Joseph

3 4

Treating SQL Results
by Ana Victória Ladeira 16 Nov '15

16 Nov '15

Good morning. I have an XML file of the following format: <?xml version="1.0" encoding="UTF-8"?> <resultsets> <resultset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <row> <column name="TITULO">A Adequação de Sistemas Baseados em Casos no Sequenciamento Inteligente de Ações</column> <column name="NOMECOMPLETO">José Pacheco de Almeida Prado</column> </row> <row> <column name="TITULO">Sistemas Distribuídos Inteligentes</column> <column name="NOMECOMPLETO">José Pacheco de Almeida Prado</column> </row> <row> <column name="TITULO">Uma Arquitetura para Planejamento Inteligente</column> <column name="NOMECOMPLETO">José Pacheco de Almeida Prado</column> </row> <row> <column name="TITULO">Navegação de robôs móveis com Lógica Paracpmsistente Anotada e método de Campos Potenciais</column> <column name="NOMECOMPLETO">José Pacheco de Almeida Prado</column> </row> <row> <column name="TITULO">Adaptation in a Traffic Control System</column> <column name="NOMECOMPLETO">José Pacheco de Almeida Prado</column> </row> ... The file contains titles for articles and their authors.. I was asked to develop a query that returns the author's name followed by the titles of all of its articles, so the result would be something like this: <Author1>:<Article1>, <Article2>, <Article3>... <Author2>:<Article1>, <Article2>... ... Any ideas on how can I do it? Thank you in advance.

2 1

database triggers
by Alexander Haus 16 Nov '15

16 Nov '15

Hello BaseX team, I've recently read, that you are currently working on a trigger mechanism for BaseX. Can you already tell, when you are planning to include it in a future release? Thanks a lot, Alex

2 1

Typo in docs?
by Marco Lettere 13 Nov '15

13 Nov '15

Hi all, Just one question. The documentation page related to the json module states: *Signatures* |*json:serialize*($input as node()) as xs:string| |*json:serialize*($input as node(), $options as map(xs:string, xs:string)) as xs:string| *Summary* Serializes the node specified by|$input|as JSON, and returns the result as|xs:string|instance. The node is expected to conform to the output created by thejson:parse() <http://docs.basex.org/wiki/JSON_Module#json:parse>function. All other items will be serialized as specified for thejson output method <http://docs.basex.org/wiki/XQuery_3.1#JSON_Serialization>of the official specification. Items can also be serialized as JSON if theSerialization Parameter <http://docs.basex.org/wiki/Serialization>|method|is set to|json|. The|$options|argument can be used to control the way the input is serialized. But I think the signature should be for item()? for the first parameter (since it allows also for maps) isn't it? In effect inspect:function(json:serialize#2) returns <function name="json:serialize" uri="http://basex.org/modules/json"> <argument type="item()" occurrence="?"/> <argument type="item()" occurrence="?"/> <return type="xs:string"/> </function> If it's confirmed I could proceed with fixing it. Regards, Marco.

2 1

Re: [basex-talk] Fwd: Bash command
by meumapple 10 Nov '15

10 Nov '15

I can add that if I use "cat file" on the bash I get the right Greek characters, while the same command via proc:execute returns gibberish characters (the optional argument "utf-8" in proc:execute has no effect). Thanks. Joseph Il giorno 09/nov/2015, alle ore 19:04, meumapple <meumapple(a)gmail.com> ha scritto: Hi, I do not have any LANG variable set. If I redirect into a file the output of the bash command, I get a message with "utf-8" when I analyze this file with the command "file". And indeed the characters are displayed correctly. So the problem arises only when I run everything through proc:execute. If I save the result of the xquery script and analyze them with "file" I get the message "output: ERROR : line 22 : regexec error 17, (illegal byte sequence). And of course the Greek characters are messed up (but not the ASCII ones for English) Hope this helps... Joseph Il giorno 09/nov/2015, alle ore 15:18, Jens Erat <jens.erat(a)uni-konstanz.de> ha scritto: Please report the encoding of your actual files. What is the output of `env | grep LANG` (when stored in your bash script)? If you pipe the output in another file, what will `file -i [output-file]` report (`file` applies some magic to discover file formats and encoding)? > Am 09.11.2015 um 14:06 schrieb meumapple: > Hi All, > > I have been able to run the command just putting it in a bash file and then running this file with proc:execute. It works. The problem now is with the encoding of the text returned by proc:execute. In my bash I can display Greek characters, but they are messed up if returned by the proc:execute function. I also added utf-8 as a parameter for the function but nothing changes. How can I solve this? Thanks. > > Joseph > > Il giorno 09/nov/2015, alle ore 10:41, Jens Erat <jens.erat(a)uni-konstanz.de> ha scritto: > > Hi Joseph, > > As far as I understand the documentation, BaseX `proc:system` does not > use a shell interpreter by default. `<` or also the pipe `|` are > features offered by your shell, not the operating system. > > Instead of passing a single parameter, pass the command as parameter to > `/usr/bin/env sh -c`, which will make BaseX run a shell that interpretes > I/O indirection: > > proc:system('/usr/bin/env', ('sh', '-c', 'opennlp postagger > model.bin < file.txt')) > > This might be a reasonable thing to provide as a built-in function, I'm > also not sure whether one can connect some kind of stream/strings to > STDIN, STDOUT and STDERR. > > Regards from Lake Constance, Germany, > Jens > > Am 09.11.2015 um 10:19 schrieb Christian Grün: >>> Any suggestions? >> >> Any example? ;) >> Christian >> >> >> >> Thanks. >>> >>> Joseph >>> >>> >>> Inizio messaggio inoltrato: >>> >>> Da: meumapple <meumapple(a)gmail.com> >>> Data: 06 novembre 2015 20:52:08 CET >>> A: BaseX <basex-talk(a)mailman.uni-konstanz.de> >>> Oggetto: Bash command >>> >>> Hi all, >>> >>> I am using proc:system for a bash command but I cannot run it. It looks like >>> opennlp postagger model.bin < file.txt. I suspect that the problem is the >>> less-than sign, which is converted into < Is there any way to keep the >>> less-than sign and pass it to the bash properly? Thanks. >>> >>> Joseph -- Jens Erat Universität Konstanz Kommunikations-, Infomations-, Medienzentrum (KIM) Abteilung Basisdienste D-78457 Konstanz Mail: jens.erat(a)uni-konstanz.de

1 0

Re :Collection DataBase Locking
by Adi Babu 10 Nov '15

10 Nov '15

While updating the Basex collection data base if it fails because of some unexpected reason the particular database cannot be opened anymore and the message shown in loggers is "*Database ... is being updated, or update was not completed*". I have found that this is because of *upd.basex*. My question is how can I found and remove this file through code to proceed further with a safe execution of the collection database creation process. Thanks & Regards, Adi

2 1

Re: [basex-talk] Sub : Adding Special Characters to the Collection Database
by Christian Grün 10 Nov '15

10 Nov '15

Hi Madhan, I see you are using BaseX 7.9. Does the problem persist when using our latest release? What is the output of db:info() (or INFO DB)? Best, Christian On Mon, Nov 9, 2015 at 3:33 PM, madhan gopal <madhangopal828(a)gmail.com> wrote: > Dear Christian, > > Here the problem is,the xml attribute values are replacing with Alphabet > "A" > > Example: > in my Xml file i have the tag <MyTag myAttribute="10" > ..</MyTag> > if i add this xml to collection database, the value of attribute > myAttribute is replacing 10 with 'A' > > Note:Please check once attachment. > > > > On Mon, Nov 9, 2015 at 4:52 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Madhan, >> >> > I am facing the problem of adding special characters to the collection >> > Database in below cases, >> >> What do you mean by "problem", and what are "special characters"? >> Christian >> >> >> > Case-1: if the xml file size is huge(in GBs),at the time the collection >> > Database creation tacking more time, >> > Case-2: if the more no.of xml files are adding to the particular >> > collection >> > Database continuously >> > >> > Example: >> > >> > in Xml file <MyTag myAttribute="10" ...</MyTag> >> > >> > The values are adding like the following >> > in collection <MyTag myAttribute="A" ...</MyTag> >> > >> > Sample Collection adding code : >> > -------------------------------------------- >> > /* >> > This method takes the xml files list as input and process the list >> > Create the xml collection DB, adds xml to the collection if already >> > exists >> > replace it >> > */ >> > >> > public static void createCollection(ArrayList<String> listFiles){ >> > ClientSession clientSession = null; >> > String strIndexedFile=""; >> > try { >> > clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, >> > "<<USER>>", "<<PWD>>"); >> > String strDbName = "Sample_Collection"; >> > String strKeysFileLocation = >> > "D:\\TEMP\\7GB_Collection_TestingTeam\\Files"; >> > >> > String res = clientSession.execute("XQUERY db:exists('" + >> > strDbName + "')"); >> > //DB existence checking >> > if (!res.equalsIgnoreCase("true")) { >> > clientSession.execute("CREATE DB " + strDbName); >> > } else { >> > clientSession.execute("OPEN " + strDbName); >> > } >> > >> > for (int i = 0; i < listFiles.size(); i++) { >> > strIndexedFile = listFiles.get(i); >> > String tempFlag=clientSession.execute("XQUERY >> > db:exists('" + >> > strDbName + "','" + strIndexedFile + "')"); >> > clientSession.execute("ADD " + strKeysFileLocation + >> > File.separator + strIndexedFile); >> > } >> > System.out.println("InFormation:: >> > "+clientSession.execute("INFO")); >> > //optimizing the current opend DB using OPTIMIZE ALL command >> > clientSession.execute("OPTIMIZE ALL"); >> > System.out.println("InFormation:: >> > "+clientSession.execute("INFO")); >> > >> > } catch (Exception e) { >> > System.out.println("Exception files DB Updating: " + >> > e.getMessage()); >> > e.printStackTrace(); >> > } finally { >> > if (clientSession != null) { >> > try { >> > clientSession.close(); >> > } catch (IOException ex) { >> > ex.printStackTrace(); >> > } >> > } >> > } >> > } >> > >> > Hardware Specifications of Basex Sever : >> > ---------------------------------------------------------- >> > Operating System : Red Hat Enterprise Linux 64bit >> > OS Version : Linux 2.6.32 >> > Hard Disk : 500 GB >> > Memory Capacity : 16 GB Ram >> > Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 >> > Processor Speed : 2.80 GHz >> > Physical Processors : 8 >> > Virtual Processors/cores : 4 >> > Server Version : basex 7.9 >> > -- >> > Thanks & Regards >> > Madhangopal.P > > > > > -- > Thanks & Regards > Madhangopal.P

2 2

Fwd: basic JSON problem
by Florent Gallaire 10 Nov '15

10 Nov '15

Hi Christian, that's what I was afraid, http://www.w3.org/TR/xslt-30/#func-xml-to-json is just too complex to understand for me. What about your simple-json try ? to have an easy: my_request&method=json&json=format=simple Best regards Florent 2015-11-09 10:17 GMT+01:00 Christian Grün <christian.gruen(a)gmail.com>: > Hi Florent, > > When using a JSON format other than jsonml, your XML document to be > output must comply with the rules specified in the particular format > [1,2]: > > xml-to-json(<string > xmlns="http://www.w3.org/2005/xpath-functions">JSON</string>) > > ...versus... > > xml-to-json(<string>JSON</string>) > > [1] http://docs.basex.org/wiki/Json_Module > [2] http://docs.basex.org/wiki/XQuery_3.1#fn:json-to-xml > > > On Mon, Nov 9, 2015 at 4:19 AM, Florent Gallaire <fgallaire(a)gmail.com> wrote: >> I would like to use the new "basic" JSON via REST. >> With basex 8.3.1 and 8.3.2beta, >> my_request&method=json&json=format=jsonml >> works fine, but >> my_request&method=json&json=format=basic >> returns : >> [FOJS0006] Element 'article' has invalid namespace: ''. >> >> What's the problem ? >> >> Best regards >> >> Florent >> >> -- >> FLOSS Engineer & Lawyer -- FLOSS Engineer & Lawyer -- FLOSS Engineer & Lawyer

2 3

Re: [basex-talk] Sub : Adding Special Characters to the Collection Database
by Christian Grün 09 Nov '15

09 Nov '15

Hi Madhan, > I am facing the problem of adding special characters to the collection > Database in below cases, What do you mean by "problem", and what are "special characters"? Christian > Case-1: if the xml file size is huge(in GBs),at the time the collection > Database creation tacking more time, > Case-2: if the more no.of xml files are adding to the particular collection > Database continuously > > Example: > > in Xml file <MyTag myAttribute="10" ...</MyTag> > > The values are adding like the following > in collection <MyTag myAttribute="A" ...</MyTag> > > Sample Collection adding code : > -------------------------------------------- > /* > This method takes the xml files list as input and process the list > Create the xml collection DB, adds xml to the collection if already exists > replace it > */ > > public static void createCollection(ArrayList<String> listFiles){ > ClientSession clientSession = null; > String strIndexedFile=""; > try { > clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, > "<<USER>>", "<<PWD>>"); > String strDbName = "Sample_Collection"; > String strKeysFileLocation = > "D:\\TEMP\\7GB_Collection_TestingTeam\\Files"; > > String res = clientSession.execute("XQUERY db:exists('" + > strDbName + "')"); > //DB existence checking > if (!res.equalsIgnoreCase("true")) { > clientSession.execute("CREATE DB " + strDbName); > } else { > clientSession.execute("OPEN " + strDbName); > } > > for (int i = 0; i < listFiles.size(); i++) { > strIndexedFile = listFiles.get(i); > String tempFlag=clientSession.execute("XQUERY db:exists('" + > strDbName + "','" + strIndexedFile + "')"); > clientSession.execute("ADD " + strKeysFileLocation + > File.separator + strIndexedFile); > } > System.out.println("InFormation:: > "+clientSession.execute("INFO")); > //optimizing the current opend DB using OPTIMIZE ALL command > clientSession.execute("OPTIMIZE ALL"); > System.out.println("InFormation:: > "+clientSession.execute("INFO")); > > } catch (Exception e) { > System.out.println("Exception files DB Updating: " + > e.getMessage()); > e.printStackTrace(); > } finally { > if (clientSession != null) { > try { > clientSession.close(); > } catch (IOException ex) { > ex.printStackTrace(); > } > } > } > } > > Hardware Specifications of Basex Sever : > ---------------------------------------------------------- > Operating System : Red Hat Enterprise Linux 64bit > OS Version : Linux 2.6.32 > Hard Disk : 500 GB > Memory Capacity : 16 GB Ram > Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 > Processor Speed : 2.80 GHz > Physical Processors : 8 > Virtual Processors/cores : 4 > Server Version : basex 7.9 > -- > Thanks & Regards > Madhangopal.P

1 0

basic JSON problem
by Florent Gallaire 09 Nov '15

09 Nov '15

I would like to use the new "basic" JSON via REST. With basex 8.3.1 and 8.3.2beta, my_request&method=json&json=format=jsonml works fine, but my_request&method=json&json=format=basic returns : [FOJS0006] Element 'article' has invalid namespace: ''. What's the problem ? Best regards Florent -- FLOSS Engineer & Lawyer

2 1

Keeping a database optimized
by Paul Swennenhuis 05 Nov '15

05 Nov '15

What is the best way to keep the indexes of a database up-to-date? I have created a database with the option UPDINDEX set to ON (and db-info tells me that it is indeed switched on) and I have also created text and attribute indexes. Even so, after several insertions (using XQUERY insert node) I see that the UPTODATE flag is no longer checked and I have to run OPTIMIZE ALL to get back my performance. a) Is there a way to keep UPTODATE checked under all circumstances? And if not b) Is is advisable to run OPTIMIZE ALL after every insertion? (I guess not)

2 4

PHP client update
by Paul Swennenhuis 04 Nov '15

04 Nov '15

Hi all, I wrote an update for the BaseX PHP client, to handle the Digest authentication method as introduced in BaseX 8.0 See attachment. Regards, Paul

1 0

request concerning unittest in the GUI
by Rob Stapper 02 Nov '15

02 Nov '15

Hi Christian, For debugging my xquery-code I using the test-facilities from the unit-module in combination with trace and prof:dump command in the code for code-tracing. The problem is that when a unit-test fails it overwrites the trace-output in the Query-info window. So the code-tracing-info is gone just when I need it. It would make debugging a lot easier if the trace-output is appended after the unit-test-output in the Info-window of the GUI. Best, Rob --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 5

Best way to batch up a set of DB creates, updates and queries?
by Hondros, Constantine (ELS-AMS) 01 Nov '15

01 Nov '15

I'm looking for the best way to batch up a series of updating operations into a single pipeline that can be run at once. To give you an idea, we have to create multiple databases, perform a number of updating post-processes on each, before finally pulling certain fields out. I've got a lot of parameters that need to externalised somehow - database creation options, location of DTDs, XSDs, Catalogs, and unless I am trying this the wrong way I am finding it quite difficult. I can't write it all XQuery - I have a mixture of updating and non-updating functions. I can't get much traction out of the BXS scripting syntax - for example, I can't seem to create and use variables using this. What do other people out there use to glue these sorts of operations together? TIA, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 5

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk November 2015