- BaseX-Talk - mailman.uni-konstanz.de

loading basex from a cron job
by rsr 13 Feb '13

13 Feb '13

I am just getting started with basex and have been attempting to load data into a basex db via a cron job. I am running the basexclient inside my job redirecting a bxs command file to the client. this works well when i run my script off the command line by fails when run via cron. i've checked the obvious stuff (relative paths, permissions etc). Does anyone have experience loading basex via cron that could point me in the right direction. thanks /home/xxx/basex/bin/basexclient -Uxxx -Pxxx < /home/xxx/load.bxs

5 10

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 13 Feb '13

13 Feb '13

Hi Fabrice and list I am dealing with data-centric XML rather than documents and so there is a fairly high node to content ratio. I have about 250 million nodes and I find that having about 15 million nodes per database seems to work well, but this is just a guesstimate and I am really looking for some performance profiles or some heuristics so that I can limit the numbers of nodes in each database before the performance degrades. Cheers Peter > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, fetanchaud(a)questel.com, >BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Tue, 12 Feb 2013 09:07:40 +0000 > >>Dear Peter, >> >>I'm just a BaseX user, and Christian's team will correct me, but >from my experience, document size does not matter, at least for >querying. >> >>Why do you talk about distributing data ? Did you reach the 2 >billion nodes limit ? >> >>As BaseX indexes all nodes, depending on the values distribution, >creating a new collection containing hand made indices can speed up >your queries. >> >>For example, for append only collections, I'm used to creating a >index collection like this : >><index> >> <item value='value to be indexed'> >> the 'pre' pointer to the indexed element >> </tem> >> <item>... >></index> >> >>And access that 'index' something like this : >> >>for $i in >> //item[@value='searched value'] >>return >> db:open-pre('mydb', $i) >> >> >>And a big number of documents may slow down the properties window >display in the GUI, because of the document tree view. >> >> >>Question to the BaseX 's team : would 'user defined' indices be a >interesting feature ? >> >> >>Regards >> >>-----Message d'origine----- >>De : pw(a)themail.co.uk [mailto:pw@themail.co.uk] >>Envoyé : lundi 11 février 2013 17:13 >>À : Fabrice Etanchaud; pw(a)themail.co.uk; >BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : RE: [basex-talk] handling large files: is there a >streamingsolution? >> >>Thanks Fabrice, I am making good progress following your advice. Do >you have any heuristics for the best way to distribute data for >performant searches and subsetting of data? Am I better having lots >of small files or a few large files in a collection? >> >>> >>> >>> >>>---- Original Message ---- >>>From: fetanchaud(a)questel.com >>>To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >>>Subject: RE: [basex-talk] handling large files: is there a >>>streamingsolution? >>>Date: Mon, 11 Feb 2013 14:38:54 +0000 >>> >>>>Dear Peter, >>>> >>>>Did you try to create a collection with the files (CREATE command) >? >>>>You should start that way, I don't see the point in using file: >>>module for import. >>>>I think that once in the database, file size does not matter >(until >>>you reach millions of file in the collection, and do a lot of >document >>>related operations (list, etc...)) >>>> >>>> >>>> >>>>-----Message d'origine----- >>>>De : basex-talk-bounces(a)mailman.uni-konstanz.de >>>[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >>>pw(a)themail.co.uk >>>>Envoyé : lundi 11 février 2013 15:33 >>>>À : BaseX-Talk(a)mailman.uni-konstanz.de >>>>Objet : [basex-talk] handling large files: is there a streaming >>>solution? >>>> >>>>Hello List >>>>I am wanting to do a join with some large (3-400Mb) XML files and >>>would appreciate guidance on the optimal strategy. >>>>At present these files are on the filesystem and not in a database >>>> >>>>Is there any equivalent to the Zorba streaming xml:parse()? >>>> >>>>Would loading the files into a database directly be the approach, >or >>>is it better to split them into smaller files? >>>> >>>>Is the file: module a suitable route through which to import the >>>files? >>>> >>>>Thanks for your help >>>> >>>>Peter >>>> >>>>_______________________________________________ >>>>BaseX-Talk mailing list >>>>BaseX-Talk(a)mailman.uni-konstanz.de >>>>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>> >> >> >>

1 0

XQDoc for BaseX
by John Snelson 12 Feb '13

12 Feb '13

Hi all, The xquerydoc project is a 100% XQuery 1.0 project which can parse XQuery modules and extract XQDoc style documentation. I've recently extended it to be able to parse XQuery Update and XQuery Full Text, which I believe solves the majority of problems if you want to use it with modules written for BaseX. The project is available here: https://github.com/xquery/xquerydoc I'm happy to add any other syntax that BaseX supports as well, but I'll need help understanding what that is if there is anything. Enjoy your XQuery documentation! John

3 3

Re: [basex-talk] handling large files: is there a streamingsolution?
by pw＠themail.co.uk 12 Feb '13

12 Feb '13

Thanks Fabrice, I am making good progress following your advice. Do you have any heuristics for the best way to distribute data for performant searches and subsetting of data? Am I better having lots of small files or a few large files in a collection? > > > >---- Original Message ---- >From: fetanchaud(a)questel.com >To: pw(a)themail.co.uk, BaseX-Talk(a)mailman.uni-konstanz.de >Subject: RE: [basex-talk] handling large files: is there a >streamingsolution? >Date: Mon, 11 Feb 2013 14:38:54 +0000 > >>Dear Peter, >> >>Did you try to create a collection with the files (CREATE command) ? >>You should start that way, I don't see the point in using file: >module for import. >>I think that once in the database, file size does not matter (until >you reach millions of file in the collection, and do a lot of >document related operations (list, etc...)) >> >> >> >>-----Message d'origine----- >>De : basex-talk-bounces(a)mailman.uni-konstanz.de >[mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >pw(a)themail.co.uk >>Envoyé : lundi 11 février 2013 15:33 >>À : BaseX-Talk(a)mailman.uni-konstanz.de >>Objet : [basex-talk] handling large files: is there a streaming >solution? >> >>Hello List >>I am wanting to do a join with some large (3-400Mb) XML files and >would appreciate guidance on the optimal strategy. >>At present these files are on the filesystem and not in a database >> >>Is there any equivalent to the Zorba streaming xml:parse()? >> >>Would loading the files into a database directly be the approach, or >is it better to split them into smaller files? >> >>Is the file: module a suitable route through which to import the >files? >> >>Thanks for your help >> >>Peter >> >>_______________________________________________ >>BaseX-Talk mailing list >>BaseX-Talk(a)mailman.uni-konstanz.de >>https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>

2 1

handling large files: is there a streaming solution?
by pw＠themail.co.uk 11 Feb '13

11 Feb '13

Hello List I am wanting to do a join with some large (3-400Mb) XML files and would appreciate guidance on the optimal strategy. At present these files are on the filesystem and not in a database Is there any equivalent to the Zorba streaming xml:parse()? Would loading the files into a database directly be the approach, or is it better to split them into smaller files? Is the file: module a suitable route through which to import the files? Thanks for your help Peter

2 1

Load xml files to BaseX in a loop
by Nicholas OR Rita Stevens 10 Feb '13

10 Feb '13

I am new at BaseX so please pardon the newbie question. I am trying to run an xquery script to load a set of xml files in a directory into to BaseX 7.5. If I run replace.xq, shown below, it fails with Out of Main Memory declare %updating function local:upd($path,$file) { db:replace("eams",$path,$file) }; let $files:=for $file in file:list("d:\xmlfiles") return "d:\xmlfiles\"||$file for $file in $files return local:upd("pd",$file) But if I run file replace.bxs, it updates with no problem <commands> <replace path="p1">d:\xmlfiles\drr1.xml</replace> <replace path="p1">d:\xmlfiles\drr2.xml</replace> <replace path="p1">d:\xmlfiles\drr3.xml</replace> <replace path="p1">d:\xmlfiles\dw1.xml</replace> <replace path="p1">d:\xmlfiles\dlp1.xml</replace> </commands> (The same set of files is being processed by both scripts; I checked) The only idea I have is to use xquery to generate the file replace.bxs and then run it. A better way? Thanks.

2 1

Binding external variable to sequence
by Lansing, Jeff J CIV SPAWARSYSCEN-PACIFIC, 56250 09 Feb '13

09 Feb '13

How can I bind an external variable to a sequence? I see the syntax trick for escaping commas, but I still don't get a bound sequence. I tried this: import org.basex.core.*; import org.basex.core.cmd.*; import org.junit.*; public class BindingTest { String var = "$ids="; String seq = "('a',,'b',,'c')"; String query = "" + "declare variable $ids external; " + "let $count := count($ids) " + "return if($count gt 1) then element ids { $ids } " + "else error(QName('urn:err', 'not_enough_ids'), xs:string($count)) " ; /** * test binding external variable to sequence */ @Test public void test() { try { Context context = new Context(); new Set(Prop.BINDINGS, var + seq).execute(context); String result = new XQuery(query).execute(context); context.close(); System.out.println(result); } catch(Exception e) { System.err.println(e); } } } I expect to get <ids>...</ids> but instead I get the error. I'm using v7.5, Java 7u13.

2 1

Document modification timestamp
by Bob Myers 09 Feb '13

09 Feb '13

Sorry for the newbie question, but how do I obtain a modification timestamp for a document in a database? Is there any clean way to integrate with a GNU make-based make system? -- Bob

2 1

BaseX has been updated on Filecluster.com
by FileCluster Support 07 Feb '13

07 Feb '13

Dear Developer, You have received this email because we have updated BaseX and this was the contact email for the application. Your application update status BaseX has been updated on FileCluster.com to the 7.6 version. You can check out your application on our site at: http://www.filecluster.com/Business-Office/Database-Management/Download-Bas… If you want to let your users know about your software page on FileCluster.com, to rate it and review it, you can also use this banner to link it: Promote your software As a software website, it`s our duty to help the developers promote their software. Fortunatelly, our software promotions are cost free, for more informations please visit our promotion page. Submit your press releases We are also accepting press releases from software developers. All you have to do is to submit your press release and we will take care of the rest. Useful developer links All the essential links you need as a developer on FileCluster.com: Link to us - download banners or text links for linking to FileCluster.com Submit your software - Submit your software via PAD or manually Submit your press release - submit your press releases using our online form Contact Us - contact our editors Promote your software - our promotions page If you do not wish to receive any further notifications or confirmations from FileCluster.com, please send us an email and request a contact removal (email removal). Keep in mind that this means you will not receive ANY contact from us, including: software promotions, update confirmations or security alerts (antivirus and spyware test results). Regards, Renta Cezar - Senior Editor FileCluster Labs www.filecluster.com

1 0

BaseX 7.6 Update - Soft-Files.com
by Soft-Files Team 07 Feb '13

07 Feb '13

Dear Developer, Your product BaseX 7.6 has been updated. Your product review page is located at: http://www.soft-files.com/basex_72311.html. Sincerely, The Soft-Files Team

1 0