- BaseX-Talk - mailman.uni-konstanz.de

Re: [basex-talk] Fwd: Bash command
by meumapple 10 Nov '15

10 Nov '15

I can add that if I use "cat file" on the bash I get the right Greek characters, while the same command via proc:execute returns gibberish characters (the optional argument "utf-8" in proc:execute has no effect). Thanks. Joseph Il giorno 09/nov/2015, alle ore 19:04, meumapple <meumapple(a)gmail.com> ha scritto: Hi, I do not have any LANG variable set. If I redirect into a file the output of the bash command, I get a message with "utf-8" when I analyze this file with the command "file". And indeed the characters are displayed correctly. So the problem arises only when I run everything through proc:execute. If I save the result of the xquery script and analyze them with "file" I get the message "output: ERROR : line 22 : regexec error 17, (illegal byte sequence). And of course the Greek characters are messed up (but not the ASCII ones for English) Hope this helps... Joseph Il giorno 09/nov/2015, alle ore 15:18, Jens Erat <jens.erat(a)uni-konstanz.de> ha scritto: Please report the encoding of your actual files. What is the output of `env | grep LANG` (when stored in your bash script)? If you pipe the output in another file, what will `file -i [output-file]` report (`file` applies some magic to discover file formats and encoding)? > Am 09.11.2015 um 14:06 schrieb meumapple: > Hi All, > > I have been able to run the command just putting it in a bash file and then running this file with proc:execute. It works. The problem now is with the encoding of the text returned by proc:execute. In my bash I can display Greek characters, but they are messed up if returned by the proc:execute function. I also added utf-8 as a parameter for the function but nothing changes. How can I solve this? Thanks. > > Joseph > > Il giorno 09/nov/2015, alle ore 10:41, Jens Erat <jens.erat(a)uni-konstanz.de> ha scritto: > > Hi Joseph, > > As far as I understand the documentation, BaseX `proc:system` does not > use a shell interpreter by default. `<` or also the pipe `|` are > features offered by your shell, not the operating system. > > Instead of passing a single parameter, pass the command as parameter to > `/usr/bin/env sh -c`, which will make BaseX run a shell that interpretes > I/O indirection: > > proc:system('/usr/bin/env', ('sh', '-c', 'opennlp postagger > model.bin < file.txt')) > > This might be a reasonable thing to provide as a built-in function, I'm > also not sure whether one can connect some kind of stream/strings to > STDIN, STDOUT and STDERR. > > Regards from Lake Constance, Germany, > Jens > > Am 09.11.2015 um 10:19 schrieb Christian Grün: >>> Any suggestions? >> >> Any example? ;) >> Christian >> >> >> >> Thanks. >>> >>> Joseph >>> >>> >>> Inizio messaggio inoltrato: >>> >>> Da: meumapple <meumapple(a)gmail.com> >>> Data: 06 novembre 2015 20:52:08 CET >>> A: BaseX <basex-talk(a)mailman.uni-konstanz.de> >>> Oggetto: Bash command >>> >>> Hi all, >>> >>> I am using proc:system for a bash command but I cannot run it. It looks like >>> opennlp postagger model.bin < file.txt. I suspect that the problem is the >>> less-than sign, which is converted into < Is there any way to keep the >>> less-than sign and pass it to the bash properly? Thanks. >>> >>> Joseph -- Jens Erat Universität Konstanz Kommunikations-, Infomations-, Medienzentrum (KIM) Abteilung Basisdienste D-78457 Konstanz Mail: jens.erat(a)uni-konstanz.de

1 0

Re :Collection DataBase Locking
by Adi Babu 10 Nov '15

10 Nov '15

While updating the Basex collection data base if it fails because of some unexpected reason the particular database cannot be opened anymore and the message shown in loggers is "*Database ... is being updated, or update was not completed*". I have found that this is because of *upd.basex*. My question is how can I found and remove this file through code to proceed further with a safe execution of the collection database creation process. Thanks & Regards, Adi

2 1

Re: [basex-talk] Sub : Adding Special Characters to the Collection Database
by Christian Grün 10 Nov '15

10 Nov '15

Hi Madhan, I see you are using BaseX 7.9. Does the problem persist when using our latest release? What is the output of db:info() (or INFO DB)? Best, Christian On Mon, Nov 9, 2015 at 3:33 PM, madhan gopal <madhangopal828(a)gmail.com> wrote: > Dear Christian, > > Here the problem is,the xml attribute values are replacing with Alphabet > "A" > > Example: > in my Xml file i have the tag <MyTag myAttribute="10" > ..</MyTag> > if i add this xml to collection database, the value of attribute > myAttribute is replacing 10 with 'A' > > Note:Please check once attachment. > > > > On Mon, Nov 9, 2015 at 4:52 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: >> >> Hi Madhan, >> >> > I am facing the problem of adding special characters to the collection >> > Database in below cases, >> >> What do you mean by "problem", and what are "special characters"? >> Christian >> >> >> > Case-1: if the xml file size is huge(in GBs),at the time the collection >> > Database creation tacking more time, >> > Case-2: if the more no.of xml files are adding to the particular >> > collection >> > Database continuously >> > >> > Example: >> > >> > in Xml file <MyTag myAttribute="10" ...</MyTag> >> > >> > The values are adding like the following >> > in collection <MyTag myAttribute="A" ...</MyTag> >> > >> > Sample Collection adding code : >> > -------------------------------------------- >> > /* >> > This method takes the xml files list as input and process the list >> > Create the xml collection DB, adds xml to the collection if already >> > exists >> > replace it >> > */ >> > >> > public static void createCollection(ArrayList<String> listFiles){ >> > ClientSession clientSession = null; >> > String strIndexedFile=""; >> > try { >> > clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, >> > "<<USER>>", "<<PWD>>"); >> > String strDbName = "Sample_Collection"; >> > String strKeysFileLocation = >> > "D:\\TEMP\\7GB_Collection_TestingTeam\\Files"; >> > >> > String res = clientSession.execute("XQUERY db:exists('" + >> > strDbName + "')"); >> > //DB existence checking >> > if (!res.equalsIgnoreCase("true")) { >> > clientSession.execute("CREATE DB " + strDbName); >> > } else { >> > clientSession.execute("OPEN " + strDbName); >> > } >> > >> > for (int i = 0; i < listFiles.size(); i++) { >> > strIndexedFile = listFiles.get(i); >> > String tempFlag=clientSession.execute("XQUERY >> > db:exists('" + >> > strDbName + "','" + strIndexedFile + "')"); >> > clientSession.execute("ADD " + strKeysFileLocation + >> > File.separator + strIndexedFile); >> > } >> > System.out.println("InFormation:: >> > "+clientSession.execute("INFO")); >> > //optimizing the current opend DB using OPTIMIZE ALL command >> > clientSession.execute("OPTIMIZE ALL"); >> > System.out.println("InFormation:: >> > "+clientSession.execute("INFO")); >> > >> > } catch (Exception e) { >> > System.out.println("Exception files DB Updating: " + >> > e.getMessage()); >> > e.printStackTrace(); >> > } finally { >> > if (clientSession != null) { >> > try { >> > clientSession.close(); >> > } catch (IOException ex) { >> > ex.printStackTrace(); >> > } >> > } >> > } >> > } >> > >> > Hardware Specifications of Basex Sever : >> > ---------------------------------------------------------- >> > Operating System : Red Hat Enterprise Linux 64bit >> > OS Version : Linux 2.6.32 >> > Hard Disk : 500 GB >> > Memory Capacity : 16 GB Ram >> > Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 >> > Processor Speed : 2.80 GHz >> > Physical Processors : 8 >> > Virtual Processors/cores : 4 >> > Server Version : basex 7.9 >> > -- >> > Thanks & Regards >> > Madhangopal.P > > > > > -- > Thanks & Regards > Madhangopal.P

2 2

Fwd: basic JSON problem
by Florent Gallaire 10 Nov '15

10 Nov '15

Hi Christian, that's what I was afraid, http://www.w3.org/TR/xslt-30/#func-xml-to-json is just too complex to understand for me. What about your simple-json try ? to have an easy: my_request&method=json&json=format=simple Best regards Florent 2015-11-09 10:17 GMT+01:00 Christian Grün <christian.gruen(a)gmail.com>: > Hi Florent, > > When using a JSON format other than jsonml, your XML document to be > output must comply with the rules specified in the particular format > [1,2]: > > xml-to-json(<string > xmlns="http://www.w3.org/2005/xpath-functions">JSON</string>) > > ...versus... > > xml-to-json(<string>JSON</string>) > > [1] http://docs.basex.org/wiki/Json_Module > [2] http://docs.basex.org/wiki/XQuery_3.1#fn:json-to-xml > > > On Mon, Nov 9, 2015 at 4:19 AM, Florent Gallaire <fgallaire(a)gmail.com> wrote: >> I would like to use the new "basic" JSON via REST. >> With basex 8.3.1 and 8.3.2beta, >> my_request&method=json&json=format=jsonml >> works fine, but >> my_request&method=json&json=format=basic >> returns : >> [FOJS0006] Element 'article' has invalid namespace: ''. >> >> What's the problem ? >> >> Best regards >> >> Florent >> >> -- >> FLOSS Engineer & Lawyer -- FLOSS Engineer & Lawyer -- FLOSS Engineer & Lawyer

2 3

Re: [basex-talk] Sub : Adding Special Characters to the Collection Database
by Christian Grün 09 Nov '15

09 Nov '15

Hi Madhan, > I am facing the problem of adding special characters to the collection > Database in below cases, What do you mean by "problem", and what are "special characters"? Christian > Case-1: if the xml file size is huge(in GBs),at the time the collection > Database creation tacking more time, > Case-2: if the more no.of xml files are adding to the particular collection > Database continuously > > Example: > > in Xml file <MyTag myAttribute="10" ...</MyTag> > > The values are adding like the following > in collection <MyTag myAttribute="A" ...</MyTag> > > Sample Collection adding code : > -------------------------------------------- > /* > This method takes the xml files list as input and process the list > Create the xml collection DB, adds xml to the collection if already exists > replace it > */ > > public static void createCollection(ArrayList<String> listFiles){ > ClientSession clientSession = null; > String strIndexedFile=""; > try { > clientSession = new ClientSession("<<IP_ADDRESS>>", 1984, > "<<USER>>", "<<PWD>>"); > String strDbName = "Sample_Collection"; > String strKeysFileLocation = > "D:\\TEMP\\7GB_Collection_TestingTeam\\Files"; > > String res = clientSession.execute("XQUERY db:exists('" + > strDbName + "')"); > //DB existence checking > if (!res.equalsIgnoreCase("true")) { > clientSession.execute("CREATE DB " + strDbName); > } else { > clientSession.execute("OPEN " + strDbName); > } > > for (int i = 0; i < listFiles.size(); i++) { > strIndexedFile = listFiles.get(i); > String tempFlag=clientSession.execute("XQUERY db:exists('" + > strDbName + "','" + strIndexedFile + "')"); > clientSession.execute("ADD " + strKeysFileLocation + > File.separator + strIndexedFile); > } > System.out.println("InFormation:: > "+clientSession.execute("INFO")); > //optimizing the current opend DB using OPTIMIZE ALL command > clientSession.execute("OPTIMIZE ALL"); > System.out.println("InFormation:: > "+clientSession.execute("INFO")); > > } catch (Exception e) { > System.out.println("Exception files DB Updating: " + > e.getMessage()); > e.printStackTrace(); > } finally { > if (clientSession != null) { > try { > clientSession.close(); > } catch (IOException ex) { > ex.printStackTrace(); > } > } > } > } > > Hardware Specifications of Basex Sever : > ---------------------------------------------------------- > Operating System : Red Hat Enterprise Linux 64bit > OS Version : Linux 2.6.32 > Hard Disk : 500 GB > Memory Capacity : 16 GB Ram > Processor Family : Intel (R) Xeon(R) CPU E5 -2680 V2 > Processor Speed : 2.80 GHz > Physical Processors : 8 > Virtual Processors/cores : 4 > Server Version : basex 7.9 > -- > Thanks & Regards > Madhangopal.P

1 0

basic JSON problem
by Florent Gallaire 09 Nov '15

09 Nov '15

I would like to use the new "basic" JSON via REST. With basex 8.3.1 and 8.3.2beta, my_request&method=json&json=format=jsonml works fine, but my_request&method=json&json=format=basic returns : [FOJS0006] Element 'article' has invalid namespace: ''. What's the problem ? Best regards Florent -- FLOSS Engineer & Lawyer

2 1

Keeping a database optimized
by Paul Swennenhuis 05 Nov '15

05 Nov '15

What is the best way to keep the indexes of a database up-to-date? I have created a database with the option UPDINDEX set to ON (and db-info tells me that it is indeed switched on) and I have also created text and attribute indexes. Even so, after several insertions (using XQUERY insert node) I see that the UPTODATE flag is no longer checked and I have to run OPTIMIZE ALL to get back my performance. a) Is there a way to keep UPTODATE checked under all circumstances? And if not b) Is is advisable to run OPTIMIZE ALL after every insertion? (I guess not)

2 4

PHP client update
by Paul Swennenhuis 04 Nov '15

04 Nov '15

Hi all, I wrote an update for the BaseX PHP client, to handle the Digest authentication method as introduced in BaseX 8.0 See attachment. Regards, Paul

1 0

request concerning unittest in the GUI
by Rob Stapper 02 Nov '15

02 Nov '15

Hi Christian, For debugging my xquery-code I using the test-facilities from the unit-module in combination with trace and prof:dump command in the code for code-tracing. The problem is that when a unit-test fails it overwrites the trace-output in the Query-info window. So the code-tracing-info is gone just when I need it. It would make debugging a lot easier if the trace-output is appended after the unit-test-output in the Info-window of the GUI. Best, Rob --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 5

Best way to batch up a set of DB creates, updates and queries?
by Hondros, Constantine (ELS-AMS) 01 Nov '15

01 Nov '15

I'm looking for the best way to batch up a series of updating operations into a single pipeline that can be run at once. To give you an idea, we have to create multiple databases, perform a number of updating post-processes on each, before finally pulling certain fields out. I've got a lot of parameters that need to externalised somehow - database creation options, location of DTDs, XSDs, Catalogs, and unless I am trying this the wrong way I am finding it quite difficult. I can't write it all XQuery - I have a mixture of updating and non-updating functions. I can't get much traction out of the BXS scripting syntax - for example, I can't seem to create and use variables using this. What do other people out there use to glue these sorts of operations together? TIA, Constantine ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 5