January 2015 - BaseX-Talk - mailman.uni-konstanz.de

Linux lessons
by Wendell Piez 08 Jan '15

08 Jan '15

Dear BaseX people, I am a hopeless newbie when it comes to Linux. Seriously, my ignorance is astonishing. I am now running Linux Mint. It's a huge amount of fun but I'm left guessing about many simply things. I am running the current BaseX 7.9. I am using it to provide a pipeline architecture calling lots of XSLT2, producing HTML and SVG. It works well for this. To my RESTXQPATH I wish to append to - but not override (if possible) - a path. By editing my .basex configuration I have managed to reset it, but not simply to append to it. That is, RESTXQPATH = /home/wendell/basex/webapp:/home/wendell/XQuery-project/restxq With this, the server skips the basex/webapp directory. If I remove the second segment and restart, it goes live again. What is the simplest way for me to set up my BaseX so that I can maintain my project xqm files (including a project's RestXQ), in the project, but keep BaseX otherwise (including its data directories) separate? Over the longer term I can see several such directories where I'd like BaseX to look. On another operating system I used to start up using different shell scripts; is that the best way here (and if so does my shell script have to call Java or can I use the scripts in the basex/bin and pass in a value somehow)? In general, what recommendations can you offer for no-fuss configuration of RestXQ for separate projects? Thanks! Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

2 2

BaseX 8.0 snapshot on OpenShift
by Andy Bunce 08 Jan '15

08 Jan '15

Hi, I am having difficulty getting this to work. I had something working with 7.9 (see https://github.com/Quodatum/openshift-basex-quick-start/tree/releases/0.2 ) Using a recent snapshot. I am running ${OPENSHIFT_DATA_DIR}/basex/bin/basexhttp -n ${OPENSHIFT_DIY_IP} -h ${OPENSHIFT_DIY_PORT} -p ${PORT_DATABASE} -e ${PORT_EVENT} -s ${PORT_STOP} -U ${BASEX_USER} -P ${BASEX_PASSWORD} The debug o/p is STOPPORT: 15007 PASSWORD: admin USER: admin EVENTPORT: 15006 DEBUG: true PORT: 15005 SERVERPORT: 15005 HOST: 127.13.28.129 STOPPORT: 15007 PASSWORD: admin USER: admin EVENTPORT: 15006 DEBUG: true PORT: 15005 SERVERPORT: 15005 HOST: 127.13.28.129 WEBPATH: /var/lib/openshift/549d9d794382ec39b8000105/app-root/data/basex/webapp HTTP Server was started (port: 8080) java.net.BindException: Permission denied at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) at java.net.ServerSocket.bind(ServerSocket.java:376) at java.net.ServerSocket.bind(ServerSocket.java:330) at org.basex.BaseXHTTP$StopServer.<init>(BaseXHTTP.java:403) at org.basex.BaseXHTTP.<init>(BaseXHTTP.java:139) at org.basex.BaseXHTTP.main(BaseXHTTP.java:50) Server is running or permission was denied. https://github.com/BaseXdb/basex/blob/0af4ead914d3d1946e5a8ce9672811a34f933… I wonder if SERVERHOST should be set here? As with SERVER PORT.. https://github.com/BaseXdb/basex/blob/0af4ead914d3d1946e5a8ce9672811a34f933… /Andy

2 2

BaseX Concurrency
by Bondeson, Carl 07 Jan '15

07 Jan '15

I have a multithreaded web application running under JBoss 6.2. It seems that 2 threads are attempting access concurrently. Any suggestions. 01/07/15 10:07:14 WARN [server ]: Error creating connection to BaseX server - attempt 0 java.io.IOException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_71 OS: Windows Server 2008 R2, amd64 Stack Trace: java.nio.channels.OverlappingFileLockException sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1056) org.basex.io.random.TableDiskAccess.lck(TableDiskAccess.java:183) org.basex.io.random.TableDiskAccess.sharedLock(TableDiskAccess.java:170) org.basex.io.random.TableDiskAccess.<init>(TableDiskAccess.java:83) org.basex.data.DiskData.init(DiskData.java:122) org.basex.data.DiskData.<init>(DiskData.java:92) org.basex.core.cmd.Open.open(Open.java:70) org.basex.core.cmd.Open.run(Open.java:36) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.run(Command.java:117) org.basex.core.cmd.Check.run(Check.java:38) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145) at com.pilotfish.eip.basex.BaseXClient.execute(BaseXClient.java:79) at com.pilotfish.eip.basex.BaseXClient.execute(BaseXClient.java:91) at com.pilotfish.eip.basex.txlog.BaseXSubmitter.createBaseXClient(BaseXSubmitter.java:54) at com.pilotfish.eip.basex.txlog.BaseXSubmitter.getBaseXClient(BaseXSubmitter.java:45) at com.pilotfish.eip.basex.txlog.BaseXSubmitter.execute(BaseXSubmitter.java:111) at com.pilotfish.eip.rest.impl.TransactionLogServiceImpl.constructHealthComponentSettings(TransactionLogServiceImpl.java:726) at com.pilotfish.eip.rest.impl.TransactionLogServiceImpl.setHealthComponentConfiguration(TransactionLogServiceImpl.java:696) at com.pilotfish.eip.rest.impl.TransactionLogServiceImpl.setHealthComponentConfiguration(TransactionLogServiceImpl.java:700) at com.pilotfish.eip.rest.impl.TransactionLogServiceImpl.setHealthComponentConfiguration(TransactionLogServiceImpl.java:700) at com.pilotfish.eip.rest.impl.TransactionLogServiceImpl.getHealthComponentConfiguration(TransactionLogServiceImpl.java:607) at com.pilotfish.eip.rest.services.TransactionLogResource.getHealthComponentConfiguration(TransactionLogResource.java:247) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:353) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:343) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:318) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:983) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:359) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:372) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:218) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:926) at java.lang.Thread.run(Thread.java:745) Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson(a)ct.gov<mailto:carl.bondeson@ct.gov> [cid:image001.jpg@01D02A62.E91E67F0]<http://www.ct.gov/dph>

2 2

Feature request: RNG and Schematron validation
by Glatza, Philipp, le-tex 07 Jan '15

07 Jan '15

Just a short question: are there any plans to (not) support Schematron and/or RNG/RNC validation in the validation module? Philipp -- Philipp Glatza Content Engineering le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 128, Fax +49 341 355356 528 philipp.glatza(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard Vöckler

3 3

Xquery update question
by Erol Akarsu 06 Jan '15

06 Jan '15

Hi, I have several xml document index keeping products features and have one master index file of products.. Currently, we have about 25 million products that have size about 25G xquery index size. My xquery application is reading features data from auxiliary xml document and enhancing master document. In order to do it, I am inserting data into master document with a generic insert command: insert node <FEATURES_NODE> into <PRODUCT_NODE> But xquery update is executing update process after all insert requests collected. But with huge size of my index, the memory of machine (35G) is exhausted easily and can NOT finish process. Is there way to execute individual xquery update requests one by one so that process can finish? Erol Akarsu

2 2

Fetching a binary document from BaseX with C# using the server API
by Boxstart, Bjorn 06 Jan '15

06 Jan '15

Hello, I'm currently working on the integration of BaseX in part of our .Net based software. One part of the functionality will be to store and retrieve binary files. I haven't been able to retrieve the raw files from the server yet using C#. I based my solution on the example of the C# client in the source code repository of BaseX. With my current solution I receive only the first few (3 to 7) bytes of the file (at least I think they are...). The code I have at the moment is: /// <summary> /// Retrieves a RAW document from the database. The database must be opened before executing this command. /// </summary> /// <param name="path">The path to the binary data to be retrieved</param> /// <returns>A byte array with the binary data</returns> public byte[] Retrieve(string path) { //stream.WriteByte((byte)BaseXCommand.QUERY); Send("RETRIEVE " + path); //Send(path); using (MemoryStream ms = new MemoryStream()) { Receive(ms); info = Receive(); if (!Ok()) { throw new IOException(info); } return ms.ToArray(); } } Can anybody please tell me what the problem with this solution can be or give me a hint for a correct way to retrieve the raw bytes? I know I can use the files on disk directly, but would rather not go that way. Thanks in advance for your reaction! Kind regards, Björn Boxstart

2 2

Question / Suggestion : Template mechanisms and operator overloading for XQUERY 3.++ ?
by jean-marc Mercier 06 Jan '15

06 Jan '15

Hello, I was wondering if it is possible to improve template programming and operator overloading with XQUERY 3.? 1) template programming : XQUERY provides a quite natural template mechanism using type switch and instance: declare function yell($animal) { if ($animal instance of element()) then return typeswitch($animal) case element(cow) return "Moo" default return "..." else "this is not an animal" }; However, I wonder whether it could be possible to provide such a mechanism directly at the interpreter level ? For instance, the following code seems coherent and concise : declare function yell($animal as element(cow) ) {"Moo"} declare function yell($animal as element() ) {".."} declare function yell($animal ) {"This is not an animal"} As far as I understand, XQUERY functions identification seems to be achieve through a couple (name, arity). Would it be difficult to extend it with (name,signature or predicate) ? 2) Operator overloading (quite linked to the previous question) I don't know how to overload operators with XQUERY. For instance I would like to define something like declare function array:operator+($left,$right){array:for -each-pair($left,$right,function($a + $b) { $a + $b } )} Is it possible ? I know that I can survive writing declare function array:plus($left,$right){array:for-each-pair($left,$right, function($a + $b) { $a + $b } )} but when one start to write algorithmic code, it is far easier and understandable to write $a+$b than array:plus($a,$b)

2 1

copy-namespaces declaration
by Alex R. Loayza 06 Jan '15

06 Jan '15

Hello, I've found this same issue on version 7.9: https://mailman.uni-konstanz.de/pipermail/basex-talk/2012-May/003294.html Is there now some fix I'm not aware of? Thank you. Alex

2 1

Out Of Memory
by Mansi Sheth 06 Jan '15

06 Jan '15

Hello, I have a use case, where I have to extract lots in information from each XML in each DB. Something like, attribute values of most of the nodes in an XML. For such, queries based goes Out Of Memory with below exception. I am giving it ~12GB of RAM on i7 processor. Well I can't complain here since I am most definitely asking for loads of data, but is there any way I can get these kinds of data successfully ? mansi-veracode:BigData mansiadmin$ ~/Downloads/basex/bin/basexhttp BaseX 8.0 beta b45c1e2 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) Exception in thread "qtp2068921630-18" java.lang.OutOfMemoryError: Java heap space at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1857) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2073) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:744) -- - Mansi

5 17

Python client and BaseX 8.0
by Marc van Grootel 06 Jan '15

06 Jan '15

Hi, Anyone using the python client. I used it before, I think it was with 7.9, where it was still working. But now on a recent snapshot I cannot get my example to work anymore. I will report to maintainer but I would like to know if anyone could confirm having difficulties too. Anything changed in the protocol? I'm running on Python 2.7.8 :: Anaconda 2.0.1 (x86_64), on Mac that is. The error message occurs when trying to set up a session: /Users/marcvangrootel/Dropbox/notebooks/BaseXClient.py in __init__(self, host, port, user, password, receive_bytes_encoding, send_bytes_encoding) 129 # evaluate success flag 130 if not self.server_response_success(): --> 131 raise IOError('Access Denied.') 132 133 def execute(self, com): IOError: Access Denied. Thanks, --Marc

4 9