- BaseX-Talk - mailman.uni-konstanz.de

multiple inserts/update/deletes in single transaction
by Wang, Genneva 11 Aug '16

11 Aug '16

Hi I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section: RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction. I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback. I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI. OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added. Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ? Thank you very much for your help.

4 17

BaseX - Parse out to ORM and/or pandas json
by Sayth Renshaw 08 Aug '16

08 Aug '16

Hi I am readimg the docs and can use xpath and somewhat xquery, what i can't envision is the workfow one i have data established in xbase. The parsers in seem logical http://docs.basex.org/wiki/Parsers . So thinking i have data in my basex db and i want to explore it and analyze it in pandas. Pandas accepts a wide range of formats including xml. So i create my xpath query and i see the data i want how should i create an output say json as i could later use that as a data bag in a static website. Is this easy to keep and be repeatable? I asked about ORM because i use django orm is it to create a data feed so that an orm style typical setup can stay updated from basex? Sorry for the silly conceptual questions, cant get the lay of the land. Sayth

2 2

Troubles moving queries from client side to .xq files on the server side using REST.
by Mike Engledew 07 Aug '16

07 Aug '16

I have some running code that used the "propitiatory" basex communication method over TCP/IP. The application was calling basex "client" side (from an Ant task) so this was all fine. Being used to the SQL paradigm of "preparing" an SQL statement then executing it with different "host variable" values which gives a massive performance advantage over constantly preparing an SQL statement with different literal values plugged in, I chose the same approach for my original XQuery application. String partDelete = "declare namespace xyz=\"" + BaseXML.NAMESPACE_URL + "\";\n" + "declare variable $contributor external;\n" + "delete node /xyz:driver[@name=\"" + driver.getDriverName() + "\"]/xyz:part[@contributor=$contributor]"; This statement is executed multiple times from within a loop to delete many "parts", before each execution I perform :- pd.bind("$contributor", contributor); pd.execute() Now we are trying to move the code from being client side to server side, the server is using RESTful interfaces for all external interactions so we are migrating the BaseX interaction to REST too. I reasoned that if I created a partdelete.xq file on the server that I could then perform a rest request passing in the values of driver and contributor emulating the "host variable" approach. I created partdelete.xq module namespace xyz = 'http://my.basex.server.net/xyz'; declare variable $driver external; declare variable $contributor external; delete node /zss:driver[@name=\"$driver"]/zss:part[@contributor=$contributor] Then (using postman) performed a GET request on //http:my.basex.server.ne:8984/rest/ TESTDB?run=partdelete.xq&$driver=TESTDRVR&$contributor=LINUX During my experiments I have managed to get many different error messages, but never got even close to making this work. Questions: o Am I fooling myself that creating a server side .xq file is going to perform better than building a completed query on the client and sending that via REST without "variables"? o Is there a way to perform the equivalent "prepare/execute" method from the client without creating a .xq file on the server? o Can anyone suggest how I can get the server side .xq file to work? I also have a query which is executed before the delete to obtain the list of parts to delete. I was unable to make this work either. The .xq file looked like this module namespace xyz = 'http://my.basex.server.net/xyz'; declare variable $driver external; declare variable $contributor external; for $p in /xyz:driver[@name="$driver"]/xyz:part, $o in $p/xyz:part-outputs/xyz:part-output where $p/@contributor/data() = $contributor return data($o/@type)||"/"||data($p/@class)||"/"||data($p/@hostname) The query works fine from the client side, and since this is only executed once it's no big deal to prefill all the values and dispense with the variables and send it over REST. I could (I suppose) alternatively use the <variable element as indicated in the BaseX documentation, but it's unclear how that would offer any performance advantage. Being a lover of consistency I would like to move both queries over to the BaseX server side. Examples of some of the errors I managed to produce ... [XPST0081] No namespace declared for 'xyz:driver' [XPST0003] Library modules cannot be evaluated [XPDY0002] root(): no context value bound. Any help, advice or suggestions greatly appreciated. Mike

2 3

authentication with REST API
by Claudio Cacciari 04 Aug '16

04 Aug '16

Dear All, I'm working with a BaseX instance ver 8.5.1, deployed on a tomcat 7 as war package, on a centos 7.2. I'm trying to understand how to configure the REST API authentication. Apparently, using the REST API, the authentication is required for create/write operations, but not for reading. I mean that I can download an XML resource stored in the DB, just putting the path in the browser without performing any authentication. Would it be possible to configure it in order to put even the GET calls under authentication? Is my configuration wrong? I have added a username/password in the file web.xml: <web-app> [...] <context-param> <param-name>org.basex.user</param-name> <param-value>myuser</param-value> </context-param> <context-param> <param-name>org.basex.password</param-name> <param-value>mypassword</param-value> </context-param> <context-param> <param-name>org.basex.authmethod</param-name> <param-value>Basic</param-value> </context-param> [...] <servlet> <servlet-name>REST</servlet-name> <servlet-class>org.basex.http.rest.RESTServlet</servlet-class>  <init-param> <param-name>org.basex.user</param-name> <param-value>myuser</param-value> </init-param> <init-param> <param-name>org.basex.password</param-name> <param-value>mypassword</param-value> </init-param> </servlet> [...] which results in a user.xml file like this: $cat /usr/share/tomcat/webapps/BaseX851/data/users.xml <users> <user name="admin" permission="admin"> <password algorithm="digest"> <hash>8c6f257a751d87d1eae5ab59710a818a</hash> </password> <password algorithm="salted-sha256"> <salt>3631994636103909</salt> <hash>9d25a8e3dc227e54f098c3cf3b8d1ca92b76a3bebb62e0f04299beaaf443908d</hash> </password> </user> <user name="myuser" permission="create"> <password algorithm="digest"> <hash>d3adbad8e6540a70a8b958ab27eec9b8</hash> </password> <password algorithm="salted-sha256"> <salt>3629615605296334</salt> <hash>3b64152088b90a2944b07a3d63703cdb8bffd1ccc21e5a40fe2e1f2a19307647</hash> </password> </user> </users> Thanks, Claudio

2 2

BaseX performance 7.x vs 8.x
by Bondeson, Carl 04 Aug '16

04 Aug '16

I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines? Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson(a)ct.gov<mailto:carl.bondeson@ct.gov> [logosmall]

4 10

Inconsistency of regular expression support in BaseX and GUI
by Kristian Kankainen 03 Aug '16

03 Aug '16

Hello! Today I noticed that I can successfully use the following regular expression inside XQuery but the same regexp fails when using it in the GUI's searchboxes (the ones appearing when pressing Ctrl+F) when enabling regular expression search. This is the regular expression and probably it is the block escape that is causing the error: (¹|²|³|\p{IsSuperscriptsandSubscripts}) The error message says: "Regular Expression: java.util.regex.PatternSyntaxException: Unknown character property name {issuperscriptsandsubscripts} near index 37" Probably the lowercaseing of the block escape name is causing the error and hopefully it can be fixed without too complex logics. Best regards Kristian K

2 1

Error report in baseX-dev
by Bowers, Jack 03 Aug '16

03 Aug '16

Hi, I'm trying to delete some files that I just added to my db To give context, I originally added them via Oxygen data source explorer baseXdev, but I am trying to delete them in the dba https://basex-dev.eos.arz.oeaw.ac.at/dba/database?name=dboe-xml and the error 500 message below is what occurs. The uploading process originally stopped and gave an error message and listed about 4 files that would not be included in upload but the rest appeared but now nothing else can be added or deleted. Best Jack Bowers Austrian Centre for Digital Humanities (ACDH) Österreichische Akademie der Wissenschaften (ÖAW) - Austrian Academy of Sciences http://www.oeaw.ac.at/acdh/ - http://wboe.oeaw.ac.at/ -----------------(SEE ERROR MSG BELOW)------------------- HTTP ERROR 500 Problem accessing /dba/delete. Reason: Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.3.1 Java: Oracle Corporation, 1.8.0_91 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: dboe-xml: could not create lock file. at org.basex.util.Util.notExpected(Util.java:60) at org.basex.data.DiskData.startUpdate(DiskData.java:218) at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) at org.basex.query.up.Updates.apply(Updates.java:134) at org.basex.query.QueryContext.iter(QueryContext.java:369) at org.basex.http.restxq.RestXqResponse.create(RestXqResponse.java:55) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:101) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745)

2 1

users.xml change outside basex
by Maximilian Gärber 02 Aug '16

02 Aug '16

Hi, it seems all of users.xml will be rewritten on changes. I just tried to add a <groups> element next to <users>. Any way to preserve changes in the users.xml ? Maybe allow for a <custom> section or similar? By the way: what was the reason users.xml is not stored as a basex database but on the file system? Regards, Max

2 3

db:optimize questions
by Baysdon, Joseph 02 Aug '16

02 Aug '16

I a new baseX user and I have a few questions regarding the db:optimize command. Does 'optimize all' perform all available optimizations or are there other parameters (indexing or full-text as mentioned at http://docs.basex.org/wiki/Database_Module#db:optimize) to supply? Is read/write possible to a database undergoing optimization? I’m guessing no because I get an error (“Database ‘foo’ is currently opened by another process”) whenever another connection to the database is open. On what size or content of databases does optimization have a significant impact? Currently on a 400Mb database, I don't see any performance benefits after optimization. (Note: all of my performance testing is using a simple db:list command.) Thanks in advance for any assistance you can provide. Joe

2 7

Create new users
by dali dali 02 Aug '16

02 Aug '16

Hi, I am using the DBA interface. I created a new user with the permission "create". How can I log in as the new user (not administrator)?

2 1