BaseX-Talk

basex-talk@mailman.uni-konstanz.de

9 participants
5241 discussions

Does BaseX support schema-element()?
by Malix 14 May '10

14 May '10

Hi, I am looking for a native XML DB supporting schema aware functions like let $c1 := doc("s1.xml")/s1:n1 $c1 instance of element(s1:n1,xs:string) s1.xml: <s1:n1 xmlns:s1="foo:bar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="foo:bar test1.xsd">hello</s1:n1> s1.xsd: <?xml version="1.0" encoding="UTF-8"?> <schema targetNamespace="foo:bar" elementFormDefault="qualified" xmlns="http://www.w3.org/2001/XMLSchema"> <element name="n1" type="QName"/> </schema> Does BaseX support these schema aware functions? Thanks and best regards Malix

1 0

REST api
by Robby Pelssers 12 May '10

12 May '10

Hi all, I took a go at implementing my own api for inserting/deleting/querying through the REST API. For the ones interested here is my code which is also available at http://robbypelssers.blogspot.com/2010/05/exist-db-how-to-use-rest.html If you find errors or have remarks how-to improve.... drop me a mail. Cheers, Robby Pelssers ---------------------------------------------------------------------------------------- public interface RestCommand { void setConnection(HttpURLConnection conn); HttpURLConnection getConnection(); void setInputStream(InputStream is); InputStream getInputStream(); void execute() throws IOException; String getRequestMethod(); String getURI(); void onSuccess(); void onFailure(Exception e); } ---------------------------------------------------------------------------------------- public abstract class AbstractRestCommand implements RestCommand { private HttpURLConnection connection; private InputStream inputStream; private String URI; public AbstractRestCommand(String URI) { this.URI = URI; } public HttpURLConnection getConnection() { return this.connection; } public InputStream getInputStream() { return this.inputStream; } public void setConnection(HttpURLConnection conn) { this.connection = conn; } public void setInputStream(InputStream in) { this.inputStream = in; } public String getURI() { return this.URI; } public void onSuccess() {} public void onFailure(Exception e) {} } ---------------------------------------------------------------------------------------- public class PutCommand extends AbstractRestCommand { public PutCommand(String URI) { super(URI); } public void execute() throws IOException { // Set an output connection. getConnection().setDoOutput(true); OutputStream out = getConnection().getOutputStream(); for (int c = getInputStream().read(); c != -1; c = getInputStream().read()) { out.write(c); } out.flush(); getInputStream().close(); out.close(); int responseCode = getConnection().getResponseCode(); if(responseCode == HttpURLConnection.HTTP_CREATED) { onSuccess(); } else { onFailure(new InvalidResponseCodeException(responseCode)); } } public final String getRequestMethod() { return "PUT"; } } ---------------------------------------------------------------------------------------- public class PostCommand extends AbstractRestCommand { public PostCommand(String URI) { super(URI); } public void execute() throws IOException { getConnection().setRequestProperty("Content-Type", "application/query+xml"); // Set an output connection. getConnection().setDoOutput(true); OutputStream out = getConnection().getOutputStream(); for (int c = getInputStream().read(); c != -1; c = getInputStream().read()) { out.write(c); } out.flush(); getInputStream().close(); out.close(); int responseCode = getConnection().getResponseCode(); if(responseCode == HttpURLConnection.HTTP_OK) { onSuccess(); } else { onFailure(new InvalidResponseCodeException(responseCode)); } } public final String getRequestMethod() { return "POST"; } } ---------------------------------------------------------------------------------------- public class DeleteCommand extends AbstractRestCommand { public DeleteCommand(String URI) { super(URI); } public final String getRequestMethod() { return "DELETE"; } public void execute() throws IOException { int responseCode = getConnection().getResponseCode(); if(responseCode == HttpURLConnection.HTTP_OK) { onSuccess(); } else { onFailure(new InvalidResponseCodeException(responseCode)); } } } ---------------------------------------------------------------------------------------- public class XmlDBAuthenticator extends Authenticator { private String username; private String password; public XmlDBAuthenticator(String username, String password) { this.username = username; this.password = password; } protected PasswordAuthentication getPasswordAuthentication() { return new PasswordAuthentication(username, password.toCharArray()); } } ---------------------------------------------------------------------------------------- public interface XmlDBClient { Authenticator getAuthenticator(); /** * @param command: PutCommand * @param path: the absolute path of the resource to insert * @throws FileNotFoundException */ void put(PutCommand command, String path) throws FileNotFoundException; /** * @param command: PutCommand * @param is: input stream of the resource to insert */ void put(PutCommand command, InputStream is); /** * @param command: DeleteCommand */ void delete(DeleteCommand command); /** * @param command: PostCommand * @param path: the absolute path of the xquery to be executed * @throws FileNotFoundException */ void post(PostCommand command, String path) throws FileNotFoundException; /** * @param command: PostCommand * @param is: input stream of the xquery to be executed */ void post(PostCommand command, InputStream is); } ---------------------------------------------------------------------------------------- public class XmlDBClientImpl implements XmlDBClient { private Authenticator authenticator; public XmlDBClientImpl(Authenticator authenticator) { this.authenticator = authenticator; } public Authenticator getAuthenticator() { return this.authenticator; } protected void executeCommand(RestCommand command, InputStream is) { URL url = null; HttpURLConnection conn = null; try { Authenticator.setDefault(getAuthenticator()); url = new URL(command.getURI()); conn = (HttpURLConnection) url.openConnection(); conn.setRequestMethod(command.getRequestMethod()); command.setConnection(conn); command.setInputStream(is); command.execute(); } catch (IOException e) { command.onFailure(e); } finally { // Close connection. if (conn != null) { conn.disconnect(); } } } public void put(PutCommand command, InputStream is) { executeCommand(command, is); } public void put(PutCommand command, String path) throws FileNotFoundException { put(command, new FileInputStream(path)); } public void delete(DeleteCommand command) { executeCommand(command, null); } public void post(PostCommand command, String path) throws FileNotFoundException { post(command, new FileInputStream(path)); } public void post(PostCommand command, InputStream is) { executeCommand(command, is); } } ---------------------------------------------------------------------------------------- public class XmlDBClientTest { private XmlDBClient client; @Before public void setup() { client = new XmlDBClientImpl(new XmlDBAuthenticator("robby", "pelssers")); } @After public void tearDown() { client = null; } @Test public void testInsertDocument() throws FileNotFoundException { String URI = "http://localhost:8080/exist/rest/db/released/products/TESTPRODUCT.xml"; String path = "D:/nxp/repository/released/products/TESTPRODUCT.xml"; client.put( new PutCommand(URI) { @Override public void onSuccess() { System.out.println("TESTPRODUCT.xml was succesfully inserted"); } @Override public void onFailure(Exception e) { System.out.println("TESTPRODUCT.xml was NOT succesfully inserted: " + e.getMessage()); } }, path ); } @Test public void testDeleteDocument() { String URI = "http://localhost:8080/exist/rest/db/released/products/TESTPRODUCT.xml"; client.delete( new DeleteCommand(URI) { @Override public void onSuccess() { System.out.println("TESTPRODUCT.xml was succesfully deleted"); } @Override public void onFailure(Exception e) { System.out.println("TESTPRODUCT.xml was NOT succesfully deleted: " + e.getMessage()); } } ); } @Test public void testQueryProductsByAuthor() { String queryURI = "http://localhost:8080/exist/rest/db"; client.post( new PostCommand (queryURI) { @Override public void onSuccess() { System.out.println("Query was executed succesfully"); try { BufferedReader br = new BufferedReader(new InputStreamReader(getConnection().getInputStream(), "UTF-8")); // Print all lines of the result. String line; while((line = br.readLine()) != null) { System.out.println(line); } br.close(); } catch (Exception e) { e.printStackTrace(); } } @Override public void onFailure(Exception e) { System.out.println("Query was NOT executed succesfully: " + e.getMessage()); } }, getClass().getClassLoader().getResourceAsStream("productsByAuthor.xquery") ); } } ----------------------------------------------------------------------------------------

2 1

Font size increasing on '}' input
by xdupont＠laposte.net 11 May '10

11 May '10

Hi, I really enjoy using your software. It's very cool. But each time I type a closing brace, the font size increases. Am I the only one experiencing this behaviour ? I'm experiencing this on Windows 7.

3 2

fulltext queries that span elements / reducing result sets to most specific matches
by Imsieke, Gerrit, le-tex 10 May '10

10 May '10

Dear Team, I found that fulltext queries that do not retrieve text nodes are significantly slower than querying text nodes as in text()[. contains text 'sometext'] Example: There's some sample data that may contain both CO2 and CO2. I want a query that returns both kinds of results. With my sample data of approx. 2700 documents, 70 MB, 2.6M nodes, height 18, a query for text()[. contains text 'CO'] takes approx. 5 seconds and returns approx. 900 results. A query for *[. contains text 'CO'] takes 27 seconds (and returns 1600 results). Now *[. contains text 'CO2'] also takes 27 seconds and returns 1300 results. Since the subscript is properly marked up in the sample documents, text()[. contains text 'CO2'] returns 0 results. Can you think of a way that will speed up fulltext queries to element content, i.e., the *[. contains text '...'] scenario? -------- A related addition (thereby violating the "one topic, one mailing list posting" dogma slightly): Given a document <doc><sect>CO2</sect></doc>, the query *[. contains text 'CO2'] will contain 3 results: doc, sect, and p. But I'm only interested in the "shortest possible" or "most specific" result, i.e., a result that does not contain any other element satisfying the same query. In above example, the shortest possible result will be p. This may be achieved by the following query: let $prelim := //*[. contains text 'CO2' ] for $d in $prelim[every $c in * satisfies empty($c intersect $prelim)] return <result>{ $d }</result> The predicate [every $c in * satisfies empty($c intersect $prelim)] doesn't slow down the query significantly, at least with BaseX operating on my sample data. Thus it's important to optimize the initial //*[. contains text 'CO2' ] query. Would it help if BaseX knew that we only need the innermost element satisfying the FT query? So if there were, for example, an option //*[. contains text 'CO2' using option basex:element-scope "most-specific"], could BaseX use this in order to make use of its indexes and accelerate the query? I'm not sure, because if some text occurs within a certain element way down the tree, a distinct occurence of same text may be found higher up in the tree, immediately below an ancestor element: <a>text foo <c>text</c> bar</a> So it's important to look at all the elements in the tree, with no speed gain if you look at the bottom-most elements first. So I think it boils down to the issue of accelerating fulltext queries that span element boundaries. ------ You could argue that if I knew in advance which elements to search and use specific elements instead of *, optimization will be easier (so do the qizx people, if I remember correctly). But there are different document types in the DB (other types may be added at any time without prior notice), so it's hard to specify in advance which elements should be queried/returned. And even if you focused your query on, e.g., para elements in DocBook documents, there are situations like <para><orderedlist><listitem><para>CO<subscript>2</subscript></para></listitem></orderedlist></para> where you cannot restrict a query to paras because you will get the occuring text twice, in the absence of additional filtering. Or even worse with CALS tables in DocBook: an entry may or may not contain a para. Searching for paras that contain some text will return (1) the para that contains the whole table plus (2) a para within this table ONLY IF the author chose to mark up the inner para (which he doesn't have to). Filtering the results or using the fictitious "most-specific" scope option, the result will be narrowed down to either the entry or, if present, the para within the entry. You don't need to filter *[self::para[not(descendant::tgroup or descendant::orderedlist or ...)] or self::entry or self::simpara or ...][. contains text 'CO2'] wihch might ultimately work for DocBook but not for the general case (in our environment, there will be DocBook, XHTML, dozens of customer-specific schema variants and an unpredictable number of 3rd-party schemas). ------ This message has become quite lengthy. I hope you don't mind. Gerrit -- Gerrit Imsieke Geschäftsführer / Managing Director le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 110, Fax +49 341 355356 510 gerrit.imsieke(a)le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Geschäftsführer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard Vöckler

2 1

some generic BaseX questions
by Robby Pelssers 10 May '10

10 May '10

Hi all, * I'm wondering where baseX stores it's database by default and how one can override the location? * What approach is mostly taken to backup the database /and how can I restore it? * What would a production server typically consume in memory? * Did anyone encounter database problems (e.g. corrupt) and what could be the root cause (out-of-memory, ...) ? Basically I'm looking for people who have hands-on experience with using BaseX in a high performance critical production environment and what their experience is so far. Cheers, Robby Pelssers

1 0

question about inserting new data (using JAX-RX)
by Robby Pelssers 10 May '10

10 May '10

When reading following snippet from the documentation I start wondering how-to specify that I want to add this resource to a specific collection in the database. And the same basically applies to GET, DELETE, ... How does one specifically target a resource from a collection using JAX-RX? Kind regards, Robby Pelssers ---------------------------------------- PUT Requests PUT can be used to create or update a database resource. To create a new resource called XMark, the following URL http://localhost:8984/basex/jax-rx/XMark must be sent to the server as PUT request, followed by the XML file in the HTTP body. If the process was successful, a HTTP response with status code 201 (CREATED) is sent back. Otherwise, 404 will be sent. ----------------------------------------

2 1

jax-rx and external variables
by Hans-Peter Oeri 10 May '10

10 May '10

Hi! While learning about xml databases and how to access them, I came across your jax-rx project. While most of REST is quite straightforward, I don't see any method to define external variables. I don't quite understand why the possibility to define not only the xquery, but also the values for its external variables has been left out? It would - after all - be nice... Just like parameter substitution in SQL APIs... POST could get variable values and value types in an additional <variables> block. GET could accept parameters like var.* - probably deriving the type from the variable declaration in the xquery. If the http querystring (which always is a string...) cannot be converted accordingly, it could throw. What do you think? HPO

2 3

XQuery injection and auto-increment
by Kim Stebel 10 May '10

10 May '10

Hi! I'm planning to use BaseX as the database behind a web service. While implementing this web service, I ran into these issues: 1) Is there a way to have an auto-incremented attribute added on every insert of a certain element type? 2) How do I avoid XQuery injection? Currently I just use a whitelist of characters that are allowed in user input. But what if I want users to be able to input special characters? How would I escape them in my queries? Did I overlook something in the API? regards Kim

2 1

is there a function to update attributes?
by Beatriz Sancho 10 May '10

10 May '10

Hello, I'm trying to update or create some attributes in a node. ¿how can i do that with BaseX? This is my DTD: <!ELEMENT SITUATION ((ROLES)?,(USERS)?,(GROUPS)?,(OBJECTS)?)> <!ATTLIST SITUATION id ID #REQUIRED name CDATA #IMPLIED desc CDATA #IMPLIED start CDATA #IMPLIED finish CDATA #IMPLIED> category_scope (small | large | community | other) #IMPLIED size_scope CDATA #IMPLIED environment ( synchronous | asynchronous | face-to-face | blended | others) #IMPLIED educational_level CDATA #IMPLIED experience_level (none|elementary|advanced|expert) #IMPLIED> and i want to update or create attributes in the SITUATION node. I have seen a function called insertAttributes but i don't know how to use it with baseX, with the prefix basex: it doesn't works... i am a little lost. Could you help me, please?

2 2

ways to insert and query for documents
by Robby Pelssers 07 May '10

07 May '10

Hi all, I was wondering if BaseX also supports the concept of having collections in the database. The reason why I need collections is that I have xml files with the same identifier. My customer exports product information for different states (workflow | work in progress | released) to the file system currently but we would like to export everything to an xml database. So I'd like to create 3 separate collections for the 3 states and being able to query for product 'X' in state 'Y'. Is this possible? Robby Pelssers

2 3

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk