- BaseX-Talk - mailman.uni-konstanz.de

Create Options are deleted from .basex config file
by marc＠crosseyed.nl 05 Jun '17

05 Jun '17

Options manually added to the .basex config file are deleted by BaseX. For example, these Create Options all get deleted from the .basex file after running a standalone basex script: ADDARCHIVES = true CREATEFILTER = *.anx STRIPNS = true INTPARSE = true LANGUAGE = Dutch (also tried 'nl') Basex 8.5.3 on Linux Regards, Marc

3 2

Combining modules into a single file
by George Sofianos 02 Jun '17

02 Jun '17

Hi, and thanks for your awesome engine again! I wonder if there is an easy and complete way of combining xquery files into a single file. To give an example, I have about 16-17 xquery files, some are only 50-100 lines but some can be up to 1500 lines. Because of multiple reasons, (legacy web application behaviour that makes it hard to deploy updates, network file system that doesn't always work), it is preferable to have one single file to execute on the XQuery engine. I know it can be done with ant for example, but it's not very easy to cover all cases, so maybe someone has a better idea :) I don't mind implementing something in a different language, as long as I know it will be doable. Thanks, George

2 2

restxq output cdata-section-elements
by Maximilian Gärber 01 Jun '17

01 Jun '17

Hi, when setting %output:cdata-section-elements("KSP DESCRIPTION") on a restxq request, I get <DESCRIPTION><![CDATA[ ]]><p>Zur Aktivierung des...</p><![CDATA[ ]]><ul> <li>außerordentlich hoher Biotingehalt (3 Millionen µg pro kg)</li> <li>gut bioverfügbares Zinksulfat</li> <li>ausgewogenes Biotin-Zink-Verhältnis</li> </ul><![CDATA[ ]]><p> I would expect everything inside the DESCRIPTION element to be part of CDATA. But CDATA ends before the first child element This happens with Basex 8.5.2 Br, Max

2 2

Create user doesn’t work if DBPATH folder does not exist
by Simon Chatelain 31 May '17

31 May '17

Hello all, I just faced an issue in my application using BaseX 8.6.4 when trying to create a user. If the folder that will contain the databases (DBPATH in .basex configuration file) does not exist when you try to create a user, the creation of the “users.xml” file fails. To reproduce, make sure that the folder defined by DBPATH does not exist, and execute: final Context ctx = new Context(); try (final Session session = new LocalSession(ctx);) { session.execute(new CreateUser("USERNAME", "USERPASSWORD")); } Then nothing happens. The users.xml file is not created. While executing this: final Context ctx = new Context(); try (final Session session = new LocalSession(ctx);) { session.execute(new CreateDB("TESTDB")); session.execute(new CreateUser("USERNAME", "USERPASSWORD")); } Works as expected as the CreateDB() command will create the missing folder. Is it on purpose or was it overlooked? Best regards Simon Chatelain

2 1

Evaluating XPath within custom QueryModule
by Johannes Echterhoff 29 May '17

29 May '17

Hello, I am writing a custom QueryModule that can be used to parse information from a given XML element. The module has a function like the following: public ParsedType parse (Object arg) throws QueryException { if (arg instanceof ANode) { return utils.parse((ANode) arg); } else if (arg instanceof BXElem) { return utils.parse((Element) arg); } else { ... } } It turned out that parsing requires additional information that can be found either on the XML element itself or - more importantly for my question - in one of its ancestors. The XPath expression to determine the information looks like this: <context XML element>/ancestor-or-self::*[@attributeX or nsX:SomeElement/*/@attributeX][1]. I can evaluate this expression in the XQuery before calling method "parse" from my custom query module, and add the value of attributeX as an argument. To do so, I actually have a function to determine the value of attributeX: declare function local:determineAttributeX($elmt as element()) { let $elementWithAttributeX := $elmt/ancestor-or-self::*[@attributeX or nsX:SomeElement/*/@attributeX][1] return if (empty($elementWithAttributeX)) then () else if($elementWithAttributeX/@attributeX) then $ elementWithAttributeX/data(@attributeX) else $ elementWithAttributeX/nsX:SomeElement/*/data(@attributeX) }; However, I wonder if it is possible to evaluate the expression also within the query module itself, i.e. when method "parse" is called. Is there a way to do so? That would reduce the complexity of the XQuery that calls "parse" (and the value of "attributeX" is always of interest when "parse" is called, so ideally "parse" can determine the value itself). I searched the internet (the archive of this mailing list as well as stackoverflow) for quite some time but could not find an answer to this question. Any help would therefore be greatly appreciated. Best regards, Johannes Echterhoff

2 6

Running BaseX on a parallel cluster
by Kristian Kankainen 25 May '17

25 May '17

Hi all! Is there any way to make BaseX run in parallel on a cluster? Through school I have access to several clusters and I got interested in trying out if BaseX can take advantage of parallel computing. I know my question is vague. To try to be more specific - can I compile BaseX with some extra parameters? Is there anything written about this topic? For the background information I got interested in trying out running parallel XQuery because from somewhere (that I can't find in the mailing list archive) I read that XQuery falls under the "dataflow" programming paradigm [1] and as such should be parallelizable out-of-the-box. Also being a functional language with no side-effects makes automatic parallelization easier. I am not a computer scientist and/but I am aware it isn't that easy as I might make it sound. Cheers Kristian K [1] https://en.wikipedia.org/wiki/Dataflow_programming

2 5

Profiling queries
by Iwan Briquemont 25 May '17

25 May '17

Hello, I was wondering if there are ways to access how much time is spent in parts of a query/program, to find out inefficient functions or expressions? Something like the total time spent in functions would be quite useful. I saw the Profiling module, but it requires you to change the code and it also won't aggregate results when you run it in a loop. It's still doable if you add labels and then do some parsing on stderr to aggregate the results yourself, but it's quite cumbersome. Some thoughts as a side note: A nice way to avoid having to change the code to do profiling (or other debugging like breakpoints) would be to change or add hooks into the AST of the query. That way you could for example add a prof:time() around all the function bodies, with the function name as label. And to help interpret the results of debugging functions like prof:time or trace, it would be nice to have a way to format them and output them as a stream of items (or even multiple streams). Then you could create XQuery scripts that consume the streams to aggregate the information in a human readable way. Best regards, Iwan

2 3

Approximating ML's Reverse Queries Mechanism?
by Eliot Kimber 23 May '17

23 May '17

I’m working on a project that currently depends on MarkLogic’s reverse query mechanism. This is a feature whereby you store documents that contain MarkLogic-specific queries (queries in their cts namespace). These get indexed in some way (no idea how the index works). You can then search for these stored queries in the context of specific nodes and ML will return those query documents that would match the specified nodes. Their driving use case is alerting-type applications where when new docs get added you see what queries they apply to and then use those queries to do something. My use case is classification: given a node to be classified, find all queries that match it and from the query get the classification details (preferred term, variant forms, associated taxonomy, etc.). This process definitely depends on MarkLogic’s full-text search features, for example to match any form of a non-preferred term to a full-text search that would match it. I have a large corpus to classify (approximately 45 million objects at current count). The processing is inherently parallelizable so I’m looking at non-ML options that would allow us to scale to the limits of our hardware budget. Even if each node was less efficient than ML we would be able to implement massive throughput for this classification operation. In theory we could scale to one processor per object if budget were unlimited (it is not unlimited), so even a brute-force solution would perform well at larger scales. So I guess I have two questions really: 1. Can anyone share ways they use BaseX for doing classification in general? I’ve so far just been focused on analyzing the current system to find performance bottlenecks so I haven’t yet had a chance to think through the classification process in general, but there must be well-understood strategies. I suspect that one could build the equivalent of ML’s reverse query index in BaseX. 2. Is there a direct equivalent to ML’s reverse query facility in BaseX or an obvious route to building one? Thanks, Eliot -- Eliot Kimber http://contrext.com

1 0

Querying documents by a full path pattern
by Jacob Borisov 19 May '17

19 May '17

Hello. Is it possible to select documents in a database by a path pattern? For example, there is a database storing following documents: dir1/a.xml dir1/b.xml dir2/a.xml dir2/b.xml I want to select documents named 'b.xml' only. I guess something like db:open('dbname', '*/b.xml')/* but it returns empty result. Thanks!

2 1

BaseX 8.6.4
by Christian Grün 18 May '17

18 May '17

Dear all, Another patch release of BaseX release is available. It includes one critical fix for fn:tail. If you use this function, an update is definitely recommended. Cheers, Christian

1 0