- BaseX-Talk - mailman.uni-konstanz.de

Text index on uncommon node
by Matthew Dziuban 30 Oct '23

30 Oct '23

Hi all, I'm working with a database structured like so: <data> <element>...</element> <element>...</element> <element>...</element> </data> There are a total of about 1.5 million <element> nodes in the database. Each <element> has many child nodes, one of which is uncommon -- it only appears in 727 <element>s. I'm writing a query that has a condition on this uncommon field, but the query takes about 20 seconds to run, whereas another with a condition on a child node that appears in every <element> only takes about 20 milliseconds to run. Based on the Info in the GUI, it does appear that the text index is being used -- I see 'apply text index for "..."'. Is it expected that the query time would be this much longer? Is the text index somehow built differently for nodes that don't appear often in the database? Thanks in advance, Matt

2 4

RESTXQ Custom Multipart Response
by Jean-Marc ANSEL 29 Oct '23

29 Oct '23

Hello, I'm trying to do a custom multipart response with basex RESTXQ module. The Custom Response paragraph of https://docs.basex.org/wiki/RESTXQ indicates : Custom responses can be generated in XQuery by returning an |rest:response| element, an |http:response| child node that matches the syntax of the EXPath HTTP Client Module <http://expath.org/spec/http-client> specification, and optional child nodes that will be serialized as usual. So, the §4.1 of http://expath.org/spec/http-client (EXPath HTTP Client Module <http://expath.org/spec/http-client>) indicates : <http:response status = integer message = string> (http:header*, (http:multipart | http:body)?) </http:response> But when I try : declare %rest:path('media4') function page:testjm4() { <rest:response> <http:response status="200" message="OK"> <http:header name="Content-Length" value="100"/> <http:body media-type="text"> TEST </http:body> </http:response> </rest:response>}; The answer is irremediably : Stopped at /root/basex/webapp/test.xqm, 46/39: [basex:restxq] Unexpected node: <http:body xmlns:http="http://expath.org/ns/http-client" media-type="text">
TEST 
</http:body> Neither hhtp:body or http:multipart element can be placed as a child of http:response, that seems contradictory with the expath.org spec... Any help would be appreciated... Thanks o lot! JM Using : BaseX Version 10.7 httpserver : basexhttp Operating System: Ubuntu 18.04.6 LTS Kernel: Linux 4.15.0-197-generic Architecture: x86-64

2 1

Db:token() vs db:attribute(): Attribute is much slower -- why
by Eliot Kimber 26 Oct '23

26 Oct '23

Using 10.6 on macOS: I have a data file with elements like: <record language="en-US" family="pce-test-data" bundlename="bundle-appportman-application-portfolio-management" topicpath="product/application-portfolio-management/concept/application-portfolio-management-landing-page.dita" notviewed="viewed" viewcount="13579"> One such entry for each topic in our public docs server that is accessed at least once (Adobe page views data). I generate a report for all the topics used from a root map that includes the page views data for each topic for which there is an entry. The database with this data set is indexed with an attribute and a token index. In my testing I’m finding that lookup using db:attribute() on the @topicpath attribute (which is always a single value) is about 10x slower than using db:token(). That is, I have this code: $records := db:token($analyticsmgmt:analyticsDatabase, $docPath, 'topicpath')/.. When I run this for a set of about 1200 topics, the db:token version takes about 400ms but the db:attribute version takes 9 seconds! I’m surprised at this result—I would expect the two forms of index to perform with the same speed: the attribute is a single blank-delimited token and I would expect the string match under the covers to be using the same code path. But using the db:token database is clearly faster by a lot. If I report the time each lookup uses, the times vary quite a lot, but for the token lookup they tend to be in the 10-60ms range, but for the attribute lookup, many are in the 1 or 2 second range, which explains the much longer run time. I’m wondering if this result is expected due to implementation details of the attribute index as distinct from the token index or if this indicates some logic bug in the attribute index implementation. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 4

Crash when starting basexgui
by Joe Wicentowski 24 Oct '23

24 Oct '23

Hi all, I am experiencing a crash when starting basexgui. The splash screen appears briefly, and then the message pasted in below appears, and I've also attached the referenced error report file and the crash report that appears in the Mac crash dialog. I'm using BaseX 10.7, installed via Homebrew (which installed BaseX's openjdk dependency and thus is using openjdk 21), on macOS Sonoma (14.0). I'd be happy to provide any other info that would help in identifying the cause of the crash. Joe % basexgui 2023-10-24 07:34:22.308 java[69449:3862142] WARNING: Secure coding is automatically enabled for restorable state! However, not on all supported macOS versions of this application. Opt-in to secure coding explicitly by implementing NSApplicationDelegate.applicationSupportsSecureRestorableState:. # # A fatal error has been detected by the Java Runtime Environment: # # SIGILL (0x4) at pc=0x00007ff80d2b1b73, pid=69449, tid=259 # # JRE version: OpenJDK Runtime Environment Homebrew (21.0) (build 21) # Java VM: OpenJDK 64-Bit Server VM Homebrew (21, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-amd64) # Problematic frame: # C [AppKit+0xc8fb73] _NSCarbonMenuCrashIfNeeded+0x258 # # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /Users/joe/workspace/hsg-project/repos/hsg-shell/hs_err_pid69449.log # # If you would like to submit a bug report, please visit: # https://github.com/Homebrew/homebrew-core/issues # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # zsh: abort basexgui

1 0

New escape-solidus option
by Thompson, Timothy 23 Oct '23

23 Oct '23

Hello, I was very happy to see that a new escape-solidus option had been defined for XQuery/XPath 4.0. Is this available in the latest BaseX snapshot? I tried this: <fn:map> <fn:string key="Escape/Solidus">Test</fn:string> </fn:map> => serialize(map {"method": "json", "json": map {"format": "basic", "escape-solidus": "no"}}) But it returns: {"Escape\/Solidus":"Test"} Thank you, Tim -- Tim A. Thompson (he, him) Librarian for Applied Metadata Research Yale University Library

2 3

Access denied using client
by Csaba Fekete 21 Oct '23

21 Oct '23

Hello, A total newbie here, I'm a first time user. I have installed basex on Ubuntu and I'm running the server attached to 127.0.0.1, on port 1984. It looks like this is working and the server is listening. Now I create a password for admin: basex@cac5ba21e248:/opt/basex/bin$ ./basex BaseX 10.8 beta [Standalone] Try 'help' to get more information. > PASSWORD xxx Password of user 'admin' changed. Now I'm trying to connect using the client: basex@cac5ba21e248:/opt/basex/bin$ ./basexclient -n127.0.0.1 -p1984 -Uadmin -Pxxx Access denied: admin. The server log confirms the same: basex@cac5ba21e248:~/basex/.logs$ cat 2023-10-20.log 18:51:30.575 SERVER admin OK Server was started (port: 1984). 18:51:42.316 172.22.0.1:41314 admin ERROR Access denied: admin. 30.7 ms 18:53:04.079 172.22.0.1:59892 admin ERROR Access denied: admin. 23.59 ms What am I missing? Thanks and regards PS. I'm doing this inside a docker container if that matters.

3 6

Advice on accepting UTF-16 XML (and others) from RESTXQ?
by Clark, Ash 20 Oct '23

20 Oct '23

Hello! In brief, I’m looking for any advice folks have on accepting XML of various encodings into a single database. Are there any considerations I should take into account with, say, RESTXQ form parameters? Or with storage, indexing, querying, etc.? For context, I’m currently (re)building a RESTXQ API for a service that publishes TEI. The site, TAPAS, is open to anyone with TEI that might need a place to store it and show it off online. We do some minimal testing on uploaded files [1], but up to this point we haven’t applied any limitations on the character encoding that folks use. While I expect that many people use UTF-8 encoding for TEI, I would like to try to ensure that folks using other encodings can use the service as well. The previous version of the TAPAS-xq API was not tested in this regard (and in other ways as well). I’m trying to do better in writing this new version. One problem that I’m running into is simply retrieving UTF-16 XML from a multipart form parameter.[2] I think BaseX is serializing the file as a string, but when I try to parse it as XML, the file is flagged as ill-formed (“Content is not allowed in prolog”). Is there a way to be flexible about uploaded XML while giving BaseX any parsing/serialization hints it might need? I’m about at the limit of my ability to figure out how different encodings might be working within BaseX; I appreciate any and all information or advice! Warmly, Ash [1]: Our minimal testing: Is this well-formed XML? Is it in the TEI namespace? Is the outermost element <TEI> rather than <teiCorpus> or anything else? We also try to make sure the XML doesn’t have Javascript that might make it into a reader’s browser. [2]: I first discovered this might be a problem when creating a unit test<https://github.com/NEU-DSG/tapas-xq/blob/d66066e65a661a4b909c21ae852d6479f0…> for this function<https://github.com/NEU-DSG/tapas-xq/blob/d66066e65a661a4b909c21ae852d6479f0…> — the UTF-16 file-as-string could not be parsed as XML. I then tested a UTF-16 file against the RESTXQ endpoint with curl, and found the same problem (though I do not know if the string has been read as UTF-8 or -16). In contrast, when I tried to create a standalone test module<https://gist.github.com/amclark42/9f8d8135a30e4659774a673627a263ac>, a UTF-16 file-as-string could be parsed as XML, but not when encoded as binary and then decoded again. Ash Clark (my pronouns are e/em/eir) XML Applications Developer Digital Scholarship Group Northeastern University Libraries as.clark(a)northeastern.edu (617) 373-5983

2 2

Trying to get process details on macOS
by Eliot Kimber 20 Oct '23

20 Oct '23

In the context of a 10.6 server running as a web app, I’m trying to implement a function that uses the process module to run commands to get information about the running BaseX servers, in particular, the PID and port each server is on. I start a number of “worker” BaseXHTTP instances to handle long-running processes. On Linux this code works: let $cmdResult := proc:execute(‘ps’, ('-o', 'pid,%cpu,command')) >From which I can parse out the PID and port for BaseX processes: let $lines as xs:string* := ($cmdResult/output/text() => tokenize('
')) return map{ 'data' : let $servers as map(*) := map:merge( for $line at $p in $lines[contains(., 'BaseXHTTP')] let $pid as xs:string := tokenize($line, '\s+')[1] let $port as xs:string := tokenize(substring-after($line, ' -p'),'\s+')[1] return map{ $port : map{ 'port' : $port, 'pid' : $pid } } ) return $servers } However, on macOS there’s not an exact equivalent of the linux version of ps, but there is the “pgrep” command, which, from the command line, lets me get the data I need: pgrep -lf basex However, when I try to run this from within BaseX, I get “sysmon request failed with error: sysmond service not found pgrep: Cannot get process list” I assume this is a function of how the HTTP servers are running on macOS. I’m starting them like so: "${basexBinDir}/basexhttp" ${debugFlag} -S -p${clientPort} -h${httpPort} -s${stopPort} & So as a service Any ideas how I might make this work on mapOS? Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 3

More options for http-client?
by Andreas Hengsbach | nexoma 15 Oct '23

15 Oct '23

Hi Christian, are there any plans for the http client to become even more configurable? I have seen in the BaseX sources that the HttpClient.Builder (org.basex.io.IOUrl:173) is mostly served with standards. (except sslContext, connectTimeout, followRedirects). For example, I have a special case where I need to force the version to HTTP/1.1. Tried as a workaround to create my own instance via HttpClient.Builder using the XQuery Java binding, but failed on the interfaces. Thanks a lot Andreas

2 1

use-character-maps vs parameter-document with db:export()
by Zimmel, Daniel 09 Oct '23

09 Oct '23

Hi, when using db:export, the serialization parameter use-character-maps works just fine. But can someone explain why it is not possible to use parameter-document? At least, it does not work here with 10.3. As in: db:export($dbname,$path, map{ 'indent':'no', 'omit-xml-declaration':'no', 'parameter-document':'map.xml' }) OK I can see that there is no parameter-document option in the table I can find here defined for fn:serialize(): https://www.w3.org/TR/xpath-functions-31/#func-serialize When not using db:export(), all is well as described: https://docs.basex.org/wiki/Serialization#Character_mappings So in short, I think the answer is the spec. Is it? Have a nice weekend, Daniel

2 3