- BaseX-Talk - mailman.uni-konstanz.de

count(//elem) not optimized, even though `elem` is in the index
by Gioele Barabucci 23 Feb '17

23 Feb '17

Hello, I was stressing the BaseX 8.6 planner/optimizer when I noticed that expressions like `count(//elem)` are not optimized at all, even though they are correctly indexed, as demonstrated by `index:element-names()`. The current database is a 300 MB TEI document. All the elements are in the `http://www.tei-c.org/ns/1.0` namespace. The following test case will report the correct number, but it will take a couple of seconds to run, instead of a few milliseconds. ``` declare namespace tei="http://www.tei-c.org/ns/1.0"; let $n := index:element-names("monier")[. = 're']/@count let $c := count(//tei:re) return <res><in-index>{$n}</in-index><in-doc>{$c}</in-doc></res> ``` I wonder if the presence of the namespace somehow confuses the optimizer. The same problem can be observed running the same test case with ``` declare default element namespace "http://www.tei-c.org/ns/1.0"; [...] let $c := count(//re) ``` Regards, -- Gioele Barabucci <gioele.barabucci(a)uni-koeln.de>

3 5

XSLT caching
by Tom De Herdt 19 Feb '17

19 Feb '17

Hi all, I'm evaluating BaseX as an alternative (and very attractive) platform for an XML/XSLT-based website that needs to be migrated from ASP.NET. The website relies heavily on XSLT. Each page is generated on-the-fly with Saxon.NET, using a complex set of stylesheets. To get reasonable performance, stylesheets are compiled on first use and cached for subsequent requests. This is crucial, as XSLT compilation is typically orders of magnitude slower than execution; without caching, the server would spend most of the time compiling the same stylesheets over and over again. I was happy to find that BaseX can use Saxon, but as far as I can see, xslt:transform() does not cache compiled stylesheets. Can anyone confirm this? If not, are there any plans to support stylesheet caching in the future? Or is there a way to reuse compiled stylesheets manually? Thanks, Tom De Herdt

2 7

Allowing for comments in code...
by France Baril 18 Feb '17

18 Feb '17

I was wondering if it would be possible to either: * Allow for an omit-comment option in HTML5 serializer * Allow for {(: Nothing but a comment :)} in xquery code. I often have long pieces of HTML to output and clients don't want me to output them with the result. However, I can't use {(::)}. I have been using useless code around my comments just to get around this issue: { let $t := '' return $t (: The comment I can't leave without the bogus code:) } Thanks! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 6

in memory
by Patrice Drolet ift.a., MD 18 Feb '17

18 Feb '17

Hello, I am trying to use the option MAINMEM to get faster access to data. I can’t make it work. How to use this option in java? When it is set, I get an error with new CreateDB(dbname).execute(context); Thanks, Patrice Drolet

2 1

Retrieve large amount of data from XML-resource
by Jan Stapel 15 Feb '17

15 Feb '17

Hello, i am getting an BaseXException when trying to get large amount of data from a XML-resource with a JAVA-Client. For example for a xml-file with the size of 79167874 ("FACTOR", as shown by the web-gui) "session.execute(new XQuery("doc('db/file.xml')"));" will result in an exception[0]. My assumption is that a timeout is exceeded somewhere.. so my next question is how to properly set the timeout (Or in general, how to run specific commands via java-client, I can't find it in the wiki. "session.query("timeout[0]");" seems to not have any influence. Query large files in general, for example query its number of instances, seems to not be the problem. Increasing the -Xmx in the basex-script did not help. # Options for virtual machine (can be extended by global options) BASEX_JVM="-Xmx10G $BASEX_JVM" Increasing the -Xmx of the java-application did not help as well. I hope you can help me. best regards, Jan [0] org.basex.core.BaseXException: Interrupted. at org.basex.api.client.ClientSession.receive(ClientSession.java:191) at org.basex.api.client.ClientSession.execute(ClientSession.java:160) at org.basex.api.client.ClientSession.execute(ClientSession.java:165) at org.basex.api.client.Session.execute(Session.java:36) at net.ddns.janstapel.datamanagement.Main.main(Main.java:37)

2 5

Re: [basex-talk] Folder operations in BaseX
by Christian Grün 15 Feb '17

15 Feb '17

Hi Bhushan (cc to the list), So you are exclusively working with binary resources, right? In that case, a query like the following one can be used: let $db := 'db' let $src-path := 'src/' let $trg-path := 'trg/' for $src in db:list($db, $src-path) let $trg := $trg-path || substring-after($src, $src-path) return db:store($db, $trg, db:retrieve($db, $src)) I have added a slightly revised version of this query in our Wiki [1]. Hope this helps, Christian [1] http://docs.basex.org/wiki/Database_Module#db:store On Wed, Feb 15, 2017 at 6:11 AM, Nanday, Bhushan <bhushan.nanday(a)sap.com> wrote: > Hi Christian, > > Thanks for reaching out. I am currently trying to use the db:store(source, destination, file) in an xqeury. This approach works well when the source is a file but doesn’t work when the source is a folder. I need something that will move/copy and entire folder and not just a file. > > > > > > On 2/13/17, 1:46 PM, "Christian Grün" <christian.gruen(a)gmail.com> wrote: > > Hi Bhushan, > > > Hope everyone is doing good. I am currently trying to copy an entire > > folder from the source location to a destination location. > > How did you proceed so far? > > Regards, > Christian > >

1 0

Folder operations in BaseX
by Nanday, Bhushan 13 Feb '17

13 Feb '17

Hi All, Hope everyone is doing good. I am currently trying to copy an entire folder from the source location to a destination location. I have tried rename but its too expensive from time response stand point when the schema size is too large. To go around the problem I am trying to run the xquery using db:store (). But db:store doesn’t seem to work on folders. So I am wondering if there is any other efficient way ( apart from rename and db:store) which will work on the folder. I need to copy the entire folder from source location to destination and it may have a hierarchy of sub-folders in it. Any help on that is really appreciated! Thanks, Bhushan

2 1

DFDL module and XSLT module
by Kristian Kankainen 13 Feb '17

13 Feb '17

Hello all. I am making a simple wrapper java module for Daffodil so that I can run DFDL parsing and unparsing from XQuery. I got communication working between BaseX and my module allready, but I can only send and return Strings. Because the module is so similar to the XSLT module of BaseX, I wonder if the java source code is available. I can't find it in the GitHub repository. Thanks Kristian K

2 1

Update facility
by meumapple 09 Feb '17

09 Feb '17

I have written a function consisting in a copy/modify expression: declare function local:mine($s) { copy $e:= $s modify (...) return $e }; It seems it is not possible to somehow make it recursive ( ... return local:mine($e) ): can anyone confirm this? What is the best way to have a recursive function while using the update facility? Best, Giuseppe

2 1

Re: [basex-talk] Send multipart binary with http client
by Christian Grün 09 Feb '17

09 Feb '17

Hi Max, Do you have an MCVE for us? Thanks in advance, Christian Am 30.01.2017 4:24 nachm. schrieb "Maximilian Gärber" <mgaerber(a)arcor.de>: Hi, this seems related to https://github.com/BaseXdb/basex/issues/1203 Problem 1: When setting Content-Transfer-Encoding manually (via <http:header>), it will appear twice in the request that is sent Problem 2: I cannot override base64 because it happens automatically based on the content I am sending and for some reason the web server that receives this request does not recognize the base64 encoding correctly Regards, Max 2017-01-30 15:35 GMT+01:00 Maximilian Gärber <mgaerber(a)arcor.de>: > Hi, > > I can't figure out how to send a file in binary (not base64) when > using multipart [1] > > On the body, I've tried method="basex" because the docs says this > would write raw bytes to the request > > Regards, > Max > > > [1] > > declare function _:send-multipart($url, $file-name, $path) as item()* { > > > let $uri := file:path-to-uri($path || $file-name) > let $type := fetch:content-type($uri) > > let $binary := file:read-binary($path || $file-name) > let $boundary_id := concat("batch_", random:uuid()) > > let $req := > <http:request method="post" send-authorization="false" > override-media-type="text/plain"> > > <http:header name="Accept" value="application/json"/> > <http:header name="Accept-Encoding" value="gzip, deflate"/> > > <http:multipart media-type="multipart/form-data" > boundary="{$boundary_id}"> > > <http:header name="Content-Transfer-Encoding" value="binary"/> > <http:header name="Content-Disposition" value="form-data; > name='FILE'; filename='{$file-name}'"/> > <http:header name="Content-Type" value="{$type}"/> > >  > <http:body media-type="application/octet-stream" method="binary"> > {$binary} > </http:body> > > </http:multipart> > </http:request> > > let $resp := http:send-request($req, $url) > return json:parse($resp[2]) > > };

2 5