XQuery performance degradation as number of json files grows
The performance of XQuery updates degrades as the number of JSON resources in my BaseX instance grows. In what ways might the number of JSON resources affect the performance of an XQuery update for a particular XML resource? I was under the impression that JSON resource were not indexed. (Note: this particular performance degradation is most dramatic when the BaseX server points to an NFS mount for the BaseX database.) Thanks, Joe
In what ways might the number of JSON resources affect the performance of an XQuery update for a particular XML resource?
What kind of updates are you performing? How do you store your JSON resources?
Below is a representative XQuery expression, wherein the JSON file is added via db:rename, and that addition is journaled to an XML log file via the write-log-event function. *I’ve noticed that the update to the xml log is adversely affected as the number of JSON files in the db grows.* Here’s the XQuery: declare namespace _="http://com.sap.ariba.metadata.event/log"; declare namespace log="http://com.sap.ariba.metadata.event/log"; declare variable $logFilename external := "2017-01-31.xml"; declare variable $logDirectory external := "log"; declare variable $dbSchema external := "schemaRepo"; declare function _:write-log-event() as empty-sequence() { let $event := <entry id="6669" timestamp="2017-01-31T14:24:49.83-05:00"> <command toPath="meta/services/demo/serviceversions/v1/docs/demo/biggerform.json/1" isBinary="true" dbSchema="schemaRepo" type="ADD_BINARY" ></command></entry> let $logPath := $logDirectory || '/' || $logFilename let $fullPath := $dbSchema || '/' || $logPath return insert node $event as last into doc($fullPath)//log:log }; declare variable $tempFileUri external := "temp/7b04aab6-6ee6-47c7-89e0-e5132e505daf/meta/services/demo/serviceversions/v1/docs/demo/biggerform.json/1"; declare variable $toPath external := "meta/services/demo/serviceversions/v1/docs/demo/biggerform.json/1"; if (not(db:exists($dbSchema, $logDirectory || '/' || $logFilename))) then ( error(QName('http://com.sap.ariba.metadata/err', 'LogFileReq'), "write operation logFile not exist.") ) else ( try{ db:rename( $dbSchema, $tempFileUri, $toPath), _:write-log-event(), db:output("SUCCESSFULLY") } catch * { fn:string("FAILED" || ": " || $err:description) } ) Thanks, Joe On 1/27/17, 6:11 AM, "Christian Grün" <christian.gruen@gmail.com> wrote: > In what ways might the number of JSON resources > affect the performance of an XQuery update for a particular XML resource? What kind of updates are you performing? How do you store your JSON resources?
Hi Joe,
Below is a representative XQuery expression, wherein the JSON file is added via db:rename, and that addition is journaled to an XML log file via the write-log-event function.
Sorry, I can’t follow. How/why do you use db:rename to add resources? Do you store all JSON files as binaries (raw files)? In that case, it’s probably your file system that does not support large numbers of files in the same directory. Did you check out our Wiki article on binary files [1]? Christian [1] http://docs.basex.org/wiki/Binary_Data
participants (2)
-
Baysdon, Joseph -
Christian Grün