Load xml files to BaseX in a loop
I am new at BaseX so please pardon the newbie question. I am trying to run an xquery script to load a set of xml files in a directory into to BaseX 7.5. If I run replace.xq, shown below, it fails with Out of Main Memory declare %updating function local:upd($path,$file) { db:replace("eams",$path,$file) }; let $files:=for $file in file:list("d:\xmlfiles") return "d:\xmlfiles\"||$file for $file in $files return local:upd("pd",$file) But if I run file replace.bxs, it updates with no problem <commands> <replace path="p1">d:\xmlfiles\drr1.xml</replace> <replace path="p1">d:\xmlfiles\drr2.xml</replace> <replace path="p1">d:\xmlfiles\drr3.xml</replace> <replace path="p1">d:\xmlfiles\dw1.xml</replace> <replace path="p1">d:\xmlfiles\dlp1.xml</replace> </commands> (The same set of files is being processed by both scripts; I checked) The only idea I have is to use xquery to generate the file replace.bxs and then run it. A better way? Thanks.
Hi Nicholas,
If I run replace.xq, shown below, it fails with Out of Main Memory
as you already experienced in your tests, the replace command takes less memory than the db:replace() function, as the second one will first cache all update operations on the Pending Update List [1] before actually executing them.
The only idea I have is to use xquery to generate the file replace.bxs and then run it.
This is indeed a popular approach. If all of your files are stored in a single directory, you could as well try to delete your documents and add all files via a single db:add call: db:delete("database"), db:add("database", "/path/to/dir") Hope this helps, Christian [1] http://docs.basex.org/wiki/XQuery_Update#Pending_Update_List
participants (2)
-
Christian Grün -
Nicholas OR Rita Stevens