I may need to load hundreds of thousands of items each with dozens of sub-nodes.
I am looking for a solution that will not be at risk of running out of heap memory.
Hi,
Just about building maps, in general: If you look at map:merge in http://docs.basex.org/wiki/
Map_Module it shows a way to build a map with a for expression. If the question updating maps in a loop was about this, then it is probably preferable to have the result of the for expression be a map, rather than updating a map, iteratively.
Kendall
From: <basex-talk-bounces@mailman.
uni-konstanz.de > on behalf of "E. Wray Johnson" <wray.johnson@gmail.com>
Date: Sunday, November 19, 2017 at 5:15 AM
To: Michael Seiferle <ms@basex.org>
Cc: BaseX <basex-talk@mailman.uni-konstanz.de >
Subject: Re: [basex-talk] Loading data
Thanks. I tried doing #2 inside the iteration and that did not work. I will try it your way and let you know if it works.
Wray Johnson
(m) 704-293-9008
On Nov 19, 2017, at 6:30 AM, Michael Seiferle <ms@basex.org> wrote:Dear Wray,
I tried to come up with a sketch of what–I think–might help you:
Basically it boils down to:
1. Fetch the JSON
2. Convert that JSON to an XQuery item
3. Iterate over each array entry and explicitly construct the XML representation you want
So in a nutshell, in XQuery 3.1, something like the following:
fetch:text('https://gist.
githubusercontent.com/ ')Miserlou/c5cd8364bf9b2420bb29/ raw/ 2bf258763cdddd704f8ffd3ea9a3e8 1d25e2c6f6/cities.json => parse-json() (: Convert to XQuery item representation :)
=> array:for-each(function($map){ (: For each entry in that array, do :)
element item { (: Construct an XML element named item :)
$map => map:keys()
=> for-each(function($key){ (: For each key in the map, do: :)
element { $key } { (: Return an element named $key :)
$map($key) (: …and the value of $map($key) :)
}
})
}
})
=> array:flatten() (: Converts the array to a sequence :)
I assume you are using BaseX, so maybe the json:parse() function might be another option, that is based on our own implementation and will create an XML representation right away:
fetch:text('https://gist.
githubusercontent.com/ ')Miserlou/c5cd8364bf9b2420bb29/ raw/ 2bf258763cdddd704f8ffd3ea9a3e8 1d25e2c6f6/cities.json => json:parse()
For more info on our BaseX JSON-Module see: http://docs.basex.org/
wiki/JSON_Module
As I can only guess what your XQuery code actually looks like I hope this comes somewhat close to what you want.
Hope this helps ;-)
Best from Konstanz
Michael
Am 19.11.2017 um 06:25 schrieb E. Wray Johnson <wray.johnson@gmail.com>:
I have a JSON file
https://gist.githubusercontent.com/ Miserlou/c5cd8364bf9b2420bb29/ raw/ 2bf258763cdddd704f8ffd3ea9a3e8 1d25e2c6f6/cities.json I want to load it into a database in a different XML format where I can use gml:Point in place of longitude and latitude as separate elements. However my for loop does not emit individual XML elements for each json object in the loaded array.