Copy/modify, Update, or file:append - help appreciated
Hi all - I'm trying to keep track of some HTTP POST results in a file, something akin to the following: file:write-text( "/tmp/bridger-test.csv", "book" || "," || "title" || "," || "sip" || out:nl() ), for $i in 1 to 20 (: some http:send-request stuff :) return( prof:sleep(250), if ($request[1]//@status/data() = 201) then file:append( "/tmp/bridger-test.csv", "book: " $i || "," || "title: " $i || "," || "" ) else "something went wrong with the request; the response was: " || $request[1]//@status/data() I have something that kinda works, but I'd like it to be better XQuery, avoid the two step create-file-then-append, and leverage more of built-in awesomeness in the language. I'm sure that I need to have a better understanding of the Update syntax, and/or %updating annotation, but would someone be willing to give some hints and/or suggestions for a better way to handle situations like this? Thanks in advance for your time and trouble. Best, Bridger
Dear Bridger, I have attached one possible way to avoid file appends: * In the first run, all results will be collected. * The results will be represented as arrays. This way, it will be easier to iterate through the results, as a single result may contain two items (the header and an optional body). * Next, the CSV headers and records will be prepared for all successful results. In the example, I use the 'xquery' CSV serialization format [1], but the other variants may be more familiar. * Next, the data is written to disk, using the CSV serializer and custom CSV options. * The advantage of serializing data via the CSV features is that you don’t have to care about values with special characters, such as commas or newlines; that will all be handled by BaseX. * In the last step, information on failed responses will be returned. I added a position counter, but you can obviously output other response information here. As none of the data is updated, I didn’t use any update constructs. Maybe you would have expected something different? Feel free to ask questions if parts of the code are opaque. Christian let $results := ( for $i in 1 to 20 return [ (: some http:send-request stuff :) (: array will contain header and optional body :) ] ) return ( let $headers := [ 'book', 'title', 'sip' ] let $records := ( for $result at $i in $results?1[.//@status = 201] return [ 'book: ' || $i, 'title' || $i, '' ] ) return file:write( map { 'headers': $headers, 'records': $records }, map { 'method': 'csv', 'csv' : map { 'header': true() }} ), for $result at $i in $results let $status := $result?1//@status where $status != 201 return $i || ' failed: ' || $status ) [1] https://docs.basex.org/wiki/CSV_Module On Thu, Jul 1, 2021 at 9:35 PM Bridger Dyson-Smith <bdysonsmith@gmail.com> wrote:
Hi all - I'm trying to keep track of some HTTP POST results in a file, something akin to the following:
file:write-text( "/tmp/bridger-test.csv", "book" || "," || "title" || "," || "sip" || out:nl() ),
for $i in 1 to 20 (: some http:send-request stuff :) return( prof:sleep(250), if ($request[1]//@status/data() = 201) then file:append( "/tmp/bridger-test.csv", "book: " $i || "," || "title: " $i || "," || "" ) else "something went wrong with the request; the response was: " || $request[1]//@status/data()
I have something that kinda works, but I'd like it to be better XQuery, avoid the two step create-file-then-append, and leverage more of built-in awesomeness in the language. I'm sure that I need to have a better understanding of the Update syntax, and/or %updating annotation, but would someone be willing to give some hints and/or suggestions for a better way to handle situations like this? Thanks in advance for your time and trouble. Best, Bridger
participants (2)
-
Bridger Dyson-Smith -
Christian Grün