Hello Kristian,

thanks for the pointer! The documentation is certainly misleading, I agree. However, it is not wrong and I think the issue here is that RUN can be used for two completely separate things: run an xquery file (where it is indeed a single transaction, so the comment is appropriate) and run a basex command script.

I personally would remove the second functionality (run a basex command script) from the RUN command and instead include the functionality in the already existing EXECUTE function. That way it would be clear: Use RUN for xquery files, use EXECUTE for bxs.

I will discuss this with Christian Grün when he's back.

Cheers Dirk

On 05/20/2016 09:41 AM, Kristian Kankainen wrote:

Maybe it would be wise to change the documentation or specify what exactly is meant by the phrase "This command can be used to run several commands in a single transaction". Clearly it can be understood in a wrong way.

Cheers Kristian

20.05.2016 09:59 Dirk Kirsten kirjutas:

...

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote:

...

Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

Hi Basex Guru’s

Thanks! So I’ve been trying few things. I think the bath insert works, for both db:add and db:store. So right now I’m trying the delete and store together.

I have:

db:delete('dbTestRepo', 'testdir/someresource.xml'), db:store('dbTestRepo', 'testdir/someresource.xml', file:read-binary('/Users/i843335/work/extensibility/dbtest/buyer2.json’))

It am assuming that in this case, it should do delete first (if the path exist), and then do the store to the same path. However, I find that regardless of whether the existing resource exist for the same resource, both are not being committed.

It does seem that the results have 2 updated item, but nothing gets persisted when I do db:list(‘dbTestRepo’).

Is there something that I have missed ? Thanks.

Regards, -Genneva

From: <basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Dirk Kirsten <dk@basex.orgmailto:dk@basex.org> Date: Thursday, May 19, 2016 at 11:59 PM To: "basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de" <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] multiple inserts/update/deletes in single transaction

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote: Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

Thanks Christian,

I tried it out so it does seemed to behave the way you just described. So the documentation (page 198~199 http://files.basex.org/releases/8.4/BaseX84.pdf, also extracted below seems to be misleading.)

Seems like db:store should be in the same as db:replace ?

db:replace Signatures db:replace($db as xs:string, $path as xs:string, $input as item()) as empty-sequence() db:replace($db as xs:string, $path as xs:string, $input as item(), $options as map(*)) as empty-sequence() Summary Replaces a resource, specified by $path, in the database $db with the contents of $input, or adds it as a new resource:

db:store Signatures db:store($db as xs:string, $path as xs:string, $input as item()) as empty-sequence() Summary Stores a binary resource specified by $input in the database $db and the location specified by $path.

Thanks! Regards, -Genneva

On 5/26/16, 11:10 PM, "Christian Grün" christian.gruen@gmail.com wrote:

Hi Genneva,

In XQuery Update, it does not matter in which order you supply update commands (see [1]). The delete can be omitted, as existing binary resources will be replaced (so in a way STORE is more similar to REPLACE).

Cheers, Christian

[1] http://docs.basex.org/wiki/XQuery_Update#Pending_Update_List

On Fri, May 27, 2016 at 2:05 AM, Wang, Genneva genneva.wang@sap.com wrote:

...

Hi Basex Guru’s

Thanks! So I’ve been trying few things. I think the bath insert works, for both db:add and db:store. So right now I’m trying the delete and store together.

I have:

db:delete('dbTestRepo', 'testdir/someresource.xml'), db:store('dbTestRepo', 'testdir/someresource.xml', file:read-binary('/Users/i843335/work/extensibility/dbtest/buyer2.json’))

It am assuming that in this case, it should do delete first (if the path exist), and then do the store to the same path. However, I find that regardless of whether the existing resource exist for the same resource, both are not being committed.

It does seem that the results have 2 updated item, but nothing gets persisted when I do db:list(‘dbTestRepo’).

Is there something that I have missed ? Thanks.

Regards, -Genneva

From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Dirk Kirsten dk@basex.org Date: Thursday, May 19, 2016 at 11:59 PM To: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] multiple inserts/update/deletes in single transaction

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote:

Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

Hi Christian,

Sorry, I meant that db:store should have the same “Summary” (or equivalent version) as db:replace ?

Thanks for suggesting revising documentation, however, considering writing is probably my worst subject since school days, I would pass on the opportunity. However, I would be more than happy to provide any feedback that I find with the documentation as I go through some testing. :) Thanks!

Regards, -Genneva

On 5/27/16, 1:39 AM, "Christian Grün" christian.gruen@gmail.com wrote:

Hi Genneva,

...

Seems like db:store should be in the same as db:replace ?

I didn’t get this?

Wiki edits are welcome. Would you be interested in revising the documentation?

Christian

On Fri, May 27, 2016 at 8:35 AM, Wang, Genneva genneva.wang@sap.com wrote:

...

Thanks Christian,

I tried it out so it does seemed to behave the way you just described. So the documentation (page 198~199 http://files.basex.org/releases/8.4/BaseX84.pdf, also extracted below seems to be misleading.)

Seems like db:store should be in the same as db:replace ?

db:replace Signatures db:replace($db as xs:string, $path as xs:string, $input as item()) as empty-sequence() db:replace($db as xs:string, $path as xs:string, $input as item(), $options as map(*)) as empty-sequence() Summary Replaces a resource, specified by $path, in the database $db with the contents of $input, or adds it as a new resource:

db:store Signatures db:store($db as xs:string, $path as xs:string, $input as item()) as empty-sequence() Summary Stores a binary resource specified by $input in the database $db and the location specified by $path.

Thanks! Regards, -Genneva

On 5/26/16, 11:10 PM, "Christian Grün" christian.gruen@gmail.com wrote:

...

Hi Genneva,

In XQuery Update, it does not matter in which order you supply update commands (see [1]). The delete can be omitted, as existing binary resources will be replaced (so in a way STORE is more similar to REPLACE).

Cheers, Christian

[1] http://docs.basex.org/wiki/XQuery_Update#Pending_Update_List

On Fri, May 27, 2016 at 2:05 AM, Wang, Genneva genneva.wang@sap.com wrote:

...

Hi Basex Guru’s

Thanks! So I’ve been trying few things. I think the bath insert works, for both db:add and db:store. So right now I’m trying the delete and store together.

I have:

db:delete('dbTestRepo', 'testdir/someresource.xml'), db:store('dbTestRepo', 'testdir/someresource.xml', file:read-binary('/Users/i843335/work/extensibility/dbtest/buyer2.json’))

It am assuming that in this case, it should do delete first (if the path exist), and then do the store to the same path. However, I find that regardless of whether the existing resource exist for the same resource, both are not being committed.

It does seem that the results have 2 updated item, but nothing gets persisted when I do db:list(‘dbTestRepo’).

Is there something that I have missed ? Thanks.

Regards, -Genneva

From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Dirk Kirsten dk@basex.org Date: Thursday, May 19, 2016 at 11:59 PM To: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] multiple inserts/update/deletes in single transaction

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote:

Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

Hi Genneva,

I’m not exactly sure what you are after.. Do you want to store multiple binary files in a single transaction?

I guess you have already tried the store() function of the client, but this function is indeed limited to a single file.

You’ll probably need to send multiple files to the server (and store them e.g. in a temporary directory) and then use another XQuery function to store all of them in the database. This will give you better performance anyway.

Christian

Hi basex gurus,

Given that inserts can be done in single transaction based on the context below. I’m wondering, if we do bulk retrieval, for example

db:open('testdb', 'abc.xml'),db:open(’testdb', 'xzy.xml')

This returns both content in concatenated results. I’m wondering is there a way to parse the results so there is an easy way to figure out what are the contents for abc.xml and what are the contents for xyz.xml? Any tips would be greatly appreciated. Thanks!

Thanks, Regards, -Genneva

From: <basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Dirk Kirsten <dk@basex.orgmailto:dk@basex.org> Date: Thursday, May 19, 2016 at 11:59 PM To: "basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de" <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] multiple inserts/update/deletes in single transaction

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote: Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22

Thanks Christian!

Thanks for the pointers. I’m also newbie in xquery, any idea what are the delimiter or end of file marker to look for to parse the file? Or any example of how that is done? Thanks!

Regards, -Genneva

On 7/20/16, 1:43 PM, "Christian Grün" christian.gruen@gmail.com wrote:

Hi Genneva,

XQuery is a full and sophisticated programming language, so you can do virtually anything you want with it, in particular when it comes to data processing. In other words: Yes, you can parse the results to figure out the contents, etc.

Christian

On Wed, Jul 20, 2016 at 10:29 PM, Wang, Genneva genneva.wang@sap.com wrote:

...

Hi basex gurus,

Given that inserts can be done in single transaction based on the context below. I’m wondering, if we do bulk retrieval, for example

db:open('testdb', 'abc.xml'),db:open(’testdb', 'xzy.xml')

This returns both content in concatenated results. I’m wondering is there a way to parse the results so there is an easy way to figure out what are the contents for abc.xml and what are the contents for xyz.xml? Any tips would be greatly appreciated. Thanks!

Thanks, Regards, -Genneva

From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Dirk Kirsten dk@basex.org Date: Thursday, May 19, 2016 at 11:59 PM To: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] multiple inserts/update/deletes in single transaction

Hello Genneva,

this is expected behaviour. BaseX script executed each command separately. This is by design and required by the nature of XQuery (Update), which by design always executes as one transaction - Which can be difficult do deal with if you have a large amount of data and have to e.g. hold it in memory.

Each XQuery runs in one single transaction, so you can simply add your documents via XQuery, e.g.

db:add("DB", "embedded7.xml"), db:add("DB", "embedded8.xml")

Cheers

Dirk

On 05/19/2016 08:11 PM, Wang, Genneva wrote:

Hi

I’m trying to figure out whether batched transaction would work for basex, and I came across the RUN and EXECUTE command specified in http://files.basex.org/releases/8.4/BaseX84.pdf page 58-59. The document mentioned following in the summary section:

RUN:Evaluates the contents of file as XQuery expression. If the file ends with the suffix .bxs, the file content will be evaluated as command script. This command can be used to run several commands in a single transaction EXECUTE: Evaluates the specified input as command script. This command can be used to run several commands in a single transaction.

I’ve tried this myself For example, I’d like to know if I batch 2 inserts, if one fails, would both inserts get rollback.

I’m using a simple .bxs (attached and also listed below for your convenience) and run the execute using GUI.

OPEN test ADD TO embedded7.xml <root>embedded</root> ADD TO embedded8.xml <root>embedded CLOSE

The second ADD (embedded8.xml) is not a valid xml format so I’m expecting it to fail, and rollback both files if any of them have already added. But after executing this script, I’m seeing embedded7.xml have been added.

Would you be able to confirm if this is the expected behavior ? If so, is there anyway that I can batch multiple inserts and ensures that these batch of files can be committed at the same time, and rollback all together if any of the file has an error ?

Thank you very much for your help.

-- Dirk Kirsten, BaseX GmbH, http://basexgmbh.de |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22