Automated testing for XML databases
Greetings! I'm Shuxin, an undergraduate CS student, who is currently an intern at NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these! I would like to ask whether you find these reports useful and would also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks! Best Regards, Shuxin Li
Dear Shuxin Li, Thanks for sharing the bug reports with us. I appreciate that, they’re definitely helpful. You could as well think about creating and sending pull requests for your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations. I'm looking forward to the upcoming reports. Thanks again, Christian [1] https://github.com/w3c/qt3tests Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023, 10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern at NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
Dear Shuxin Li, I’m impressed by how you repeatedly manage to discover new bugs in BaseX. How do you proceed to uncover the errors? Is it a trial-and-error approach, or are the tests generated automatically? Do you/does your group plan to write an article that describes your approach? All the best from Germany, Christian On Fri, Mar 17, 2023 at 10:22 AM Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
Thanks for sharing the bug reports with us. I appreciate that, they’re definitely helpful.
You could as well think about creating and sending pull requests for your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations.
I'm looking forward to the upcoming reports.
Thanks again, Christian
[1] https://github.com/w3c/qt3tests
Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023, 10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern at NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
Hi Shuxin Li, It certainly is very impressive work and I am also keen to know more. I see you have also been working with eXist [1] I assume it builds on "Query Plan Guidance (QPG), a test case generation method with the guidance of query plans."? [2] Best wishes /Andy [1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin [2] https://zenodo.org/record/7623847 On Thu, 30 Mar 2023 at 10:06, Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
I’m impressed by how you repeatedly manage to discover new bugs in BaseX. How do you proceed to uncover the errors? Is it a trial-and-error approach, or are the tests generated automatically? Do you/does your group plan to write an article that describes your approach?
All the best from Germany, Christian
On Fri, Mar 17, 2023 at 10:22 AM Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
Thanks for sharing the bug reports with us. I appreciate that, they’re
definitely helpful.
You could as well think about creating and sending pull requests for
your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations.
I'm looking forward to the upcoming reports.
Thanks again, Christian
[1] https://github.com/w3c/qt3tests
Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023, 10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern at
NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would
also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
TEST's about statement is now available in StratML (ISO 17469-1) format at https://stratml.us/drybridge/index.htm#TEST Soon it will be discoverable in the query service at https://search.aboutthem.info/ running on BaseX. I look forward to learning whether TEST can help identify and resolve issues with the full-text query capability of BaseX, particularly with respect to the treatment of spaces between words. I don't see any evidence of that at https://nus-test.github.io/bugs/ Owen Amburhttps://www.linkedin.com/in/owenambur/ On Thursday, March 30, 2023 at 06:27:49 AM EDT, Andy Bunce <bunce.andy@gmail.com> wrote: Hi Shuxin Li, It certainly is very impressive work and I am also keen to know more. I see you have also been working with eXist [1] I assume it builds on "Query Plan Guidance (QPG), a test case generation method with the guidance of query plans."? [2] Best wishes/Andy [1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin[2] https://zenodo.org/record/7623847 On Thu, 30 Mar 2023 at 10:06, Christian Grün <christian.gruen@gmail.com> wrote: Dear Shuxin Li, I’m impressed by how you repeatedly manage to discover new bugs in BaseX. How do you proceed to uncover the errors? Is it a trial-and-error approach, or are the tests generated automatically? Do you/does your group plan to write an article that describes your approach? All the best from Germany, Christian On Fri, Mar 17, 2023 at 10:22 AM Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
Thanks for sharing the bug reports with us. I appreciate that, they’re definitely helpful.
You could as well think about creating and sending pull requests for your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations.
I'm looking forward to the upcoming reports.
Thanks again, Christian
[1] https://github.com/w3c/qt3tests
Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023, 10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern at NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
Dear Shuxin, I just wanted to take the opportunity to thank you for the bug reports you've submitted - both here and to the eXist-db project, as Andy noted. I really like Christian's suggestion to contribute your test cases to the official W3C test suite. But whichever means you choose, we welcome your reports and thank you for your team's work on improving these great open source tools. Joe On Thu, Mar 30, 2023 at 6:27 AM Andy Bunce <bunce.andy@gmail.com> wrote:
Hi Shuxin Li,
It certainly is very impressive work and I am also keen to know more.
I see you have also been working with eXist [1] I assume it builds on "Query Plan Guidance (QPG), a test case generation method with the guidance of query plans."? [2]
Best wishes /Andy
[1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin [2] https://zenodo.org/record/7623847
On Thu, 30 Mar 2023 at 10:06, Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
I’m impressed by how you repeatedly manage to discover new bugs in BaseX. How do you proceed to uncover the errors? Is it a trial-and-error approach, or are the tests generated automatically? Do you/does your group plan to write an article that describes your approach?
All the best from Germany, Christian
On Fri, Mar 17, 2023 at 10:22 AM Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
Thanks for sharing the bug reports with us. I appreciate that, they’re
definitely helpful.
You could as well think about creating and sending pull requests for
your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations.
I'm looking forward to the upcoming reports.
Thanks again, Christian
[1] https://github.com/w3c/qt3tests
Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023,
10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern at
NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would
also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
Dear Christian, Andy, Owen Ambur and Joe, Thank you so much for your emails! It's really great to hear from you that you are interested in our project and appreciate the bug report. It is also a big encouragement for us. The tests are automatically generated, and the approach we currently take is to have the results compared between different engines. We do have the plan to write an article on this and release the tool. The tool should have the ability of performing regression tests too, and if you are interested and that it might be helpful, we could help to try and integrate the tool as part of Continuous Integration too (even before publishing the paper). Contributing to the W3C test suite to make the tests helpful also sounds great to us. We had planned to collect a few more bug inducing tests and will submit them to the W3C test suite together. Whether we could test BaseX's ability on full text search is also very interesting! We could try to look into it and see if it is possible (might take some time since it might require different test oracles I guess!). We had just started this work on testing XML databases not long ago, therefore we haven't uploaded the latest information on the website yet, and perhaps we will update it after the article is published. Thank you all again for your appreciation and suggestions! Best regards, Shuxin Li On Fri, Mar 31, 2023 at 12:17 AM Joe Wicentowski <joewiz@gmail.com> wrote:
Dear Shuxin,
I just wanted to take the opportunity to thank you for the bug reports you've submitted - both here and to the eXist-db project, as Andy noted. I really like Christian's suggestion to contribute your test cases to the official W3C test suite. But whichever means you choose, we welcome your reports and thank you for your team's work on improving these great open source tools.
Joe
On Thu, Mar 30, 2023 at 6:27 AM Andy Bunce <bunce.andy@gmail.com> wrote:
Hi Shuxin Li,
It certainly is very impressive work and I am also keen to know more.
I see you have also been working with eXist [1] I assume it builds on "Query Plan Guidance (QPG), a test case generation method with the guidance of query plans."? [2]
Best wishes /Andy
[1] https://github.com/eXist-db/exist/issues/created_by/Twilight-Shuxin [2] https://zenodo.org/record/7623847
On Thu, 30 Mar 2023 at 10:06, Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
I’m impressed by how you repeatedly manage to discover new bugs in BaseX. How do you proceed to uncover the errors? Is it a trial-and-error approach, or are the tests generated automatically? Do you/does your group plan to write an article that describes your approach?
All the best from Germany, Christian
On Fri, Mar 17, 2023 at 10:22 AM Christian Grün <christian.gruen@gmail.com> wrote:
Dear Shuxin Li,
Thanks for sharing the bug reports with us. I appreciate that, they’re
definitely helpful.
You could as well think about creating and sending pull requests for
your test cases to the official W3 test suite [1]. This way, all implementors of the standard who run the tests may benefit from your observations.
I'm looking forward to the upcoming reports.
Thanks again, Christian
[1] https://github.com/w3c/qt3tests
Shuxin Li <shuxin.li.lv@gmail.com> schrieb am Fr., 17. März 2023,
10:01:
Greetings!
I'm Shuxin, an undergraduate CS student, who is currently an intern
at NUS Test lab. I’ve been working on an automated testing tool for XML processors and databases. My goal is to find bugs in important systems, in order to improve these systems’ reliability. In the past few days, I submitted a few bug reports on Github (Github id: Twilight-Shuxin). Thanks for having already fixed these!
I would like to ask whether you find these reports useful and would
also appreciate future reports if I find more bugs? Any feedback for us is highly appreciated. We also plan to open-source this tool for public usage after it's fully realized, and truly hope that it might be of help to the community. Thanks!
Best Regards, Shuxin Li
participants (5)
-
Andy Bunce -
Christian Grün -
Joe Wicentowski -
Owen Ambur -
Shuxin Li