Hi Srikumar,

3. Without "<?xml version="1.0" encoding="ISO-8859-1"?>" encoding line in XML file, we are failing store XML file(i.e., ISO-8859-1 encoded file) with below error: [FOUP0002] "C:/Sharath/outputxml_06000001800029e2p.xml" (Line 1): Invalid byte 1 of 1-byte UTF-8 sequence.

Have you revised the way how you export files from BaseX? Are you still using String.getBytes() ? If you want your exported docs to be ISO-8859-1-encoded, and if you want to output the XML declaration, you can declare serialization options in the prolog of your query [1]. Best, Christian [1] https://docs.basex.org/wiki/Serialization On Mon, Sep 7, 2020 at 9:47 AM Srikumar Choudhury <schoudhu@opentext.com> wrote:

Hi Christian,

1. Currently by default, we are able to store UTF-8 encoding XML files. 2. Storing of different encoding files are succeeding if we specify the below line: <?xml version="1.0" encoding="ISO-8859-1"?> 3. Without "<?xml version="1.0" encoding="ISO-8859-1"?>" encoding line in XML file, we are failing store XML file(i.e., ISO-8859-1 encoded file) with below error: [FOUP0002] "C:/Sharath/outputxml_06000001800029e2p.xml" (Line 1): Invalid byte 1 of 1-byte UTF-8 sequence. How do we make basex server to store certain format of encoded file(In above example, it is ISO-8859-1 encoded file)? Thanks, Srikumar

________________________________ From: Christian Grün <christian.gruen@gmail.com> Sent: Sunday, September 6, 2020 2:05 PM To: Sharath Babu Pulumati <spulumat@opentext.com> Cc: James Ball <basex-talk@jamesball.co.uk>; BaseX <basex-talk@mailman.uni-konstanz.de>; Rajiv Pandya <rpandya@opentext.com>; Srikumar Choudhury <schoudhu@opentext.com> Subject: Re: [EXTERNAL] - Re: Re. Encoding support for the BaseX server

Please note that your code has various flaws that might lead to the erroneous result. For example, you should avoid calling String.getBytes, as it will use the default character encoding of your platform, not necessarily UTF-8 (see [1]).

I have attached the output that I get for your updated example (it doesn’t trigger any errors). Did you check the encoding of your document that’s output by your Java code (outputxml_06000001800029e2p.xml)?

[1] https://urldefense.com/v3/__https://docs.oracle.com/javase/7/docs/api/java/l...

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ createDatabase: started creating database=DEMO_ENCODING [10:29:20.124] INFO: XQConnection-1 created, the databaseName property was never specified at the XQDataSource level. xqs.getConnection completed [10:29:20.124] TRACE: XQConnection.getStaticContext() invoked. [10:29:20.125] TRACE: XQConnection-1.createExpression() invoked. [10:29:20.183] TRACE: XQConnection.getStaticContext() invoked. [10:29:20.270] DEBUG: XQExpression-1.close() invoked. [10:29:20.270] DEBUG: XQExpression-1.close() invoked. createDatabase: completed creating database=DEMO_ENCODING +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ xqs.getConnection completed [10:29:20.278] TRACE: XQConnection.getStaticContext() invoked. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ StoreDocument: BaseX test store ANSI encoded xml doc [10:29:20.279] TRACE: XQConnection-2.createExpression() invoked. [10:29:20.279] TRACE: XQConnection.getStaticContext() invoked. [10:29:20.308] TRACE: XQExpression-2.executeQuery(), raw XQuery resolved to: db:replace('DEMO_ENCODING', '6300000180000114/06000001800029e2p.xml','c:\Users\user\Desktop\p.xml') xqpe.executeQuery completed [10:29:20.844] DEBUG: XQExpression-2.close() invoked. StoreDocument: BaseX content push for ANSI encoding is completed +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ RetrieveDocument: BaseX test retrieve xml doc [10:29:20.844] TRACE: XQConnection-2.createExpression() invoked. [10:29:20.844] TRACE: XQConnection.getStaticContext() invoked. [10:29:20.844] TRACE: XQExpression-3.executeQuery(), raw XQuery resolved to: db:open('DEMO_ENCODING', '6300000180000114/06000001800029e2p.xml') xqpe.executeQuery completed RetrieveDocument: BaseX test retrieve xml doc completed +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ReStoreDocument: BaseX test store ANSI encoded xml doc [10:29:20.851] TRACE: XQConnection-2.createExpression() invoked. [10:29:20.851] TRACE: XQConnection.getStaticContext() invoked. [10:29:20.851] TRACE: XQExpression-4.executeQuery(), raw XQuery resolved to: db:replace('DEMO_ENCODING', '6300000180000114/06000001800029e2p_re_store.xml','c:\Users\user\Desktop\outputxml_06000001800029e2p.xml') xqpe.executeQuery completed [10:29:20.858] DEBUG: XQExpression-4.close() invoked. ReStoreDocument: BaseX content push for ANSI encoding is completed +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

On Sun, Sep 6, 2020 at 10:19 AM Sharath Babu Pulumati <spulumat@opentext.com> wrote:

...

Hi Christian,

Below is the error stack trace's stating the error during the file uploading using the both BaseX GUI & Sample program:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ By Using the BaseX GUI: Error: Stopped at ., 1/11: [FOUP0002] "C:/Sharath/outputxml_06000001800029e2p.xml" (Line 1): Invalid byte 1 of 1-byte UTF-8 sequence. Optimized Query: db:replace("DEMO_ENCODING", "6300000180000114/06000001800029e2p_re_store.xml", "C:\Sharath\outputxml_06000001800029e2p.xml") Query: db:replace('DEMO_ENCODING', '6300000180000114/06000001800029e2p_re_store.xml','C:\Sharath\outputxml_06000001800029e2p.xml') Query Plan: <QueryPlan compiled="true" updating="false"> <DbReplace name="db:replace" type="empty-sequence()" size="0"> <Str type="xs:string" size="1">DEMO_ENCODING</Str> <Str type="xs:string" size="1">6300000180000114/06000001800029e2p_re_store.xml</Str> <Str type="xs:string" size="1">C:\Sharath\outputxml_06000001800029e2p.xml</Str> </DbReplace> </QueryPlan>

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

By Using the Sample Program: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ createDatabase: started creating database=DEMO_ENCODING [13:30:33.103] INFO: XQConnection-1 created, the databaseName property was never specified at the XQDataSource level. xqs.getConnection completed [13:30:33.103] TRACE: XQConnection.getStaticContext() invoked. [13:30:33.103] TRACE: XQConnection-1.createExpression() invoked. [13:30:33.135] TRACE: XQConnection.getStaticContext() invoked. [13:30:33.432] DEBUG: XQExpression-1.close() invoked. [13:30:33.432] DEBUG: XQExpression-1.close() invoked. createDatabase: completed creating database=DEMO_ENCODING +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ xqs.getConnection completed [13:30:33.432] TRACE: XQConnection.getStaticContext() invoked. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ StoreDocument: BaseX test store ANSI encoded xml doc [13:30:33.447] TRACE: XQConnection-2.createExpression() invoked. [13:30:33.447] TRACE: XQConnection.getStaticContext() invoked. [13:30:33.510] TRACE: XQExpression-2.executeQuery(), raw XQuery resolved to: db:replace('DEMO_ENCODING', '6300000180000114/06000001800029e2p.xml','C:\Sharath\p.xml') xqpe.executeQuery completed [13:30:34.369] DEBUG: XQExpression-2.close() invoked. StoreDocument: BaseX content push for ANSI encoding is completed +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ RetrieveDocument: BaseX test retrieve xml doc [13:30:34.385] TRACE: XQConnection-2.createExpression() invoked. [13:30:34.385] TRACE: XQConnection.getStaticContext() invoked. [13:30:34.400] TRACE: XQExpression-3.executeQuery(), raw XQuery resolved to: db:open('DEMO_ENCODING', '6300000180000114/06000001800029e2p.xml') xqpe.executeQuery completed RetrieveDocument: BaseX test retrieve xml doc completed +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ReStoreDocument: BaseX test store ANSI encoded xml doc [13:30:34.416] TRACE: XQConnection-2.createExpression() invoked. [13:30:34.416] TRACE: XQConnection.getStaticContext() invoked. [13:30:34.416] TRACE: XQExpression-4.executeQuery(), raw XQuery resolved to: db:replace('DEMO_ENCODING', '6300000180000114/06000001800029e2p_re_store.xml','C:\Sharath\outputxml_06000001800029e2p.xml') javax.xml.xquery.XQException: BXCE001 - Stopped at ., 1/11: [FOUP0002] "C:/Sharath/outputxml_06000001800029e2p.xml" (Line 1): Invalid byte 1 of 1-byte UTF-8 sequence. at com.test.BasexTest.ReStoreDocument(BasexTest.java:159) at com.test.BasexTest.main(BasexTest.java:32) ReStoreDocument: BaseX content push for ANSI encoding is completed

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

FYI, I have attached the new sample program along with the snippet collected from BaseX GUI while reproducing the issue. My query is regarding, why i'm unable to import the document which is exported from basex?

Thanks, Sharath.

________________________________ From: Christian Grün <christian.gruen@gmail.com> Sent: Saturday, September 5, 2020 10:44 PM To: Sharath Babu Pulumati <spulumat@opentext.com> Cc: James Ball <basex-talk@jamesball.co.uk>; BaseX <basex-talk@mailman.uni-konstanz.de>; Rajiv Pandya <rpandya@opentext.com>; Srikumar Choudhury <schoudhu@opentext.com> Subject: Re: [EXTERNAL] - Re: Re. Encoding support for the BaseX server

Please find the attached zip, which contains the sample program for replicating the encoding issue with the provided datafiles. Could you please review the provided sample and help me in resolving this issue.

After having created "testenvSSL1" database, and after having fixed your local paths, I ran your example, and it didn’t trigger any errors.

Please give me more information on what you believe goes wrong.