How to decode enums starting with # using the Java SDK ?

vinod.hirani · January 2019

Hi,

I am using the Java version of the Elektron SDK (1.2.2) and am having trouble dealing with these enums.

I am mostly using the EMA library with the FieldEntry:enumDisplay() function to get a string version of the enum. I have also tried to run the outputted string through the RmtesDecoder, and convert to a UCS2 string using the UPA library.

Neither seem to convert the string to something meaningful ( eg I can't get 0x e2 87 a7 from #DE# ) and I seem to get the same string for an uptick or a downtick for fid ACT_TP_1 even though the enumtype.def shows different values.

Any idea how I can get the correct enum display value, even if it's the hexadecimal string that represents it ? I've seen a few questions related to the C++ library but none that show the Java alternative.

Thanks.

veerapath.rungruengrayubkul · January 2019

Hi @vinod.hirani

I am able to replicate the issue in Oracle Linux 6. The issue is that the enumDisplay() returns incorrect value because of default character encoding (in my testing, it is "UTF8"). Previously, I test the issue on Windows with Eclipse which sets "ISO-8859-1" by default, so it returns data correctly.

You can set the Character encoding by providing the file.encoding system property when JVM starts e.g. java -Dfile.encoding="ISO-8859-1"...

veerapath.rungruengrayubkul · January 2019

Hi @vinod.hirani

The enums starting with # means that the hexadecimal value is in RMTES encoding. In this case, the FieldEntry:enumDisplay() returns string contains the hexadecimal value, so you need to convert it to UCS2 using ETA RmtesDecoder. Please note that the #DE# will be converted to "0x21 e7" in UCS2 which means ⇧ according to this table.

Below is the sample code.

/* create cache buffer for storing RMTES and applying partial updates */
RmtesCacheBuffer rmtesCache = CodecFactory.createRmtesCacheBuffer(100);
/* create RmtesBuffer to convert into */
com.thomsonreuters.upa.codec.RmtesBuffer rmtesBuffer =CodecFactory.createRmtesBuffer(100);
/* create RmtesDecoder used for the decoding process */
RmtesDecoder decoder = CodecFactory.createRmtesDecoder();
/*Our Buffer of data we are converting */
Buffer data = CodecFactory.createBuffer();
String enmStr = fieldEntry.enumDisplay();
data.data(java.nio.ByteBuffer.wrap(enmStr.getBytes(java.nio.charset.StandardCharsets.ISO_8859_1)));
// apply RMTES content to cache, if successful convert to UCS-2 
if ((retVal = decoder.RMTESApplyToCache(data, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error while applying to cache 
    System.out.println("Error encountered while applying buffer to RMTES cache. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else if ((retVal = decoder.RMTESToUCS2(rmtesBuffer, rmtesCache)) < CodecReturnCodes.SUCCESS)
{
    // error when converting 
    System.out.println("Error encountered while converting from RMTES to UCS-2. Error code: "
    + CodecReturnCodes.toString(retVal));
}
else
{
 //SUCCESS: Conversion was successful – application can now use converted content stored in rmtesBuffer 
    System.out.println(rmtesBuffer.toString());
}

vinod.hirani · January 2019

Hi Veerapath,

Thanks for that. I used that approach as described in the upa manual.

//SUCCESS: ....

String rmStr = rmtesBuffer.toString();

rmStr.chars().forEach( x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.println(rmStr);

results in

\u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000 ?

So the first two characters are space and a question mark U+0020 and U+003F.

I am being cheeky and running this on linux with OpenJDK and Ubuntu which I know are not officially supported. Do you get the correct U+21E7 character if you run it on a linux system with Oracle JDK and RHEL ?

Thanks,

Vinod

veerapath.rungruengrayubkul · January 2019

Hi @vinod.hirani

I have just tried your code and can get result properly. Could you add code to log the enumeration string returned from the enumDisplay() function, and then share both output and your code?

What if, you can do more test on the supported platform?

Example code:

String enm = fieldEntry.enumDisplay();
System.out.print("Enum String: ");
enm.chars().forEach( x -> {
	System.out.format("\\u%04x ",(int) x);
});

Output sample:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value:
Enum String: \u0042 \u00de 
\u0042 \u21e7 \u0000 \u0000 \u0000 \u0000 \u0000
Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 
Enum String: \u0042 \u00fe 
\u0042 \u21e9 \u0000 \u0000 \u0000 \u0000 \u0000

vinod.hirani · January 2019

Thanks Veerapath,

I've run the code and it does look like the problem is with my version Java and OS. I will see if I can run this on a supported platform. What platform did you successfully run it on btw ? Code and result is below.

Code:

System.out.print("Fid: " + entry.fieldId() + " Name = " + entry.name() + " DataType: " + DataType.asString(entry.load().dataType()) + " Value: " + entry.enumValue() + "\n");

String enm = entry.enumDisplay();

System.out.println(enm);

enm.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

}); System.out.print("\n");

// SUCCESS: Conversion was succesful - application can now use converted content stored in rmtesBuffer

String rmStr = rmtesBuffer.toString();

System.out.println(rmStr);

rmStr.chars().forEach(x -> {

System.out.format("\\u%04x ",(int) x);

});

System.out.print("\n");

Result:

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 2

�

\u0020 \ufffd

? \u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 3

� \u0020 \ufffd

? \u0020 \u003f \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

veerapath.rungruengrayubkul · January 2019

@vinod.hirani

I'm sorry that my reply is not clear. I forgot to mention that I didn't try the code with OpenJDK. I just verify your code on my environment.

I'm looking into your code and will update you again.

vinod.hirani · January 2019

Hi Veerapath,

I understand you didn't try it with OpenJDK. What platform and JVM did you use for your testing ?

Thanks,

Vinod

vinod.hirani · January 2019

Hi Veerapath,

Excellent. Using the new default character encoding I see the right unicode \u21e7 and \u21e9 ( after using the RMTESDecoder). So the result now ( using the same code as before with the -Dfile.encoding="ISO-8859-1" ) gives the result :

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 2

\u0020 \u00de

?

\u0020 \u21e7 \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Fid: 270 Name = ACT_TP_1 DataType: Enum Value: 3

�

\u0020 \u00fe

?

\u0020 \u21e9 \u0000 \u0000 \u0000 \u0000 \u0000 \u0000

Thanks very much that's been a great help !!

Vinod

vinod.hirani · January 2019

Thanks Veerapath, please keep me in the loop of any EMA changes via email.

bennetcole · August 2019

How to decode enums starting with # using the Java SDK ?

Best Answer

Answers

Categories