Elektron EMA API switch failover
The situation occurred last night, but the circuit of another edge server (10.102.210.137) is normal. If this happens, can the EMA API automatically switch to the host of the normal circuit? We have settings, but didn't switch last night
Case#: 06840766
Impacted service: Elektron Edge
Impact: LOSS OF RESILIENCY
Impacted device(s): EDGETAI0109
Circuit Details: HUTCHISON_GLOBAL_COMMUNICATIONS circuit PEP9201605
Outage Start Time: 8/15/2018 19:10 UTC
Service resumed at 20:16 UTC (15 Aug 2018).
Client API received:
2018-08-16 03:09:59 INFO[139766029948672]: 17033 REUTERS: TACNT1YDF= : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766728234752]: 17033 REUTERS: HKDTWD=R : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766061152000]: 17033 REUTERS: 0#ES: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766087546624]: 17033 REUTERS: 0#STW: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766758635264]: 17033 REUTERS: .TOPX : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766029948672]: 17084 REUTERS: TACNT1YDF= : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766061152000]: 17084 REUTERS: 0#ES: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766728234752]: 17084 REUTERS: HKDTWD=R : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766087546624]: 17084 REUTERS: 0#STW: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766758635264]: 17084 REUTERS: .TOPX : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766029948672]: 17033 REUTERS: TACNT6MDF= : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766061152000]: 17033 REUTERS: 0#JY: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766087546624]: 17033 REUTERS: 0#AD: : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766728234752]: 17033 REUTERS: KRW= : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766029948672]: 17084 REUTERS: TACNT6MDF= : Open / Suspect / None / 'Comms Outage'
2018-08-16 03:09:59 INFO[139766061152000]: 17084 REUTERS: 0#JY: : Open / Suspect / None / 'Comms Outage'
...................
2018-08-16 04:16:07 INFO[139766728234752]: 17033 REUTERS: HKDTWD=R : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766029948672]: 17033 REUTERS: TACNT1YDF= : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766758635264]: 17033 REUTERS: .TOPX : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766728234752]: 17084 REUTERS: HKDTWD=R : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766061152000]: 17084 REUTERS: 0#ES: : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766029948672]: 17084 REUTERS: TACNT1YDF= : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766758635264]: 17084 REUTERS: .TOPX : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766728234752]: 17033 REUTERS: KRW= : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766061152000]: 17033 REUTERS: 0#JY: : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766087546624]: 17084 REUTERS: 0#STW: : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766029948672]: 17033 REUTERS: TACNT6MDF= : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766728234752]: 17084 REUTERS: KRW= : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766758635264]: 17033 REUTERS: 0#JTI: : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766061152000]: 17084 REUTERS: 0#JY: : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:07 INFO[139766029948672]: 17084 REUTERS: TACNT6MDF= : Open / Suspect / None / 'Comms Restored'
...............
2018-08-16 04:16:22 INFO[139766061152000]: 17084 REUTERS: ESM9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766087546624]: 17084 REUTERS: 0#STW: : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766758635264]: 17084 REUTERS: 1093.HKd : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:22 INFO[139766029948672]: 17033 REUTERS: SINV8 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766061152000]: 17033 REUTERS: JYX8 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766087546624]: 17033 REUTERS: 0#AD: : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766061152000]: 17084 REUTERS: JYX8 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766758635264]: 17033 REUTERS: 83188.HKd : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:22 INFO[139766029948672]: 17084 REUTERS: SINV8 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766087546624]: 17084 REUTERS: 0#AD: : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766758635264]: 17084 REUTERS: 83188.HKd : Open / Suspect / None / 'Comms Restored'
2018-08-16 04:16:22 INFO[139766029948672]: 17033 REUTERS: SINH9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766061152000]: 17033 REUTERS: ESU8 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:22 INFO[139766087546624]: 17033 REUTERS: 0#BP: : Open / No Change / Failover started / 'Failover Started'
..................
2018-08-16 04:16:49 INFO[139766758635264]: 17084 REUTERS: DJTIU8-H9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:49 INFO[139766061152000]: 17033 REUTERS: ESH9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766087546624]: 17084 REUTERS: STWU8 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766758635264]: 17033 REUTERS: JTIU8-M9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:49 INFO[139766061152000]: 17084 REUTERS: ESH9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766087546624]: 17033 REUTERS: STWZ8 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766758635264]: 17084 REUTERS: JTIU8-M9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:49 INFO[139766061152000]: 17033 REUTERS: ESM9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766087546624]: 17084 REUTERS: STWZ8 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766758635264]: 17033 REUTERS: DJTIU8-M9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:49 INFO[139766061152000]: 17084 REUTERS: ESM9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766087546624]: 17033 REUTERS: STWH9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766758635264]: 17084 REUTERS: DJTIU8-M9 : Open / No Change / Failover started / 'Failover Started'
2018-08-16 04:16:49 INFO[139766061152000]: 17033 REUTERS: JYX8 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766087546624]: 17084 REUTERS: STWH9 : Open / Ok / None / 'OK'
2018-08-16 04:16:49 INFO[139766758635264]: 17033 REUTERS: SPU8 : Open / No Change / Failover started / 'Failover Started'
..................
2018-08-16 04:17:12 INFO[139766758635264]: 17033 REUTERS: SPU8 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17084 REUTERS: SPU8 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17033 REUTERS: SPU9 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17084 REUTERS: SPU9 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17033 REUTERS: SPM0 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17084 REUTERS: SPM0 : Open / No Change / Failover completed / 'Failover Completed'
2018-08-16 04:17:12 INFO[139766758635264]: 17033 REUTERS: SPZ9 : Open / No Change / Failover completed / 'Failover Completed'
................
2018-08-16 04:17:31 INFO[139766758635264]: 17033 REUTERS: .TOPX : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17084 REUTERS: .TOPX : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17033 REUTERS: 0#JTI: : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17084 REUTERS: 0#JTI: : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17033 REUTERS: TWD= : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17084 REUTERS: TWD= : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17033 REUTERS: 0#SP: : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17084 REUTERS: 0#SP: : Open / Ok / None / 'OK'
2018-08-16 04:17:31 INFO[139766758635264]: 17033 REUTERS: .SP500 : Open / Ok / None / 'OK'
2018-08-16 04:17:32 INFO[139766758635264]: 17084 REUTERS: .SP500 : Open / Ok / None / 'OK'
2018-08-16 04:17:32 INFO[139766758635264]: 17033 REUTERS: 0#SSI: : Open / Ok / None / 'OK'
EmaConfig:
<ChannelSet value="Channel_1, Channel_2"/>
Best Answer
-
Do not use StatusCodes to determine the application action.
The status code is provided as added information and does not represent the actual state of the stream and data.
You will need only the Stream State and Data State when determining the application action.
void AppClient::onStatusMsg(const StatusMsg& statusMsg,const OmmConsumerEvent&)
{
if (statusMsg.hasState()
&& statusMsg.hasName()
&& statusMsg.getDomainType() == MMT_MARKET_PRICE)
if (statusMsg.getState().getStreamState()==OmmState::StreamState::OpenEnum
&&statusMsg.getState().getDataState()==OmmState::DataState::SuspectEnum)
{
//Data Suspect
//Perform switchover to second stream
}
}The above code check if Stream is open and Data is suspect, which is the first two part of "Open / Suspect / None / 'Comms Outage'"
Stream Open and Data Suspect means data from the upstream device (in this case Elektron Edge) is not available. However, the data stream is still open. Elektron Edge is trying to recover data and will start sending the data again once recovered.
Since you want to switch over to second Edge device when data suspect, you will also have to determine;
- What will you do to the data stream of the first Edge device after switchover, will you close the stream or leave it open?
If you leave it open you will start receiving the data again when the first Edge device recover. How will you handle it? - What will you do if the second Edge device is Suspect too?
The answer for these are two points depends on your specification.
0 - What will you do to the data stream of the first Edge device after switchover, will you close the stream or leave it open?
Answers
-
Hi @daniel76
There are situations that EMA can and cannot switch over.
Situation 1: EMA disconnect from Elektron Edge.
EMA will switch to the next channel on the ChannelSet configuration. See EMA config guide for the ChannelSet configuration.Situation 2: Elektron Edge disconnect from Elektron Service.
EMA will not switch. Elektron Edge will send the comm outage status message to EMA but the connection between EMA and Elektron Edge will remain. EMA will start receiving the data again after Elektron Edge recover their connection to Elektron Service.Normally, we would suggest handling Situation 2 by having a distribution server (TREP) between EMA and Elektron Edges. TREP has a feature that allows it connect to multiple Edge device and automatically rotate the downstream request to the next available host.
Without TREP, the application must implement the feature by itself.
0 -
I'm afraid I don't have an example as it depends on how seamlessly you want for the transition.
For a perfectly seamless switch, you can always connect to both Edge servers and send the requests to both Edges but then use the response from the first Edge only. This way, you will have zero downtime at the cost of performance.
Or you can connect to both Edges but send the request to the first Edge only. You then send the request to the second Edge when you receive the data suspect status.
Or connect to the second Edge only after receiving the data suspect status.
But either way, you need two OmmConsumer objects, each use different Channel, rather than one OmmConsumer object that uses ChannelSet. You specify consumer name when you create OmmConsumer.
OmmConsumer consumer_1( OmmConsumerConfig().consumerName( "Consumer_1" ) );
OmmConsumer consumer_2( OmmConsumerConfig().consumerName( "Consumer_2" ) );Your Consumer_1 should use Channel_1 which point to the first Edge, while Consumer_2 should use Channel_2 that point to the second Edge.
EMA will notify that the data was suspect in the onStatusMsg callback which you should use statusMsg.getState() to verify the state first. Then it is up to you how to manage the switchover.
0 -
If I use the statusMsg.getState() method, which StatusCodes need to be processed? (eg connecting to the second Edg Server)
Also, the string "Open / Suspect / None / 'Comms Outage'" is not found in the following documentation. Which of the following StatusCode is this?
enum StatusCode {
NoneEnum = 0,
NotFoundEnum = 1,
TimeoutEnum = 2,
NotAuthorizedEnum = 3,
InvalidArgumentEnum = 4,
UsageErrorEnum = 5,
PreemptedEnum = 6,
JustInTimeConflationStartedEnum = 7,
TickByTickResumedEnum = 8,
FailoverStartedEnum = 9,
FailoverCompletedEnum = 10,
GapDetectedEnum = 11,
NoResourcesEnum = 12,
TooManyItemsEnum = 13,
AlreadyOpenEnum = 14,
SourceUnknownEnum = 15,
NotOpenEnum = 16,
NonUpdatingItemEnum = 19,
UnsupportedViewTypeEnum = 20,
InvalidViewEnum = 21,
FullViewProvidedEnum = 22,
UnableToRequestAsBatchEnum = 23,
NoBatchViewSupportInReqEnum = 26,
ExceededMaxMountsPerUserEnum = 27,
ErrorEnum = 28,
DacsDownEnum = 29,
UserUnknownToPermSysEnum = 30,
DacsMaxLoginsReachedEnum = 31,
DacsUserAccessToAppDeniedEnum = 32,
InvalidFormedMsgEnum = 256,
ChannelUnavailableEnum = 257,
ServiceUnavailableEnum = 258,
ServiceDownEnum = 259,
ServiceNotAcceptingRequestsEnum = 260,
LoginClosedEnum = 261,
DirectoryClosedEnum = 262,
ItemNotFoundEnum = 263,
DictionaryUnavailableEnum = 264,
FieldIdNotFoundDictionaryUnavailableEnum = 265,
ItemRequestTimeoutEnum = 266
}0 -
Hi @Warat B.
If I want to use perfectly seamless switch, and always connect to both Edge servers and send the requests to both Edges but then use the response from the first Edge only.
How do I determine if onRefreshMsg and onUpdateMsg are the first message, and filter duplicate messages? (Is there a same serial number to decide? Example code ?)0 -
When you call registerClient() it will return item identifier, a.k.a. handle.
Then you can determine the item during the event callback by comparing the handle from OmmConsumerEvent.getHandle().
Alternatively, you can include a closure in the registerClient() and the get the closure using OmmConsumerEvent.getClosure().
0 -
Hi @Warat B.
By the way , If situation 1: EMA disconnect from Elektron Edge.
EMA will switch to the next channel on the ChannelSet configuration. See EMA config guide for the ChannelSet configuration.
In this case, the already registered symbol is still auto-onUpdateMsg and does not need to be re-registered, right? Or need to do something ? We don't have server permissions, how can we test this?0 -
Hi @daniel76
It does not need to be re-registered.
You can test this with the EMA IProvider example. Just run two IProviders and have your EMA consumer connect to them. Kill the first IProvider and you will see it switch to the second without the need of re-register.
0 -
Hi @daniel76
Open / Suspect / None / 'Item is stale' means there was a problem with the <1ADF9m> data and recovery is being managed by the server. You may have to check the server log for more information.
But from an API point of view, it just has to wait and the server will recover the data for them.
Again as I mentioned in the previous post, you will need only the Stream State and Data State when determining the application action.
Stream Open and Data Suspect means data from the upstream device is not available and upstream is recovering the data.
0 -
Here is the Item State Decision Table.
This table can be found in the Elektron Transport API dev guide.0 -
I am afraid they aren't designed to be handled differently.
First, please understand that a message status consists of four members.
Memebers: "Stream State / Data State / Status Code / Status Text"
Data Type: "Enum / Enum / Enum / String "From the API point of view, the only difference between the two messages is the status text, which is a string that meant to be shown on screen or in a log. And unlike enum, the string can be changed in the future.
0
Categories
- All Categories
- 6 AHS
- 39 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 370 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 60 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛