After cloning of onUpdateMsg my memory utilization is very high it is going up to 16gb.
After cloning of onUpdateMsg my memory utilization is very high it is going up to 16gb. Below is my code of cloning update message. Is there anything wrong ?
Thank you!
Best Answer
-
Hello @MayassK
You use EmaFactory.createUpdateMsg(other) to clone all messages and decode them inside the callback method i.e. onUpdateMsg(..). This can causes memory growth problem which you are facing. The goal of cloning message is to be able to decode the message's payload outside of message callback methods.
To protect memory growth problem, you should minimize the time spent processing incoming data in the callback methods i.e. onRefreshMsg(..) and onUpdateMsg(..). Avoid making database and/or other blocking calls in the methods. The application can use an additional thread to process the received messages separately. Therefore, EMA thread spends less time in the callback methods for each data messages and it can handle the huge data messages in a timely manner. You can use the cloning functionality for message types to copy the entire encoded buffer to another objects. This would help to cache and process the EMA message types outside of the callback methods e.g. onRefreshMsg(..), onUpdateMsg(..),onStatusMsg(..). com.thomsonreuters.ema.access.EmaFactory class provides the following methods to create a clone of message:
- createAckMsg(AckMsg other)
- createGenericMsg(GenericMsg other)
- createPostMsg(PostMsg other)
- createRefreshMsg(RefreshMsg other)
- createReqMsg(ReqMsg other)
- createStatusMsg(StatusMsg other)
- createUpdateMsg(UpdateMsg other)
Hope this help.
0
Answers
-
Thank you Pimchaya for your response!
I followed the below example provided on GIT for cloning
For database calls i am using additional class threads which are processing separately.
Can you please provide me a relevant cloning example?
0 -
Should I start a separate thread like below?
public void onUpdateMsg(UpdateMsg updateMsg, OmmConsumerEvent event) {
try {
UpdateMsg cloneUpdateMsg = EmaFactory.createUpdateMsg(updateMsg);
new decode(cloneUpdateMsg).start();
} catch (Exception e) {
loggerInfo.error("on update MSG error" + e.getMessage());
}
}
0 -
Hello @MayassK
Yes. You should start a separate thread after cloning the message. However, a new thread should not be created every time an update message is received. Your CPU only maybe 4 or 8 cores, so it can't run more than that many threads at the same time anyway. It's not useful to have many threads e.g. 15 because most of them will be waiting until a CPU core is free to run them. Each thread has a call stack which spends memory. That's mean more threads use more memory.
You should use a thread pool instead. Java has a whole framework for thread pools in the java.util.concurrent package e.g. ExecutorService which can be created and limited the number of threads using Executors class.
1 -
Hi @MayassK
To manage memory when you clone messages, you need to ensure the specific messages you are cloning are unreachable after you are done with them to ensure they are eligible for Java garbage collection. If you are following these guidelines and memory is still peaking, you may need to manually force garbage collection (System.gc() / Runtime.getRuntime().gc()) periodically to reduce the memory stress.
1 -
Thank you Pimchaya and Nick,
Now i am using thread pool to call the decode method.
But i am getting blank value for cloneUpdateMsg object in decode method.
Is this the correct way i am following in below code
public void onUpdateMsg(UpdateMsg updateMsg, OmmConsumerEvent event) {
try {
UpdateMsg cloneUpdateMsg = EmaFactory.createUpdateMsg(updateMsg);
Runnable decodeUpdate =new ProcessDecode(updateMsg);
this.pool.execute(decodeUpdate);
} catch (Exception e) {
loggerInfo.error("on update MSG error" + e.getMessage());
}
}
class ProcessDecode implements Runnable{
UpdateMsg cloneUpdateMsg;
public ProcessDecode(UpdateMsg updateMsg) {
this.cloneUpdateMsg=updateMsg;
}
public void run() {
if (DataType.DataTypes.FIELD_LIST == cloneUpdateMsg.payload().dataType() && cloneUpdateMsg.hasName() && cloneUpdateMsg.hasServiceName()) {
decode(cloneUpdateMsg.payload().fieldList(), cloneUpdateMsg);
}
}
}
0 -
Hello @MayassK
Runnable decodeUpdate =new ProcessDecode(updateMsg);
The source code above is to set update message not the clone of update message for a thread. The source code should be:
Runnable decodeUpdate =new ProcessDecode(cloneUpdateMsg);
This above source code is to set the clone of update message for the thread.
0
Categories
- All Categories
- 6 AHS
- 39 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 60 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛