EMA Java: "channel out of buffers errorId=-4" error with over 3,000 RICs

skajiwara
skajiwara
in EMA

Hi,

I'm using EMA Java from Scala 2.11.

I `registerClient()` 3,558 RICs at once by the following code:

val client = new MyClient()  // sub-class of OmmConsumerClient

val rics = List("/1418.T", ...)  // Ordinary stocks in Tokyo Stock Exchange, whose length is 3558

rics.map { ric =>
  consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name(ric), client)}

Although `MyClient#on*Msg()` have almost empty implementation, my program uses a lot of memory (memory usage of the process is around 6GB on 16GB-memory OSX).

It leads to the following runtime error:

19:17:15.024 [pool-2-thread-1] ERROR c.t.e.a.OmmConsumerImpl - loggerMsg
    ClientName: EmaConsumer_1
    Severity: Error
    Text:    Call to rsslReactorDispatchLoop() failed. Internal sysError='0' Error text='channel out of buffers errorId=-4 errorText=channel out of buffers'. 
loggerMsgEnd

My questions are:

  • Is it usual to consume such a large memory with 3,558 RICs registration?
    • If it's unusual, I will use some heap profiler (no idea currently) to detect potential memory leak.
    • If it's usual, could you tell me about how much memory would be required to get 3,558 RICs' feed? I appreciate if I can hear some reasonable tuning points.

Thank you.

Tagged:

Best Answer

  • Jirapongse
    Jirapongse
    Answer ✓
    @skajiwara

    It could be the size of GuaranteedOutputBuffers used by the underlying API (ETA).

    image

    You can try to increase its value via the configuration. For example:

    <ChannelGroup>
    	<ChannelList>
    		<Channel>
    			<Name value="Channel_1"/>	
    			...
    			<GuaranteedOutputBuffers value="5000"/>
    			<Host value="hostname"/>
    			<Port value="14002"/>
    				...
    		</Channel>

Answers

  • Nipat Kunvutipongsak
    Nipat Kunvutipongsak

    @skajiwara It seems like the problem didn't occur when the application tried to send a batch request with a lot of items, but the problem happened after that 'request' state (when the application received data), am I correct?

    I have replicated this problem by putting 4000 items into a request message with EMA Java version emaj3.0.3.L1.all (from Elektron-SDK1.0.7java). However, it encountered the message below only.

    "Failed to rsslElementEntry.encode() while encoding rssl elementlist. Reason='INVALID_DATA'"

    Could you please confirm whether my understanding is correct or not?

  • Nipat Kunvutipongsak
    Nipat Kunvutipongsak

    Also, can I know the version of EMA/Elektron-SDK the application is using? Perhaps, I can't reproduce the problem because of different API version. Thanks.

  • jerome.raghunandan
    jerome.raghunandan

    Hi, I have the same error with the API version below. I don't have a EmaConfig.xml file. How can I increase the GuaranteedOutputBuffers in this version without using an EmaConfig.xml file?

    <groupId>com.thomsonreuters.ema</groupId>
    <artifactId>ema</artifactId>
    <version>3.2.0.2</version>


    This is the error I have:

    ClientName: SingleItem
    Severity: Error
    Text: Internal error: rsslChannel.submit() failed in SingleItem.submit(PostMsg)RsslChannel 0
    Error Id -3
    Internal sysError 0
    Error Location Watchlist.encodeIntoBufferAndQueue
    Error Text channel out of buffers errorId=-4 errorText=channel out of buffers
  • jerome.raghunandan
    jerome.raghunandan

    EmaConfig.zip

    I managed to resolve the issue by creating an EmaConfig.xml file and placing it in the root folder of my application. I found that is the default location the API looks in for the file. I set the GuaranteedOutputBuffers in the file and that fixed the issue. I have attached an example of the EmaConfig.xml file I created.

Categories