DSS .NET SDK performance issues - ondemand extract

mithun.ghosh
mithun.ghosh
in DSS

We are trying to fetch EOD prices through on-demand extraction .NET SDK for more than 10k instruments.This takes 20+ minutes to complete which is almost 10 times longer than time taken while using http endpoint.

Is it to be expected or we can do something to speed it up. If you need more information, please let us know.


Regards

Mithun

Tagged:

Best Answer

  • Jirapongse
    Jirapongse
    Answer ✓

    @mithun.ghosh

    From the code, you are using the ExtractWithNotes endpoint. Therefore, the response should contain extraction information. For example:

      "Notes": [
            "Extraction Services Version 13.3.41369 (50b192356928), Built Feb  3 2020 17:30:35\r\nHoliday Rollover of Universal Close Price waived.\r\nProcessing started at 03/19/2020 05:50:17.\r\nUser ID: 9008895\r\nExtraction ID: 424491044\r\nSchedule: _OnD_0x07065045ef517094 (ID = 0x07065045fba17094)\r\nInput List (2 items): _OnD_0x07065045ef517094 (ID = 07065045f4917094) Created: 03/19/2020 05:50:15 Last Modified: 03/19/2020 05:50:16\r\nSchedule Time: 03/19/2020 05:50:16\r\nReport Template (73 fields): _OnD_0x07065045ef517094 (ID = 0x07065045f0417094) Created: 03/19/2020 05:50:15 Last Modified: 03/19/2020 05:50:15\r\n(CSP,438516AC0,CPN) is inactive.\r\nColumn 'CIN Code' suppressed for lack of 'CIN Code' permission.\r\nProcessing completed successfully at 03/19/2020 05:50:18, taking 0.728 Secs.\r\nExtraction finished at 03/19/2020 05:50:18 UTC, with servers: x04A06, QSHC16 (0.3 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User 9008895, Client 65508, Template Type EOD Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      1 Corporate                                                        N/A            N/A\r\n      1 Equities                                                         N/A            N/A\r\n-------\r\n      2 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\n"
        ]

    From this information, you can compare the processing time between .NET and HTTP.

Answers

  • veerapath.rungruengrayubkul
    veerapath.rungruengrayubkul

    Hi @mithun.ghosh,

    Could you share the snippet code for .Net SDK and request's body for http endpoint? So, I can try to replicate the issue?

  • mithun.ghosh
    mithun.ghosh

    Hi Veerapath,

    Thanks for your prompt response.

    Please find the .Net SDK code below and attribute list attached with this mail.

    public ExtractionResult CreateAndRunEodPricingExtraction(

    InstrumentIdentifier[] instrumentIdentifiers, string[] contentFieldNames)

    {

    //Create an on demand extraction, of type EndOfDayPricingExtractionRequest:

    PremiumEndOfDayPricingExtractionRequest extractionEod = new PremiumEndOfDayPricingExtractionRequest

    {

    IdentifierList = InstrumentIdentifierList.Create(instrumentIdentifiers),

    ContentFieldNames = contentFieldNames

    };

    //Run the extraction.

    //This call is blocking, it returns when the extraction is completed:

    return extractionsContext.ExtractWithNotes(extractionEod);

    }

    EOD_ATTRIBUTE_LIST.txt

    We have tried to fetch EOD prices through on-demand extraction with more than 8k instruments. But for the security reason we will not able to share the instrument list with you.

    Find the sample one instrument list below :


    Identifier Type, Identifier Code,BB_UNIQUE_ID

    ISN,US0276XAA,COEK4854

  • veerapath.rungruengrayubkul
    veerapath.rungruengrayubkul

    Hi @mithun.ghosh,

    The ExtractWithNotes() function you are using with the .Net SDK will return when extraction is completed and data is received. Whereas the ExtractWithNotes HTTP endpoint uses Asynchronous mechanism which will return 202 status with Location URL if the request takes too long to process. Application needs to poll the location URL to get data when the extraction is completed.

    Could you please confirm whether the time you get data with HTTP endpoint is 10 times faster than .Net SDK?

    Also, as suggestion by Jirapongse, you can compare the Note information generated by .Net SDK and HTTP endpont. The Note information contains the time when extraction is started and completed.

Categories