Avoiding empty rows during RTH extraction
I'm trying to extract historical RTH data. I'm getting a LOT of blank rows which take up lots of memory and slow everything down. These are days that are weekends, holidays, or even days where the underlying RIC was not listed/trading at all. Is there a way to avoid returning those rows at all? Seems like a needless waste for everyone involved.
Best Answer
-
I would like to share the answer from the Tick History support team regarding this question.
To avoid blank rows in RTH extraction, we can set TimebarPersistence to false in the request, this will remove the blank rows and weekend data. Time bar Persistence is applicable for all instruments irrespective of Content sets.
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryIntradaySummariesExtractionRequest",
"ContentFieldNames": [
"Close Ask",
"Close Bid",
"High",
"High Ask",
"High Bid",
"Last",
"Low",
"Low Ask",
"Low Bid",
"No. Asks",
"No. Bids",
"No. Trades",
"Open",
"Open Ask",
"Open Bid",
"Volume"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{
"Identifier": "USH0",
"IdentifierType": "Ric"
}
],
"UseUserPreferencesForValidationOptions": "false",
"ValidationOptions": {
"AllowHistoricalInstruments": true,
"AllowInactiveInstruments": true
}
},
"Condition": {
"MessageTimeStampIn": "GmtUtc",
"ReportDateRangeType": "Range",
"QueryStartDate": "1997-06-20T00:00:00.000Z",
"QueryEndDate": "2020-06-23T00:00:00.000Z",
"SummaryInterval": "OneHour",
"TimebarPersistence": false,
"DisplaySourceRIC": true
}
}
}Tick History Intraday Summaries extraction provides transaction messages constructed on the fly from either Trade or Quote messages, summarized into time-series intervals representing aggregations at 1 second, 5 seconds, 1 minute, 5 minutes, 10 minutes, 15 minutes, and 60 minutes. Summaries are created based on rules (Refinitiv Intellectual Property) varying on a per-exchange basis and cannot be shared.
In Refinitiv Tick History v2 Intraday Summaries extraction, when the Time bar Persistence option is true the output result for quotes (Bid and Ask value) will have all the data included for the date and time range, irrespective of the same bid and ask quoted for different timings. However, when the Time bar Persistence option is false, the output result will capture only the changed values for the date and time range selected.
Time bar persistence is applicable only for quote messages and not for trade messages. The logic behind this is that the quotes are still alive in the market unless there’s a change and therefore the quotes summaries are copied down when the option is checked. An easy way of identifying if the quotes are a summary of an existing time interval or a copy from a previous time interval is with the help of fields “No. Bids” and “No. Asks” which is 0 when the values are getting copied down. For trades, copying down values from prior time bars amounts to manipulation of data as there are no trades.The output when setting TimebarPersistence to true is:
The output when setting TimebarPersistence to false is:
0
Answers
-
I could not find an option in the TickHistoryIntradaySummariesExtractionRequest that can be used to remove empty rows. You can see all available options on the REST API Reference Tree.
You may need to contact the Tick History Support team via MyRefinitv to confirm it or request to add this feature to the TickHistoryIntradaySummariesExtractionRequest.
0
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛