Removing Duplicate Identifiers using an On Demand ExtractWithNotes
Hi,
Please could we be advised on the below.
The questions is to being able to use duplicates in the request, but have them removed so for example only a single line is returned for the first example as they are the same security (with same Asset ID)
------------------------------------------------------------------------------------------------------------------
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
"ContentFieldNames": [
"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },
{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }
]
}
}
}
Same thing for
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TermsAndConditionsExtractionRequest",
"ContentFieldNames": [
"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID", "Currency Code", "MIC", "CFI Code", "Maturity Date"//, "Underlying CUSIP", "Underlying ISIN", "Entity LEI",
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "00209tab1", "IdentifierType": "Cusip" },
{ "Identifier": "IBM.N", "IdentifierType": "Ric" },
{ "Identifier": "US4592001014", "IdentifierType": "Isin" },
{ "Identifier": "B1YW440", "IdentifierType": "Sedol" }
]
}
}
}
------------------------------------------------------------------------------------------------------------------
We have discovered a separate request to check if identifier have duplicates : POST .../Extractions/InstrumentListValidateIdentifiers, however if there is a way to submit the Extraction request with the duplicates in the list item array, but only return unique records this would be the preferred method.
Best regards,
Gareth
Best Answer
-
I assume that setting is for DSS Web GUI. For the DSS REST API, you can use the KeepDuplicates property to append instruments to an instrument list.
Please contact the DSS support team via MyRefinitiv to confirm this.
I propose two solutions.
1. Use the API to create an instrument list. Then, append instruments to the created instrument list with the KeepDuplicates property set to false so the duplicated instruments will be removed. Finally, use the created instrument list in on-demand extractions.
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
"ContentFieldNames": [
"CUSIP",
"ISIN",
"SEDOL",
"Ticker",
"Issue PermID",
"Issuer PermID",
"Issuer OrgID",
"Entity LEI",
"Currency Code",
"MIC",
"CFI Code",
"Maturity Date",
"Underlying CUSIP",
"Underlying ISIN"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentListIdentifierList",
"InstrumentListId": "<List Id>"
}
}
}2. Validate instruments with the /Extractions/InstrumentListValidateIdentifiers endpoints before using them in on-demand extractions.
{
"InputsForValidation": [
{
"Identifier": "912810TH1",
"IdentifierType": "Cusip"
},
{
"Identifier": "US912810TH14",
"IdentifierType": "Isin"
}
],
"KeepDuplicates": false
}0
Answers
-
Thank you for reaching out to us.
Sorry. I am unable to find any property that can be used to remove the duplicated entries in the response.
Please kindly contact the DSS support team directly via MyRefinitiv to confirm it.
0 -
@Jirapongse Hi, thanks for the feedback, I have raised Support Ticket via Helpdesk to get this reviewed further.0
-
I tried changing my DSS GUI preference "Allow Import of Duplicate Items into Instrument Lists" from True to False and then tried importing the instruments
CUSIP - 912810TH1 and ISIN - US912810TH14 and GUI correctly identifies them as same instrument and allows import of only 1 instrument. However, when tried copying my preferences on API using the below query, I still get 2 outputs with the same result. Shouldn't API show only 1 output?
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
"ContentFieldNames": [
"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },
{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }
],
"ValidationOptions": null,
"UseUserPreferencesForValidationOptions": false
}
}}
I also tried independent validation option from the API Reference Tree by setting - "ImportOfDuplicateInstrumentsAllowed" to false under validation option. Still the output has 2 results (duplicate). This validation option is not working as well.
Please suggest a workaround on API as there is no content issue and GUI is working as expected.
0 -
You have specified UseUserPreferencesForValidationOptions as false. Set that to true.
0 -
Hello @Gurpreet I tried that as well.
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.CompositeExtractionRequest",
"ContentFieldNames": [
"CUSIP", "ISIN", "SEDOL","Ticker","Issue PermID", "Issuer PermID", "Issuer OrgID","Entity LEI", "Currency Code", "MIC", "CFI Code", "Maturity Date", "Underlying CUSIP", "Underlying ISIN"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "912810TH1", "IdentifierType": "Cusip" },
{ "Identifier": "US912810TH14", "IdentifierType": "Isin" }
],
"ValidationOptions": null,
"UseUserPreferencesForValidationOptions": true
}
}}
I still get 2 outputs (duplicates)
0 -
{
"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#DataScope.Select.Api.Extractions.ExtractionRequests.ExtractionResult",
"Contents": [
{
"IdentifierType": "Cusip",
"Identifier": "912810TH1",
"CUSIP": "912810TH1",
"ISIN": "US912810TH14",
"SEDOL": "BNLYVN1",
"Ticker": "UST",
"Issue PermID": "232792772567",
"Issuer PermID": "8589934522",
"Issuer OrgID": "95823",
"Entity LEI": "254900HROIFWPRGM1V77",
"Currency Code": "USD",
"MIC": "XBER",
"CFI Code": "DBFTFR",
"Maturity Date": "2042-05-15",
"Underlying CUSIP": null,
"Underlying ISIN": null
},
{
"IdentifierType": "Isin",
"Identifier": "US912810TH14",
"CUSIP": "912810TH1",
"ISIN": "US912810TH14",
"SEDOL": "BNLYVN1",
"Ticker": "UST",
"Issue PermID": "232792772567",
"Issuer PermID": "8589934522",
"Issuer OrgID": "95823",
"Entity LEI": "254900HROIFWPRGM1V77",
"Currency Code": "USD",
"MIC": "XBER",
"CFI Code": "DBFTFR",
"Maturity Date": "2042-05-15",
"Underlying CUSIP": null,
"Underlying ISIN": null
}
],
0
Categories
- All Categories
- 6 AHS
- 39 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 60 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛