ERROR: processing did not complete successfully !
Trying to download ZFU8 I get the following response
Chain Identifier: ChainRIC 0#ZFU8, Constituents: 2
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 31.05.2014 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 01.10.2016 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 31.08.2017 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 02.07.2006 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 06.07.2006 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 02.06.2007 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 15.06.2007 0:00:00,
Expiration Date ,
RIC ZFU8,
Start date ,
Change Date 01.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 07.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 26.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 04.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 17.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 24.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 06.10.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 04.10.2008 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 06.12.2008 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 04.07.2009 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 23.11.2009 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 26.11.2011 0:00:00,
Expiration Date 30.09.2008 0:00:00,
ZFU8, Ric, 03.07.2008 0:00:00 +00:00, 30.09.2008 0:00:00 +00:00, 6
ExtractionID: 2000000665078075
Waiting for file size...
Waiting for file size...
Waiting for file size...
Waiting for file size...
Waiting for file size...
ERROR: processing did not complete successfully !
WARNING messages:
Waiting for file size...
Waiting for file size...
Waiting for file size...
Waiting for file size...
Waiting for file size...
Press Enter to exit
Here is the code I use to access historical data
using System;
using System.Linq;
using System.Diagnostics;
using System.IO;
using DataScope.Select.Api.Extractions;
using DataScope.Select.Api.Extractions.ExtractionRequests;
using DataScope.Select.Api.Extractions.ReportExtractions;
using DataScope.Select.Api.Extractions.ReportTemplates;
using DataScope.Select.Api.Extractions.Schedules;
using DataScope.Select.Api.Extractions.SubjectLists;
using DataScope.Select.Api.Content;
using DataScope.Select.Core.RestApi;
using DataScope.Select.Api.Search;
using DataScope.Select.Api;
using System.Text;
using ICSharpCode.SharpZipLib.GZip;
using System.Text.RegularExpressions;
namespace RefinitivHistoryTesting
{
class Program
{
public static string getExtractionID(string note)
{
string pattern = @Extraction ID: ([0-9]+);
string extractionID = "";
Regex rgx = new Regex(pattern);
Match match = rgx.Match(note);
if (match.Success)
{
string[] words = match.Value.Split(':');
extractionID = words[1].ToString().Trim();
Console.WriteLine("ExtractionID: {0}", extractionID);
}
return extractionID;
}
static void Main(string[] args)
{
//-----------------------------------------------------------------
//Connect and authenticate to the DSS server:
//-----------------------------------------------------------------
Uri dssUri = new Uri("https://selectapi.datascope.refinitiv.com/RestApi/v1/");
string dssUserName = "****";
string dssUserPassword = "****";
//var out_dir = @.\Storage\;
//var out_dir = @E:\TimeTick\;
var out_dir = @C:\Users\Manuchar\Refinitiv_history\;
var SearchContext = new SearchContext(dssUri, dssUserName, dssUserPassword);
var ExtractionsContext = new ExtractionsContext(dssUri, dssUserName, dssUserPassword);
ExtractionsContext.Options.AutomaticDecompression = false;
try
{
ExtractionsContext.DefaultRequestHeaders.Add("x -direct-download", "true");
}
catch (Exception)
{ }
var availableMbPFields = ExtractionsContext.GetValidContentFieldTypes(ReportTemplateTypes.TickHistoryRaw);
var availableTnSFields = ExtractionsContext.GetValidContentFieldTypes(ReportTemplateTypes.TickHistoryTimeAndSales);
var dateRange = DateTimeRange.Create(
new DateTimeOffset(2000, 08, 01, 00, 00, 0, TimeSpan.Zero),
new DateTimeOffset(2026, 02, 20, 00, 00, 0, TimeSpan.Zero));
var request = new HistoricalChainResolutionRequest
{
ChainRics = new[] { "0#ZFU8" },
Range = dateRange
};
var chain_results = SearchContext.HistoricalChainResolution(request);
foreach (var chain in chain_results)
{
Console.WriteLine("Chain Identifier: {0} {1}, Constituents: {2}", chain.IdentifierType, chain.Identifier, chain.Constituents.Count());
foreach (var constituent in chain.Constituents)
{
var searchResults = SearchContext.HistoricalSearch(
HistoricalSearchRequest.Create(
constituent.Identifier, constituent.IdentifierType, dateRange, HistoricalResultsBy.Ric, null));
var result = searchResults.FirstOrDefault(pos => pos.Identifier.Equals("ZFU8"));
if (result != null)
{
var extractionDatesResult = ExtractionsContext.Extract(
new HistoricalReferenceExtractionRequest
{
ContentFieldNames = new DssCollection<string>()
{
"RIC", "Start date", "Change Date", "Expiration Date"
},
Condition = new HistoricalReferenceCondition
{
StartDate = result.FirstDate,
EndDate = result.LastDate
},
IdentifierList = new InstrumentIdentifierList
{
InstrumentIdentifiers = new[]
{
InstrumentIdentifier.Create(result.IdentifierType, result.Identifier)
},
ValidationOptions = new InstrumentValidationOptions
{
AllowHistoricalInstruments = true
},
UseUserPreferencesForValidationOptions = false
}
});
dynamic srcLastDate = null;
foreach (var content in extractionDatesResult)
foreach (var prop in content.DynamicProperties)
{
Console.WriteLine("{0}\t {1},\r", prop.Key, prop.Value);
if (srcLastDate == null && prop.Key == "Expiration Date" && prop.Value != null) srcLastDate = prop.Value;
}
dynamic lastDate = new DateTimeOffset(2008, 09, 30, 00, 00, 00, TimeSpan.Zero);//result.LastDate;
dynamic startDate = new DateTimeOffset(2008, 07, 03, 00, 00, 00, TimeSpan.Zero); //result.FirstDate;
Console.WriteLine("{0},\t {1},\t {2},\t {3},\t {4}", result.Identifier, result.IdentifierType, startDate, lastDate, result.DomainCode);
var extractionResult = ExtractionsContext.ExtractRaw(
new TickHistoryRawExtractionRequest
{
Condition = new TickHistoryRawCondition
{
ReportDateRangeType = ReportDateRangeType.Range,
QueryStartDate = startDate,// new DateTimeOffset(2023, 03, 21, 20, 0, 0, TimeSpan.Zero),
QueryEndDate = lastDate,// new DateTimeOffset(2023, 03, 21, 19, 59, 59, TimeSpan.Zero),
ExtractBy = TickHistoryExtractByMode.Ric,
MessageTimeStampIn = TickHistoryTimeOptions.LocalExchangeTime,
SortBy = TickHistorySort.SingleByRic,
DomainCode = TickHistoryRawDomain.MarketPrice,
DisplaySourceRIC = true
},
IdentifierList = new InstrumentIdentifierList
{
InstrumentIdentifiers = new[]
{
InstrumentIdentifier.Create(result.IdentifierType, result.Identifier)
},
ValidationOptions = new InstrumentValidationOptions
{
AllowHistoricalInstruments = true
},
UseUserPreferencesForValidationOptions = false
},
});
string extractionID = getExtractionID(extractionResult.Notes[0]);
ReportExtraction resultsize = null;
long fileSize = 0;
Boolean success = false;
string errorMsgs = "";
string warningMsgs = "";
string permissionMsgs = "";
int attempts = 0;
while ((resultsize == null || resultsize.FullFile == null) && attempts < 5)
{
System.Threading.Thread.Sleep(5000);
try
{
resultsize = ExtractionsContext.ReportExtractionOperations.Get(extractionID);
ExtractionsContext.LoadProperty(resultsize, "FullFile");
if (resultsize.FullFile != null)
{
fileSize = resultsize.FullFile.Size;
Console.WriteLine("File Size: {0}", fileSize);
success = true;
}
else
{
Console.WriteLine("Waiting for file size...");
warningMsgs += "Waiting for file size...\n";
}
}
catch (NullReferenceException ex)
{
Console.WriteLine("FullFile is not yet available, continuing to wait...");
errorMsgs += "FullFile is not yet available, continuing to wait...\n";
errorMsgs += "Exception message: " + ex.Message + "\n";
}
catch (Exception ex)
{
Console.WriteLine("An error occurred: " + ex.Message);
errorMsgs += "An error occurred: " + ex.Message + "\n";
}
attempts++;
}
if (!success)
{
Console.WriteLine("ERROR: processing did not complete successfully !\n" + errorMsgs);
}
if (warningMsgs != "")
{
Console.WriteLine("WARNING messages:\n" + warningMsgs);
}
if (permissionMsgs != "")
{
Console.WriteLine("PERMISSION ISSUES messages:\n" + permissionMsgs);
}
var fileName = out_dir + "_" + result.Identifier + "_" + startDate.DateTime.ToString("dd.MM.yyyy") + "_" + lastDate.DateTime.ToString("dd.MM.yyyy") + "_RawMarketPrice.gzip";
using (var response = ExtractionsContext.RawExtractionResultOperations.GetReadStream(extractionResult))
using (var fileStream = File.Create(fileName))
response.Stream.CopyTo(fileStream);
}
}
}
try
{
ExtractionsContext.DefaultRequestHeaders.Remove("x-direct-download");
}
catch (Exception)
{ }
Console.WriteLine("Press Enter to exit");
Console.ReadLine();
}
}
}
Can anyone help me with the problem without sending me to the A documentation?
I've read everything and tried different options.
I have a problem with the following files
ZFU8
startDate = new DateTimeOffset(2008, 07, 03, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2008, 09, 30, 00, 00, 00, TimeSpan.Zero)
ZFZ8
startDate = new DateTimeOffset(2008, 08, 16, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2008, 12, 31, 00, 00, 00, TimeSpan.Zero)
ZFU9
startDate = new DateTimeOffset(2009, 05, 23, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2009, 09, 30, 00, 00, 00, TimeSpan.Zero)
ZFU1
startDate = new DateTimeOffset(2011, 05, 21, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2011, 09, 30, 00, 00, 00, TimeSpan.Zero)
ZFM2
startDate = new DateTimeOffset(2012, 02, 18, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2012, 06, 29, 00, 00, 00, TimeSpan.Zero)
ZFU3
startDate = new DateTimeOffset(2013, 05, 18, 00, 00, 00, TimeSpan.Zero)
lastDate = new DateTimeOffset(2013, 09, 30, 00, 00, 00, TimeSpan.Zero)
Who can, please give some good code advice.
What could be the problem
I know for sure that the code is 100% functional
With this code I have downloaded all chainRIC: ZF except those with error:
ERROR: processing did not complete successfully !
0
Best Answer
-
Thank you reaching out to us.
I can run the code properly.
According to the code, the FullFile is not available. You need to check the extraction notes and report's status.
public static string getExtractionID(string note)
{
Console.WriteLine("### Notes ###\n" + note );
string pattern = @Extraction ID: ([0-9]+);
string extractionID = "";
Regex rgx = new Regex(pattern);
Match match = rgx.Match(note);
if (match.Success)
{
string[] words = match.Value.Split(':');
extractionID = words[1].ToString().Trim();
Console.WriteLine("ExtractionID: {0}", extractionID);
}
return extractionID;
}...
while ((resultsize == null || resultsize.FullFile == null) && attempts < 5)
{
System.Threading.Thread.Sleep(5000);
try
{
resultsize = ExtractionsContext.ReportExtractionOperations.Get(extractionID);
Console.WriteLine("Status: "+resultsize.Status);
Console.WriteLine("Detalied Status: "+resultsize.DetailedStatus);
ExtractionsContext.LoadProperty(resultsize, "FullFile");
if (resultsize.FullFile != null)
{
fileSize = resultsize.FullFile.Size;
Console.WriteLine("File Size: {0}", fileSize);
success = true;
}
...0
Answers
-
Thank you.
Here's what I got:
ExtractionID: 2000000665353495
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
ERROR: processing did not complete successfully !
Press Enter to exitCan you tell me what else I can modify in the code to download the necessary RICs?
P.S. Was writing a comment, but it got caught in moderation. I'll repeat it here.
0 -
Please share the extraction note.
public static string getExtractionID(string note)
{
Console.WriteLine("### Notes ###\n" + note );
string pattern = @Extraction ID: ([0-9]+);
string extractionID = "";
Regex rgx = new Regex(pattern);
Match match = rgx.Match(note);
if (match.Success)
{
string[] words = match.Value.Split(':');
extractionID = words[1].ToString().Trim();
Console.WriteLine("ExtractionID: {0}", extractionID);
}
return extractionID;
}0 -
Chain Identifier: ChainRIC 0#ZF:, Constituents: 53
Start date 29.08.2008 0:00:00,
Change Date 26.11.2011 0:00:00,
Expiration Date 30.09.2008 0:00:00,
ZFU8, Ric, 03.07.2008 0:00:00 +00:00, 30.09.2008 0:00:00 +00:00, 6
### Notes ###
Extraction Services Version 17.4.1.46677 (972294413835), Built Oct 25 2023 19:05:27
User ID: 9034323
Extraction ID: 2000000665387197
Correlation ID: CiD/9034323/0x0000000000000000/RESTAPI-I/EXT.2000000665387197
Schedule: 0x08c46b418b396db4 (ID = 0x0000000000000000)
Input List (1 items): (ID = 0x08c46b418b396db4) Created: 01/05/2024 07:05:59 Last Modified: 01/05/2024 07:05:59
Report Template: _OnD_0x08c46b418b396db4 (ID = 0x08c46b418b496db4) Created: 01/05/2024 07:03:24 Last Modified: 01/05/2024 07:03:24
Schedule dispatched via message queue (0x08c46b418b396db4), Data source identifier (CB206CFAEABC47BD868EB681D8CD60F5)
Schedule Time: 01/05/2024 07:03:25
Processing started at 01/05/2024 07:03:25
Processing completed successfully at 01/05/2024 07:06:00
Extraction finished at 01/05/2024 07:06:00 UTC, with servers: tmc28nu8xn03, TRTH (149.979 secs)
Historical Instrument <RIC,ZFU8> expanded to 1 RIC: ZFU8.
Total instruments after instrument expansion = 10 -
Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 0; Instruments Approved for Extraction: 0; Tick History Cash Quota Count After Extraction: 0, 100% of Limit; Tick History Cash Quota Limit: 0
Quota Message: ERROR: The RIC 'ZFU8' in the request would exceed your quota limits. Adjust your input list to continue.
Quota Message: WARNING: Tick History Cash Quota has been reached or exceeded
ExtractionID: 2000000665387197
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
ERROR: processing did not complete successfully !
Press Enter to exitGot a response from the server.
What are some ideas?
0 -
ZF is within my quota limit.
I downloaded the other ZF contracts
0 -
Thank you.
Here's what I got:
Chain Identifier: ChainRIC 0#ZF:, Constituents: 53
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 31.05.2014 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 01.10.2016 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC 1FVU8^0,
Start date 29.08.2008 0:00:00,
Change Date 31.08.2017 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 02.07.2006 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 06.07.2006 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 02.06.2007 0:00:00,
Expiration Date 19.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 15.06.2007 0:00:00,
Expiration Date ,
RIC ZFU8,
Start date ,
Change Date 01.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 07.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 26.07.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 04.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date ,
Change Date 17.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 24.08.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8,
Start date 29.08.2008 0:00:00,
Change Date 06.10.2007 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 04.10.2008 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 06.12.2008 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 04.07.2009 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 23.11.2009 0:00:00,
Expiration Date 30.09.2008 0:00:00,
RIC ZFU8^0,
Start date 29.08.2008 0:00:00,
Change Date 26.11.2011 0:00:00,
Expiration Date 30.09.2008 0:00:00,
ZFU8, Ric, 03.07.2008 0:00:00 +00:00, 30.09.2008 0:00:00 +00:00, 6
ExtractionID: 2000000665353495
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
Status: Completed
Detailed Status: Done
FullFile is not yet available, continuing to wait...
ERROR: processing did not complete successfully !
Press Enter to exitCan you tell me what else I can modify in the code to download the necessary RICs?
0 -
In the note, you should have this line.
According to note, the file wasn't created due to the Quota Limit.
0 -
According to note, the file wasn't created due to the Quota Limit.
0 -
Jirapongse, thanks!!!
I don't see a button to close the question, since I got the answer and it's clear to me what to do.
Please close the question as answered.
0
Categories
- All Categories
- 6 AHS
- 37 Alpha
- 161 App Studio
- 4 Block Chain
- 4 Bot Platform
- 16 Connected Risk APIs
- 47 Data Fusion
- 30 Data Model Discovery
- 608 Datastream
- 1.3K DSS
- 577 Eikon COM
- 4.9K Eikon Data APIs
- 7 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- Trading API
- 2.7K Elektron
- 1.3K EMA
- 236 ETA
- 519 WebSocket API
- 33 FX Venues
- 10 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 20 Messenger Bot
- 2 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 59 Open Calais
- 264 Open PermID
- 39 Entity Search
- 2 Org ID
- PAM
- PAM - Logging
- 8.4K Private Comments
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 20 RDMS
- 1.4K Refinitiv Data Platform
- 367 Refinitiv Data Platform Libraries
- 3 Refinitiv Due Diligence
- LSEG Due Diligence Portal API
- 3 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.1K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 10 World-Check Customer Risk Screener
- 990 World-Check One
- 44 World-Check One Zero Footprint
- 45 Side by Side Integration API
- Test Space
- 3 Thomson One Smart
- 1.2K TR Internal
- Global Hackathon 2015
- 2 Specialists Who Code
- 10 TR Knowledge Graph
- 150 Transactions
- 142 REDI API
- 1.7K TREP APIs
- 4 CAT
- 21 DACS Station
- 117 Open DACS
- 1.1K RFA
- 103 UPA
- 172 TREP Infrastructure
- 224 TRKD
- 886 TRTH
- 5 Velocity Analytics
- 5 Wealth Management Web Services
- 59 Workspace SDK
- 9 Element Framework
- 5 Grid
- 13 World-Check Data File
- Yield Book Analytics
- 46 中文论坛