bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Mar 20, 2005 02:18 PM
Msg. 1 of 17
Intraday data for days prior to 18th of March seems to be corrupted for CME futures. The times of data ticks is out of sequence in the DTN Time & Sales program, and the same problem is evident in minutes in the Chart program. I've checked @EUM5, @ESM5, @NQM5, and @BPM5 and they all exhibit the same problem.
Many thanks for investigating,
Brendan
|
DTN_MarketData
-DTN's Market Data Experts-
Posts: 785
Joined: Jul 15, 2004
|
Posted: Mar 21, 2005 07:35 AM
Msg. 2 of 17
We are currently looking into this issue. More information will be posted when available.
|
tradinoncoffee
-Interested User-
Posts: 38
Joined: Dec 9, 2004
|
Posted: Mar 28, 2005 03:42 PM
Msg. 3 of 17
Not sure if I am seeing the same problem you were referencing in this thread, but the data for the index futs (ES, NQ, ER2, and YM) is messed up (thats the best technical description I can offer) for the period 3/14-17.
If this is the same problem, and "marketdataservices" said on 3/21 they were looking into it, are they still "looking into it"???
Seems like a week ought to be plenty of time to look into and remedy a problem of this nature. When can we expect the data issue to be corrected. As it stands, it would appear that I am stuck with bad data in my TICK and Range charts since my charting software only allows me to refresh up to 6 days of history. What can DTN do to help me rectify this?
Tradinoncoffee
|
bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Mar 28, 2005 04:49 PM
Msg. 4 of 17
Yes that is the problem. I agree, seems a long time to replace some bad historical data. DTN emailed me this 45 minutes ago:
"As of yet we have not completed rewriting the bad data. This is a labor intensive process and will be released once the data is complete. Ken Heinze DTN Market Access Customer Operations Specialist "
They've agreed to credit my account for the period I've been without the futures data I need, so that softens the blow a little. I suggest you also claim compensation.
Regards,
Brendan
|
bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Mar 31, 2005 05:40 PM
Msg. 5 of 17
Just received this:
"From: IQFeed Customer Support [mailto:support@iqfeed.net] Sent: 01 April 2005 00:06 Subject: Re: @EUM5 Data Corruption (Thread:441926)
This was caused by corrupt data coming from the exchange in which we have no back up nor source of historical data. We are correcting each symbol by hand and we will continue to correct each symbol until it is completed. @ESM5 should be corrected by Monday. Thank you and we apologize for the delay Mike "
Two points spring to mind:
1. Your only function is to supply market data, for which you have "no backup nor [alternative] source of historical data"? Is the robustness of your service not worth one measly monthly subscription to an alternative data provider?
2. On finding some historical data was corrupt, instead of paying your staff $$$$ to manually type in every time and sale (which is what your email implies you are doing), you STILL don't spend $$ on a subcription to an alternative data supplier.
Am I missing something here, or has this exposed DTN's service as an absolute shambles? Does anyone have the correct intraday data for any CME futures between 3/14-17 they can send to support@iqfeed.net? Maybe we'll be better at supplying market data to DTN than DTN are to us.
Brendan
|
nsolot
-DTN Guru-
Posts: 273
Joined: Sep 4, 2004
|
Posted: Mar 31, 2005 07:04 PM
Msg. 6 of 17
|
skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004
|
Posted: Apr 1, 2005 07:49 AM
Msg. 7 of 17
"2. On finding some historical data was corrupt, instead of paying your staff $$$$ to manually type in every time and sale (which is what your email implies you are doing), you STILL don't spend $$ on a subcription to an alternative data supplier."
I certainly hope they are not wasting their time typing in time and sales, because the tick history files on their servers are automatically deleted after 8 calendar days. So even if they manage to type it all in, us poor sods that pay for this service won't be able to update our databases anyway...
|
bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Apr 1, 2005 07:52 AM
Msg. 8 of 17
I don't think it 's an April fool, as their previous email to me (see above) stated "This is a labor intensive process and will be released once the data is complete", which suggests they are actually typing the times and sales one by one. I understand the confusion though as DTN's handling of this situation is difficult to distiguish from a joke. Hopefully they'll buy the data from CME using your link rather than continue doing it their way. Edited by bdholmes on Apr 1, 2005 at 07:53 AM Edited by bdholmes on Apr 1, 2005 at 07:53 AM
|
DTN_Jay_Froscheiser
-VP, Product Operations-
Posts: 1746
Joined: May 3, 2004
DTN IQFeed/DTN.IQ/DTN NxCore
|
Posted: Apr 1, 2005 08:25 AM
Msg. 9 of 17
A couple things.
First, Yes, we have to manually correct the data. This is because no other data source has their data in IQFeed format. This wasn't done "By hand" as in someone sitting at a terminal typing in numbers. We had to build a manual import routine to get the data into our server format.
Second, We cannot just subscribe to another service and use their data. This is illegal and against not only the services terms of use but that of the exchanges. We can get the data from the exchange.
"Am I missing something here, or has this exposed DTN's service as an absolute shambles?" You are missing something....
Jay Froscheiser DTN Market Access, LLC.
|
tradinoncoffee
-Interested User-
Posts: 38
Joined: Dec 9, 2004
|
Posted: Apr 2, 2005 11:07 PM
Msg. 10 of 17
...so what are we missing? Delayed data....corrupt and seemingly unfixable data....???
Tradinoncoffee Edited by tradinoncoffee on Apr 2, 2005 at 11:08 PM
|
bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Apr 4, 2005 07:03 AM
Msg. 11 of 17
Jay,
I now understand why you cannot just subscribe to an alternative supplier, and that you have to convert the replacement data into your own format before importing it.
Please can you confirm that you now have a readily available alternative data source for CME futures, and a system in place so any corrupted data can be corrected MUCH more quickly in future. Also, for any future issues that overrun, the more information you provide your customers explaining the delay, the less likely they will jump to conclusions about the quality of your service.
@EU is the symbol I mostly use, so please can you put this near the top of the list of ones to fix.
Many thanks,
Brendan
|
bdholmes
-Interested User-
Posts: 23
Joined: Aug 16, 2004
|
Posted: Apr 10, 2005 06:22 AM
Msg. 12 of 17
Jay,
@EU is still corrupt, three weeks after I first reported it. Please can I have an ETA on a fix. At what point will I be justified in describing DTN's service an absolute shambles again? A month, 6 weeks? If you were a customer of a data provider that supplied corrupt data for this long, how would you describe the service?
Many thanks,
Brendan
|
skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004
|
Posted: Apr 10, 2005 12:09 PM
Msg. 13 of 17
not only @EU
|
skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004
|
Posted: Apr 12, 2005 07:38 AM
Msg. 14 of 17
5 minute tick history for @EU for 2005-04-10 looks "interesting"
2005-04-10 00:35:00,129490.000000,129490.000000,129490.000000,129490.000000,1,0 2005-04-10 00:50:00,129460.000000,129460.000000,129460.000000,129460.000000,1,0 2005-04-10 18:05:00,129490.000000,129490.000000,129490.000000,129490.000000,1,0 2005-04-10 18:10:00,129450.000000,129430.000000,129450.000000,129430.000000,3,0 2005-04-10 18:25:00,129480.000000,129480.000000,129480.000000,129480.000000,1,0 2005-04-10 18:45:00,129450.000000,129420.000000,129450.000000,129420.000000,1,0 2005-04-10 18:50:00,129410.000000,129410.000000,129410.000000,129410.000000,1,0 2005-04-10 19:55:00,129380.000000,129360.000000,129360.000000,129380.000000,1,0 2005-04-10 21:00:00,129420.000000,129400.000000,129420.000000,129400.000000,1,0 2005-04-10 21:25:00,129310.000000,129310.000000,129310.000000,129310.000000,1,0 2005-04-10 21:30:00,129310.000000,129310.000000,129310.000000,129310.000000,1,0 2005-04-10 21:50:00,129360.000000,129360.000000,129360.000000,129360.000000,2,0 2005-04-10 22:25:00,129410.000000,129410.000000,129410.000000,129410.000000,1,0 2005-04-10 22:30:00,129450.000000,129450.000000,129450.000000,129450.000000,1,0 2005-04-10 23:00:00,129520.000000,129520.000000,129520.000000,129520.000000,1,0 2005-04-10 23:05:00,129530.000000,129530.000000,129530.000000,129530.000000,1,0 2005-04-10 23:15:00,129610.000000,129550.000000,129550.000000,129610.000000,1,0 2005-04-10 23:20:00,129600.000000,129580.000000,129600.000000,129580.000000,1,0 2005-04-10 23:30:00,129560.000000,129560.000000,129560.000000,129560.000000,1,0 2005-04-10 23:55:00,129490.000000,129450.000000,129490.000000,129450.000000,2,0
Notice all the gaps. Notice the initial 2 bars when the market was CLOSED. Notice the obviously wrong volume. Edited by skunk on Apr 12, 2005 at 07:40 AM
|
skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004
|
Posted: Apr 12, 2005 09:42 AM
Msg. 15 of 17
Notice the position of the decimal point.
|
DTN_Jay_Froscheiser
-VP, Product Operations-
Posts: 1746
Joined: May 3, 2004
DTN IQFeed/DTN.IQ/DTN NxCore
|
Posted: Apr 12, 2005 02:53 PM
Msg. 16 of 17
This should be corrected this afternoon/evening
Jay Froscheiser DTN Market Access, LLC.
|
skunk
-DTN Evangelist-
Posts: 249
Joined: May 7, 2004
|
Posted: Apr 12, 2005 03:16 PM
Msg. 17 of 17
Thank you for attending to this problem.
|
|
|
|