Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"DTN has never given me problems. It is incredibly stable. In fact I've occasionally lost the data feed from Interactive Brokers, but still been able to trade because I'm getting good data from DTN." - Comment from Leighton
"I've been using IQFeed 4 in a multi-threaded situation for the last week or two on 2600 symbols or so with 100 simultaneous daily charts, and I have had 100% responsiveness." - Comment from Scott
"I started a trial a few weeks back before the market went wild. DTN.IQ didn’t miss anything and beat my other provider. I decided to stay with you because of the great service through all the volatility." - Comment from Mike
"I cannot believe what a difference it makes trading with ProphetX!" - Comment from Bruce in Los Angeles
"If you want customer service that answers the phone, your best bet is IQFeed. I cannot stop praising them or their technical support. They are always there for you, and they are quick. I have used ****** too but the best value is IQFeed." - Comment from Public Forum
"IQFeed version 4 is a real screamer compared to anything else I have seen." - Comment from Tom
"I'm satisfied with IQFeed. It's the most reliable and fastest quote feed I have ever used. Although I'm a resident in China, it's still very fast!" - Comment from Xiaofei
"The people at Nirvana have very nice things to say about your company and I can see why! Price and service is a potent combination." - Comment from Ed
"Thanks for all of your help. Great customer service deserves to be recognized which one the reasons I've been a customer of DTN for over 10 years!" - Comment from Stuart
"I have to tell you though that using the IQFeed API is about the easiest and cleanest I have seen for some time." - Comment from Jim
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »Data and Content Support »Data Quality?
Author Topic: Data Quality? (10 messages, Page 1 of 1)

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Feb 29, 2012 12:28 PM          Msg. 1 of 10
Hello

I have a trial account and downloaded the corn continous future. I found several quite strange quotes (See the screenshots enclosed):
CASE A:
Q1: From 12th July to 15th July the quote drops by 122.50?
Q2: I downloaded corn also from http://www.tradingblox.com . The quote on 12th July is 861.75. Far away from IQ Feed Quote

CASE B:
7 days have quotes of 259, almost 50% below other quotes.

CASE C:
6 days have quotes of 259, almost 50% below other quotes.

CASE D & E
Again, some days with 259 quotes.



How is this possible? I assume that thousdands of users have used this data before and I assume that you do some minimal data checking?

What is then the problem?

Thanks & best regards



File Attached: Corn ZC Continous.pdf (downloaded 1230 times)

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Feb 29, 2012 12:31 PM          Msg. 2 of 10
seems there is a problem with the PDF, here uploaded again



File Attached: ACFA99D.pdf (downloaded 1627 times)

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Feb 29, 2012 12:56 PM          Msg. 3 of 10
we are looking into this.

You are looking at the back-adjusted contracts within IQ (where the actual price is adjusted based on the close value of the old and new contract around roll dates). The 259 values are cases where our regular continuation contract (without back adjusting) are incorrect and showing zero values. I have reported this to our market data group for investigation.

I believe this will address all of your issues other than case A which will require some more digging.

We'll keep you updated.

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Feb 29, 2012 01:04 PM          Msg. 4 of 10
ok, thanks.
BTW:I downloaded the data with NinjaTrader

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Mar 1, 2012 10:25 AM          Msg. 5 of 10
The issues I pointed out yesterday (all but Case A) have been corrected. There was an issue in our system building the continuous contract data for that contract.

Still looking into Case A

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Mar 1, 2012 12:01 PM          Msg. 6 of 10
Case A should be resolved by monday.

However, the price values for our Back-Adjusted contracts won't match tradingblox.com because they use a different roll method than we do for continuous contracts.

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Mar 2, 2012 01:19 AM          Msg. 7 of 10
thanks for the update.

Some questions:
1. About the problem with 259 values. How can I know that you do not have similar kind of problems in many contracts? It is quite time consuming to validate each time series

2. About Case A: You found the bug and resolve on Monday or you still look for the reason? If you found the bug, what was the reason?


I did some more research with unadjusted data, will post it today

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Mar 2, 2012 04:22 AM          Msg. 8 of 10
I investigated some more:

Compared Unadjusted Corn Quotes with PiFin, Pinnacle and Zenfire
What looks good or is acceptable (for me, who needs EOD data)
- "Open" Quote mostly the same
- Some differences in High/Low/Close quotes. Normally not signifcant, sometimes more than 1%
- Sometimes dates are missing
- Sometimes quotes on a date are constructed (other providers have no quotes, iqfeed has same quote as previous day)

What is not good:
- There are days with 0-values as quotes
- 15th to 31th Dec 2003 no data
- 1st Dec to 12th Dec 2003: Large differences to other sources (between 4.3% and 7.5%)

The problem of price drop on 15th July (CASE A reported at the beginning of this post)
I compared the unadjusted with the adjusted data as well the month contracts (downloaded from turtle page). As you can see in the PDF enclosed there was neither in the July nor in the September contract any movement compared to your backadjusted data series. It looks like that you construct here the backadjusted chain wrongly?



File Attached: Corn Contract Comparision.pdf (downloaded 1496 times)

datatest
-Interested User-
Posts: 23
Joined: Feb 29, 2012


Posted: Mar 8, 2012 11:02 AM          Msg. 9 of 10
Hello

Is there any update on this topic?

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Mar 8, 2012 12:27 PM          Msg. 10 of 10
My aplogies for not updating the thread sooner.

The process that calculates all of the back adjustment amounts normally runs on the weekends (this is why I mentioned it should have been fixed on monday). However on monday, when checking it, I noticed there were still some errors in the data. I have been working with our market data group to get the data corrected but it is a fairly time consuming process. So far, the back-adjusted electronic corn contract should be good back through 2000. We are currently working on correcting 1996-1999 and I'll let you know when that process is complete (hopefully this will be later today or tomorrow at the latest).

In response to your specific concerns in the previous post:

Quote: - Sometimes dates are missing

This is actually what has been causing problems with the back-adjusted continuous contract. Most of these through 1-1-2000 should be fixed now, please let me know if you find extras.

Quote: - Sometimes quotes on a date are constructed (other providers have no quotes, iqfeed has same quote as previous day)

This could also be releated to the above issue but if you are still seeing these it is also possible that it is by design. IQFeed will show a daily datapoint anytime something trades or settles. If the exchange sent us trades or settlements on the "extra" days then we will likely have a datapoint there. If that doesn't seem to be the case, please let us know some examples and we will look into it.

Quote: - There are days with 0-values as quotes

These should have all been fixed as of my post on Mar 1, 2012 10:25 AM. Let me know if you still see issues.
Quote: - 15th to 31th Dec 2003 no data

This should also be corrected at this time.
Quote: - 1st Dec to 12th Dec 2003: Large differences to other sources (between 4.3% and 7.5%)

Can you provide me an example of the differences?

Hope this helps explain things. As I mentioned before, I'll post again as soon as we are finished with the correcting of the back-adjustment back through 1996.
 

 

Time: Mon May 27, 2024 3:36 PM CFBB v1.2.0 18 ms.
© AderSoftware 2002-2003