Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"Everything is working amazing now. I'm already impressed with the true-tick feed of IQFeed and it's ability to support my 480 symbol layout." - Comment from Tyler via Email
"I will tell others who want to go into trading that DTN ProphetX is an invaluable tool, I don't think anyone can trade without it..." - Comment from Luther
"It’s so nice to be working with real professionals!" - Comment from Len
"I noticed that ******* quotes locked up shortly after the interest rate announcement yesterday while yours stayed stable." - Comment from Ron in Utah
"I use IQ Feed, Great stuff as far as data analysis information, storage and retrieval is concerned." - Comment from Public Forum
"You are either overstaffed or people just don't have problems with your feed because customer support always answers the phone quickly." - Comment from Jay via Email
"I am keeping IQFeed, much better reliabilty than *******. I may refer a few other people in the office to switch as well." - Comment from Don
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"I like you guys better than *******...much more stable and a whole lot fewer issues." - Comment from Philip
"You are much better than lawyers or the phone company because you answer the phone when I call! I just love your customer service." - Comment from Isreal
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »IQFeed Developer Support »Duplicate Q Messages
Author Topic: Duplicate Q Messages (1 messages, Page 1 of 1)

jwelch
-Interested User-
Posts: 1
Joined: Mar 15, 2018


Posted: Mar 15, 2018 10:38 PM          Msg. 1 of 1
TLDR; - Is it normal to expect the occasional duplicate Q message in the feed? Or are these meant to represent two genuine ticks that somehow end up with exactly the same data (including TickID?)

Details:
After doing some analysis on the feed data I receive, there are occasional duplicate tick messages (as in they have exactly the same crc64 value)

For Example, here is a summary of the duplicate lines found in a sample feed from 3/7/18.


Writing CRC Collisions to: IQFeed (3-7-2018).crc
Writing duplicate entries to: IQFeed (3-7-2018).dup
Processed 11734028 ticks
Found 2045 duplicate lines in the file
T: 0
Q: 1983
P: 2
S: 0
F: 60
CRC: 0 - No collisions

Ignoring the F and P duplicates as those might expected... here is a sample of some Q messages which had the same CRC64 value.


Duplicate Line Found on Hash [f612fb7a7fa49e97]
Line 1 [864989]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Line 2 [864991]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [cada7e85403bc991]
Line 1 [866805]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Line 2 [866807]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [3ee933e4adea4f37]
Line 1 [869514]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,
Line 2 [869516]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,


The number in square brackets e.g. 'Line 1 [#]' is a local message identifier. Clearly the messages are received very close to one another in time.

This is with the field parameters as follows:

S,CURRENT UPDATE FIELDNAMES,Symbol,Exchange ID,Last,Change,Percent Change,Total Volume,High,Low,Bid,Ask,Bid Size,Ask Size,Tick,Range,Open Interest,Open,Close,Spread,Settle,Delay,Restricted Code,Net Asset Value,Average Maturity,7 Day Yield,Extended Trading Change,Extended Trading Difference,Price-Earnings Ratio,Percent Off Average Volume,Bid Change,Ask Change,Change From Open,Market Open,Volatility,Market Capitalization,Fraction Display Code,Decimal Precision,Days to Expiration,Previous Day Volume,Open Range 1,Close Range 1,Open Range 2,Close Range 2,Number of Trades Today,VWAP,TickID,Financial Status Indicator,Settlement Date,Bid Market Center,Ask Market Center,Available Regions,Last Size,Last Time,Last Market Center,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Time,Most Recent Trade Conditions,Most Recent Trade Market Center,Extended Trade,Extended Trade Size,Extended Trade Time,Extended Trade Market Center,Message Contents,Ask Time,Bid Time,Last Date,Extended Trade Date,Most Recent Trade Date


Note that TickID is included in the message, and is identical on lines which are found to have the same hash.

So, given the rather low occurrence of these duplicates (< 0.02%) this is probably not much of a concern for most analyses, but I was curious if this is expected behavior in the feed either way.

Thanks!
 

 

Time: Fri April 26, 2024 1:08 AM CFBB v1.2.0 8 ms.
© AderSoftware 2002-2003