Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"It’s so nice to be working with real professionals!" - Comment from Len
"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"Very impressed with the quality of your feed - ******* is a real donkey in comparison." - Comment from A.C. via Email
"The service is great, I see a noticeable improvement in my volume profiles over [broker]'s data feed" - Comment from Larry
"I've been using Neoticker RT with IQFeed for two months, and I'm very happy with both of the products (I've had IQFeed for two years with very few complaints). The service from both companies is exceptional." - Comment from Public Forum
"Everything is working great with the API. I love it." - Comment from Calvin
"I would just like to say that IQFeed version 4 is running very well and I am very happy with its performance. I would also like to extend a big thanks for the fast and efficient help that I always receive. My questions and concerns are always addressed promptly. Way to go!" - Comment from Josh in CO.
"You have an excellent feed. Very few spikes for Spot Forex." - Comment from Public Forum Post
"This is an excellent value, the system is generous (allowing for 500 stocks) and stable (and really is tick-by-tick), and the support is fantastic." - Comment from Shirin via Email
"Previously I was using *******. IQFeed is WAY more economical, and for my charting needs is just as good, if not better." - Comment from Public Forum Post
Home  Search  Register  Login  Blogs Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Tick data into OHLCV
Author Topic: Tick data into OHLCV (23 messages, Page 1 of 1)

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 28, 2016 04:09 PM          Msg. 1 of 23
Hello,

I have a custom application that collects tick data (realtime and history) into OHLCV bars by different timeframes and analyses the price levels. As a result I want to get the same price and volume as in your history interval data. Currently I'm testing my application on history and realtime ticks and want to have a clear understanding how should I filter ticks to get the same volume (period volume) and prices (open, high, low, close) as your historical interval data?
For example, currently I do not use the following ticks:
- Realtime ticks with "Message Contents" field that is not equal "C" (Last Qualified Trade)
- History ticks with "Basis For Last" field that is not equal "C" (Last Qualified Trade)
Should I implement any additional filtering?

Thanks.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 29, 2016 01:25 AM          Msg. 2 of 23
Good morning, we include C & E trades in our interval data. So you would want to add those in as well.

Tim Walter

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 29, 2016 05:00 AM          Msg. 3 of 23
Thank you Tim.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 29, 2016 05:10 AM          Msg. 4 of 23
Anytime, while I am thinking about it, also make note our intervals are such that the data from 11:00:00 to 11:00:59.999999 is placed in the 11:01 minute bar. This can cause some people an off by one error when comparing data.

Let me know if you have any questions,

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 30, 2016 04:38 AM          Msg. 5 of 23
Hello Tim,

I'm checking the data matching between ticks and interval and found some inconsistencies.
Dealing with BD#C, timeframe is 1 minute, date - 19 November 2015, time 2:02.

Request of interval history data was the following:
HIT,BD#C,60,20150329 030000,20160329 160000,,,,1,HIT_BD#C

Interval data from response:
HIT_BD#C,2015-11-19 02:02:00,156.66,156.61,156.63,156.65,2242,2238,

Next, requested tick data for the interval above:
HTT,BD#C,20151119 020100,20151119 020159,,,,1,HTT_BD#C

Response and volume calculations are in the attached file (ticks_BD#C_20151119_0202.xlsx). There are tick fields in the columns from "A" to "J". Look at column "T" please, there is summary volume (sum of "last size" fields) of the tick data and and interval (period) volume of the interval data.

Could you tell me the reason why these volumes are not equal?

Thank you.
Simon

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 30, 2016 05:13 AM          Msg. 6 of 23
Sorry, xlsx attachments are not supported. Wrapped in a zip archive.



File Attached: ticks_BD#C_20151119_0202.zip (downloaded 199 times)

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 02:28 AM          Msg. 7 of 23
Hi guys, i'm finding numerous inconsistencies between interval data and tick data, could someone please get back to me on the above question today? Trying to get to the bottom of this!

Really appreciate your help

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:03 AM          Msg. 8 of 23
Sorry, the file that you uploaded appears to have an issue. Could you re-upload your zip file for me to look over?

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 09:09 AM          Msg. 9 of 23
Zip-file is re-uploaded. Just in case, here is a link on dropbox: https://www.dropbox.com/s/c9t8cc08jc8p4k2/ticks_BD%23C_20151119_0202.zip?dl=0



File Attached: ticks_BD#C_20151119_02021.zip (downloaded 208 times)

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:15 AM          Msg. 10 of 23
Thanks much. Looking it over now.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 31, 2016 09:31 AM          Msg. 11 of 23
What protocol are you requesting the data under?

Note: If you are not calling SET PROTOCOL anywhere on this socket, it would be protocol 4.9.

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Mar 31, 2016 02:11 PM          Msg. 12 of 23
On startup my application connects to the Level1 and sets protocol v5.1.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Mar 31, 2016 02:14 PM          Msg. 13 of 23
Hello,

Sorry for the delay, but this one took me a little while to track down. It appears this is related to an issue that occurred back in 2015 during our initial roll-out of microseconds. This caused the discrepancies that you are seeing in our minute data and we are looking into what it will take to get it resolved. The tick data is correct though in all of the cases that I researched, so in the interim, if you build your bars from the underlying tick data, you will have the correct information.

I will try to get more information on this in the days to come. Let me know if you have any questions beyond this though and I will be happy to impart whatever I can.

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Apr 3, 2016 12:38 PM          Msg. 14 of 23
Thank you very much Tim.

Currently I'm testing my program with realtime (Level1) ticks. To build bars from tick I'm using the following felds of the Q messages: "Last", "Bid", "Ask", "Last Size", "Last TimeMS" and "Last Date".
Checks showed that value of "Last TimeMS" has a large difference sometimes (more than 5 minutes) comparing to current time. Should I use "Most Recent Trade", "Bid", "Ask", "Most Recent Size", "Most Recent TimeMS" and "Most Recent Date" instead? I want to build bars in realtime and the build bars should match your history interval data.

Thank you.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Apr 4, 2016 05:57 AM          Msg. 15 of 23
Most Recent TimeMS is what you would want. This time field only updates on a trade though, so having it go several minutes without changing, while the bid and ask values may be updating constantly can be completely normal. We would expect the time time value to update on each message that has a Message Contents of C, E, or O. Though, you should be filtering out trades with a message content value of 'O' if you want to match up to history.

Tim

semenchikus
-Interested User-
Posts: 12
Joined: Mar 28, 2016


Posted: Apr 4, 2016 09:25 AM          Msg. 16 of 23
Thank you again for quick reply!

So, in the other words, your historical ticks (response for any of HT* command) according to realtime ticks in the following way:
Last (history) = Most Recent Trade (realtime)
Last Size (history) = Most Recent Trade Size (realtime)
Time Stamp (history) = Most Recent Trade Date + Most Recent Trade TimeMS (realtime)
Bid (history) = Bid (realtime)
Ask (history) = Ask (realtime)

Is it correct?

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Apr 4, 2016 09:49 AM          Msg. 17 of 23
Assuming you are filtering out "O" trades and that you are using protocol 5.1, the new protocol usees different time fields.

Last (history) = Most Recent Trade (realtime) = true
Last Size (history) = Most Recent Trade Size (realtime) = true
Time Stamp (history) = Most Recent Trade Date + Most Recent Trade TimeMS (realtime) = true

A snapshot of these values are taken at the time of the trade message. The individual messages are not saved.

Bid (history) = Bid (realtime)
Ask (history) = Ask (realtime)

Tim

Craig
-DTN Evangelist-
Posts: 227
Joined: Apr 16, 2010


Posted: Apr 4, 2016 12:45 PM          Msg. 18 of 23
'It appears this is related to an issue that occurred back in 2015 during our initial roll-out of microseconds'

Can you please expand on this...it kind of seems like a big deal.
How invalid is the post 2015 interval data?

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Apr 4, 2016 12:55 PM          Msg. 19 of 23
There were 2 days, on the 18th and 19th of November 2015, where we received trades with a second value of 60 or more from Eurex, so we are rebuilding those intraday bars, for that exchange.

Tim

Craig
-DTN Evangelist-
Posts: 227
Joined: Apr 16, 2010


Posted: Apr 4, 2016 01:19 PM          Msg. 20 of 23
Cheers.

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Apr 8, 2016 11:08 AM          Msg. 21 of 23
Hello,

Just wanted to let you know all relevant changes were made to the futures contracts affected.

Sorry for the inconvenience,

Tim

quickTick
-Interested User-
Posts: 35
Joined: Nov 17, 2013


Posted: Apr 8, 2016 02:10 PM          Msg. 22 of 23
By the way: Some minutes actually do have 61 seconds. The last one was apparently at "June 30, 2015 at 23:59:60 UTC".

https://en.wikipedia.org/wiki/Leap_second

How is this handled by IQFeed? Wouldn't want my app to crash in such a case.... :)

DTN_Tim Walter
-DTN Guru-
Posts: 1238
Joined: Apr 25, 2006


Posted: Apr 8, 2016 02:38 PM          Msg. 23 of 23
Yep, those happen every so often. They get rolled into the previous second, luckily they happen when most everything is closed.

Tim
 

 

Time: Mon October 15, 2018 8:35 PM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003