Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I noticed that ******* quotes locked up shortly after the interest rate announcement yesterday while yours stayed stable." - Comment from Ron in Utah
"IQ feed works very well, does not have all of the normal interruptions I have grown used to on *******" - Comment from Mark
"With HUGE volume on AAPL and RIMM for 2 days, everyone in a trading room was whining about freezes, crashes and lag with *******, RealTick, TS and Cyber. InvestorRT with IQFeed was rock solid. I mean SOLID!" - Comment from Public IRC Chat
"I just wanted to let u know that your data feed/service is by far the best!!! Your unfiltered tick data is excellent for reading order flow and none of your competitors delivers this quality of data!" - Comment from Peter via Email
"I used to have *******, but they are way more money for the same thing. I have had no probs with data from DTN since switching over." - Comment from Public Forum Post
"Its working FABULOUSLY for me!! Holy cow...there has been so much I've been missing lately, and with this feed and Linnsoft software...I'm in the game now." - Comment from Chris R.
"Just a thank you for the very helpful and prompt assistance and services. You provided me with noticeably superior service in my setup compared to a couple of other options I had looked at." - Comment from John
"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"IQFeed version 4 is a real screamer compared to anything else I have seen." - Comment from Tom
Home  Search  Register  Login  Blogs Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTN_IQFeed on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Asking about historical data timestamps.
Author Topic: Asking about historical data timestamps. (8 messages, Page 1 of 1)

lgwaustralia
-Interested User-
Posts: 8
Joined: Apr 11, 2016

Trade on


Posted: Feb 27, 2018 03:13 AM          Msg. 1 of 8
I wanted to ask about the timestamps of some historical data. Also would like to ask about the timezone if it's still EST and if timestamps observer daylight savings time.

So i have this set of timestamps and trade opens at 2am EST, changes to 3am around november so that's the daylight savings, then goes back to 2am on nov 11. I thought it would have stayed at 3am until march when it reverts back to 2am: (Edit: pm to am)

(shows only specific dates where transition occurs)
(transtion from 2p to 3p on nov2016)
"2016-09-01 02:00:04.381"
"2016-09-15 02:00:03.884"
"2016-09-20 02:00:06.72"
"2016-10-20 02:00:05.545"
"2016-10-25 02:00:37.324"
"2016-10-31 03:00:05.133"
"2016-11-01 03:00:04.63"
"2016-11-02 03:00:05.146"
"2016-11-03 03:00:03.583"
"2016-11-04 03:00:03.148"
"2016-11-07 02:00:03.947"
"2016-11-10 02:00:06.397"
"2016-11-15 02:00:06.662"
"2016-11-24 02:00:03.19"
"2016-11-25 02:00:06.682"
"2016-11-28 02:00:03.653"
"2016-11-29 02:00:05.423"
"2016-11-30 02:00:05.156"

(transition from 2p to 3p in mar2017)
"2017-03-01 02:00:04.376"
"2017-03-10 02:00:06.082"
"2017-03-15 03:00:04.078"
"2017-03-20 03:00:04.517"
"2017-03-27 02:00:07.559"
"2017-03-28 02:00:04.393"
"2017-03-29 02:00:03.89"
"2017-03-30 02:00:05.692"

(transition from 2p to 3p in nov2017)
"2017-10-05 02:00:10.811"
"2017-10-10 02:00:07.772"
"2017-10-20 02:00:05.902"
"2017-10-25 02:00:07.685"
"2017-10-26 02:00:05.272"
"2017-10-27 02:00:04.593"
"2017-10-30 03:00:07.905"
"2017-11-01 03:00:08.247"
"2017-11-10 02:00:05.471"
"2017-11-15 02:00:04.3"
"2017-11-20 02:00:09.104"
"2017-11-27 02:00:10.51"
"2017-11-28 02:00:16.352"
"2017-11-29 02:00:13.293"


Edited by lgwaustralia on Feb 27, 2018 at 07:34 AM

DTN_Tim Walter
-DTN Guru-
Posts: 1181
Joined: Apr 25, 2006


Posted: Feb 27, 2018 04:30 AM          Msg. 2 of 8
What symbol are you looking at here? I'll take a look to see if I can figure something out. Everything in our system is in Eastern time, but since markets don't trade Sunday morning, DST doesn't make us much difference. Markets always open at the same time.

Tim

lgwaustralia
-Interested User-
Posts: 8
Joined: Apr 11, 2016

Trade on


Posted: Feb 27, 2018 07:30 AM          Msg. 3 of 8
Thanks for replying.

We are using symbol XG# with method HTT.

> but since markets don't trade Sunday morning, DST doesn't make us much difference.
Ah since dst change happens on a sunday, then monday comes and should still open at 2am ET is that correct?

Thanks again.

Lance

DTN_Tim Walter
-DTN Guru-
Posts: 1181
Joined: Apr 25, 2006


Posted: Feb 27, 2018 07:39 AM          Msg. 4 of 8
This is because of differences between the dates that European and American sources implement DST. We roll about a week or two earlier in the US than the European markets, so for that period, you will have this weird offset in history.

Tim

lgwaustralia
-Interested User-
Posts: 8
Joined: Apr 11, 2016

Trade on


Posted: Feb 27, 2018 08:04 AM          Msg. 5 of 8
Okay I see now. US goes, with dsts, 2nd sunday of march while they do last sunday of march and in between those dates it will appear as 3am. The same happens with the dates during transition back, last sunday of october for europe and first sunday of november in US.

Thanks.

Lance

lgwaustralia
-Interested User-
Posts: 8
Joined: Apr 11, 2016

Trade on


Posted: Mar 5, 2018 04:05 AM          Msg. 6 of 8
Hi again.

I checked the dates in question above ninjatrader using iqfeed as source and loaded up the date range in question. FYI, the times are shown in localtime (PHT) in ninjatrader.

I found that the start times shown in ninjatrader seem to follow the daylight savings time. It changes start of open from 15:00 / 3pm PHT on March 24, 2017 (Friday) to a 14:00 / 2pm PHT on March 27, 2017 (Monday).

It then goes back to starting at 15:00 / 3pm PHT on October 30, 2017.

So how do we deal with this? Should we:
- update the historical data we have in database to follow DST conventions (changes to 3am EST on march then goes back to 2am EST past last sunday of october)
- OR change them so they will all start 2am EST.

Thanks.

Lance

DTN_Tim Walter
-DTN Guru-
Posts: 1181
Joined: Apr 25, 2006


Posted: Mar 5, 2018 06:08 AM          Msg. 7 of 8
Good morning,

I guess if I was in your shoes I would push the inconsistent data towards whatever the exchange norm. But, I don't think there is a right or a wrong, it is up to how do you want to see it that matters.

Tim

lgwaustralia
-Interested User-
Posts: 8
Joined: Apr 11, 2016

Trade on


Posted: Mar 6, 2018 03:21 AM          Msg. 8 of 8
Tim,

Thanks for the response. How come IQFeed is not fixing these issues before we get the data.

Are you saying that Ninjatrader is adjusting the data in real-time as it comes in from your servers?

How do they know to do this?

Is there a flag in the data that says the following tick timestamps will all be out by X amount and you need to adjust it?

What other instruments does this happen with?

I was expecting that the data we got from IQFeed was high quality data, and did not contain issues like this that needed to be fixed by end users.

Regards.


Lance
Edited by lgwaustralia on Mar 6, 2018 at 04:17 AM
 

 

Time: Sat June 23, 2018 10:00 PM CFBB v1.2.0 0 ms.
© AderSoftware 2002-2003