Hi,

I am trying to build daily bars from the tick stream that match the 'exchange' daily bar. The problem is with instruments that don't have a typical day, such as various futures. For example, for ES, the open is at 17.00 the previous day.

The problem is that I can't seem to find any information about the opening time/closing time of a particular instrument? Can this data be retrieved from the feed? Something like below:

http://www.cmegroup.com/trading_hours/

I've done some basic testing and it seems that IQfeed's daily bars do seem to match the exchange (for @ES# at least) is there any documentation describing how IQfeed create these daily bars?

Thanks.

Unfortunately, we do not have exchange session hours in the feed at this point so your only real option is to compile this information externally and apply it to the data from the feed.

In the case of @ES (and pretty much everything else), all trades for a session get applied to day in which the session closes. So, all trades sunday night through Monday when trading stops are used to build Monday's bar. Likewise, when trading opens again Monday evening all of those trades will be applied to the Tuesday Daily bar.

Hope this helps.

Hi Steve,

Thanks for the reply. Yeah, we'll be able to get the session hours externally.

I've just been looking at the 1 minute bars for @ES# and comparing them to the daily bars, and in some cases I can't seem to match them up. Do you know why?

For example, for the 12th of February 2014:

Daily Bars:

Open = 1813.25
Close = 18.17

Which seems to match other sources

For 1 minute bars, I see (again from IQfeed):

11th Feb 17:00, Open = 1812.75
12th Feb 16:15, Close = 1817.25


Other bars seem fine, while others seem a point or so out. Thoughts?

Regards,

You won't be able to consistently build daily bars from minute data because our minute data contains Extended trades (Form T trades) which account for almost all pre/post market trades but will also be intermixed in regular trading hours as well.

Thanks for the reply.

Can the T trades be filtered out at the tick level?

Yes, tick data identifies every trade as last qualifying ("C") or not ("E" or "O"). Only last qualified trades are accounted for in OHLC but all trades are accounted for in volume.

Ok, so if I had this pseudo logic:

if (tick->type == C) {
include the tick in calculating the OHLC for the day;
}

would this end up matching the exchange day?

Yes, that should work

Steve,

Just to clarify a few things.

We can understand why the close would be different as there some logic that happens at the exchange but we're a little confused about the open. Wouldn't the open just be the fist tick?

Another question, when you say "T trades", do you mean trades that aren't electronic, but end up getting included into the feed and in the case of your (intra-day) bar data they are included?

Regards,

I didn't say "T trades"...I said "Form T trades". Form T is simply a trade condition that the exchange applies to trades. Generally speaking, Form T trades compose most of the pre/post market trading and they usually are in line with the regular trading prices so they still look nice on a chart (which is why we include them in intraday data).

As for the Open, yes, we currently build daily data bars using Open = First last-qualified trade.

Thanks for the clarification.

Sorry for more questions:

Is there any intra-day bar based fetch we can do that will get us bars that have these Form T trades excluded (again, so we can build EOD bars and match the exchange from intra day data).

Do you know when the exchange day bar is available (for futures)? If we pull it (say) 5 minutes after close, would the data be available?

Regards,

Unfortunately, no. The minute data is compiled this way at the end of each day and so the only way to pull the Form T data out will be through the tick filtering Steve mentioned and then building your bars off of that data.

For futures, it will be available on settle, maybe plus a few seconds for history to process, but the time of the settle cannot be guaranteed. Sometimes this happens very quickly, sometimes the exchange doesn't send a settle for a couple hours after close. To get the final bar as soon as possible, you can do a level one watch of a symbol and check the settle date, if the settle date equals today, then you do your request for the daily bar, given time for it to replicate through our systems.

For equities, in case you may need it, due to the extended trading that happens, the market is not closed completely until 20:00. So while there will be a daily bar available as early as around 15 minutes after the normal close time, it will continue to update, with corrections and extended trading throughout the evening. There is not a time that you could request a bar that would encompass only the normal trading time range.

Tim

Thanks Tim.

Do you know if there is any way to find out how often there is a 'delay' in settle?

Regards,

No, and it is not in any way predictable, it is just a matter of when they release it is when we show it.

The exchange does place any time guarantee on thier sending of this data, as calculations can need reworked or market events can happen that require correction prior to release or maybe something as simple as someone forgot to hit send after entering the data. These are not common occurences by any means, but I want you to be aware so you are not ambushed by them.

Tim

Hi,

We've done some testing on the approach mentioned in this thread (watching the settlement date for a change), but we're a little confused to the time it changes:

Symbol,time EST (and ticktype),Settlement Date,Day Open,Day High,Day Low,Day Close,Last Trade Date,

@ES#,16:14:59t,02/24/2014,1844.75,1851.00,1837.25,1846.00,02/25/2014,
@ES#,16:14:59t,02/24/2014,1844.75,1851.00,1837.25,1846.00,02/25/2014,
@ES#,16:30:00t,02/25/2014,1844.75,1851.00,1837.25,1846.00,02/25/2014,
@ES#,16:30:00t,02/25/2014,1844.75,1851.00,1837.25,1846.00,02/25/2014,

Why would it change at 16:30 and not at 17:15?

Regards,

Settlement logic is based on activity from 16:14:30 – 16:15:00 per the CME group website specifications on this contract, hence the timing of the release. They do not account for the afternoon activity in the settle value.

Tim

Hi Guys,

Sorry to resurrect this thread, but we're really struggling to make sense of what we're seeing with the feed. Hopefully it's something we're not understanding or doing wrong.

What we need:

To build an OHLC bar that matches the official day exchange bar for a collection of futures and get this *before* the market options.

An example:

ES, closes at: 4:15pm, and we want to get this bar before 5.00pm when the market opens again.

What we have done:

We've attempted to get this from a live tick stream using the following fields:

"S,SELECT UPDATE FIELDS,Symbol,Last,Incremental Volume,Bid,Ask,Bid Size,Ask Size,Last Trade Time,TickID,Settlement Date,Open,High,Low,Close,Settle,Last Trade Date,Total Volume\r\n"

Then placing various 'watches' on the tick stream for things like settlement date changes, and looking at the Open,High,Low fields when the market closes. We do the watch for the close so we can capture any change to the High and Low prices after the settle.

Which leads to the first question, is this approach valid (we wanted to get the data in the tick stream)? Or should we do a historical daily fetch when the market closes/ just before the market opens?

What we have seen (for Friday the 21st):

We didn't get any settlement changes for @US# or @EU#, so we couldn't work out the official close.

Any thoughts on what other approaches should we use?

Regards,

In re-reading this my last post was confusing so my apologies on that.

The market hours for @ES markets run from 17:00 EST to 16:15 EST each day. The settlement for each day is released, normally, between 16:20 and 16:30. The data used in this calculation includes data from the 16:14:30 - 16:14:59 time frame. So this means that we cannot get the settles prior to the day's close.

So if you are watching the feed and you see a settlement date change, we can say that the day is complete, given the logic above, which means that you can now read the settlement value, as well as the Open, High, and Low and know that it is the final data for the day.

Hopefully this helps, but just let me know if I can clarify anything further for you.

Tim

Hi Tim,

Thanks for the reply. Question, do you know why we wouldn't get settlement for those two other symbols?

Just to clarify, we're trying to get the OHLC values for the previous exchange traded day bar before the market opens again at 17:00.

Regarding @ES, our understanding (which has been confirmed from someone from CME E-quotes supports) is that:

The High and Low of the session can happen after settlement which is why we are watching for those values. In other words:

Settle at 3.15
but the High and Low can be set between 3.30 and 4.15.

Which is why we can't just watch the settle date change. Does that make sense to you guys (or is the CME guy wrong)?

If we abandon this tick approach, and go for a historical pull do you know how/when your daily data is updated?

Just to clarify some times:

All CT, NOT EST:

For ES:
settle at 15:15
close at 16:15

the session high low can be set between 15:30 and 16:15

I am not sure how you could get a new high or low when the market is closed. I have not seen this personally. What conditions did he say would cause the high and low to be set while trading is closed?

As to the other symbols, were you watching for P messages as well as U messages for the update? I show settles came through between 5:10 and 5:20 for the first few contracts of each of these on the 21st.

[As to the other symbols, were you watching for P messages as well as U messages for the update? I show settles came through between 5:10 and 5:20 for the first few contracts of each of these on the 21st.]

Arh, that could be the piece we're missing. We're currently only looking on Q messages (for the settle change), we'll start looking at the P messages as well (U message, what's that?).

[I am not sure how you could get a new high or low when the market is closed. I have not seen this personally. What conditions did he say would cause the high and low to be set while trading is closed?]

The market isn't closed...

For ES (EST times):

settle at 16:15
close at 17:15

So, yeah, the high and low can be set after the settle price (close).

Ok, sorry, I think I am with you now. I am verifying the times and processes used to calculate the daily bar and will reply shorty.

Tim

Ok, what you will want to do is watch for the settle and then monitor the high and low throughout the afternoon, updating any high or low values that might be set.

Alternatively, we build our daily bar for history at the close, 17:15, using this same logic throughout the afternoon, so you can wait and just pull that bar and save yourself the trouble. I will verify how long it can take for that bar to propegate throughout our systems and let you know.

Tim

]Alternatively, we build our daily bar for history at the close, 17:15, using this same logic throughout the afternoon, so you can wait and just pull that bar and save yourself the trouble. I will verify how long it can take for that bar to propegate throughout our systems and let you know.]

Thanks, this would be good to know. On a related topic, what happens if you don't get a settle before 17:15?

We would close the bar with the last trade I would assume and then the settle would trigger a refresh of that bar, correcting it, and the level one data. Though I can't think of a time when that has happened that I heard about from our customer base.

Tim

You can do your pull at 17:30 for that daily bar. It should be there just after 17:15, but to allow some time for what-ifs I tossed a buffer in. I would consider just setting up a loop to check once a minute myself, that would guarantee I have my data as soon as possible.

Tim