Joined: |
Dec 29, 2008 10:57 AM |
Last Post: |
Jul 24, 2009 03:44 PM |
Last Visit: |
Jul 24, 2009 03:49 PM |
Website: |
|
Location: |
|
Occupation: |
|
Interests: |
|
Avatar: |
|
|
AIM: |
|
ICQ: |
|
MSN IM: |
|
Yahoo IM: |
|
|
smg1 has contributed to 14 posts out of 21224 total posts
(0.07%) in 5,648 days (0.00 posts per day).
20 Most recent posts:
Is the following statement in the developer's guide still accurate in light of the move to the FIX/Fast protocol or are the "a". "b" and "t' indicators now unique?
NOTE: These are listed in "priority" order. This means that an updated bid may come included in a trade message. Likewise, a settlement (normally an 'o' message) might come through included in a trade, bid, or ask message. Most data related issues with the feed are a result of not interpreting this field correctly.
(Going for the hat trick today with forum qxns)
Thanks in advance,
SMG1
Periodically, the Level 1 message stream will throw out an "F" message followed immediately by a "P" message as indicated below.
QXN: is the "last" trade that is included in the "P" message (13:15:01t) a unique trade or will a separate "Q" message have already been received for that very same trade?
"F,@USU9,13,,,136.500000,111.671875,,,,,,,,8,30,,,,,,,,,US T-BOND SEP 2009,,,,,,,,,,, , ,,0,4,6,,15.00,8,30,12/18/2008,06/11/2009,,,,,,09/21/2009,,,"
"P,@USU9,13,116.015625,0.390625,0.003378378,125691,1,116.390625,115.656250,116.015625,116.031250,12,65,,,0.734375,13:15:01t,669417,115.859375,115.625000,0.015625,,115.625000,,0,,,,,07/24/2009,,,09/21/2009,,,,,,,,,0.15625,1,0.006329966,,4,6,59,00,,0,0,0,0,32898,,,,0,,07/23/2009,"
Thanks in advance.
SMG1
Thanks Steve....just to confirm....not sending for trade updates either (those delivered under the "Q" message type as shown in the example below?
"Q,@USU9,116.000000,1,115.984375,116.000000,111,8,13:15:29t,0,"
SMG1
All tick ID's are coming in as 0 in my Level1 Feed for @USU9
Limited field set response shown below for field order, along with an example response "Q" update. Only included one line (but all have 0 in the TickID field)
"S,CURRENT UPDATE FIELDNAMES,Symbol,Last,Incremental Volume,Bid,Ask,Bid Size,Ask Size,Last Trade Time,TickID" "Q,@USU9,116.015625,1,116.000000,116.015625,26,37,13:15:49b,0,
Thanks in advance,
SMG1
When viewing tick history, is the Bid /Ask price that is presented along with each tick,
1) The Bid/Ask showing at the time of the trade execution 2) The Bid/Ask showing Pre trade
i.e if 4 are offered at 40 and 2 are offered at 40.01 and a trade that buys 6 at the market is entered, would the tick history show: 4 @40.00 with an Ask of 40.00 and 2 @40.01 with an Ask of 40.01 or 4 @40.00 with an Ask of 40.00 and 2 @40.01 with an Ask of 40.00
I suspect the latter which is how we get buys and sells above and below the reported bid / ask?
Thanks in advance,
smg
Has to do with the data streams that CME group sends out. Those type of trades are not included in the feed that DTN is receiving at the moment (ITC 2.1). They are currently testing the FIX/Fast feed, which is the new standard. That feed has those trades in it, but they have not yet determined if they will include them. It should be noted that CQG who is on the Fix/Fast stream does not include them with their platform, so there is room for variance across vendors here should DTN decide not to pass them on.
Also confirmed that DTN does not aggregate anything either. On the ITC 2.1 feed, the exchange will occasionally aggregate onesey twosey type trades where the time stamp second is the same as well as the price and bid ask.
We all learned something on this one and I have to say Jay and DTN were great at getting to the bottom of this issue. I will be happy with their decision on the inclusion / exclusion of those exchange created spread ticks either way, though my personal bias is to include them flagged as the variant type that they are.
Feel free to PM me if you want to diiscuss further
SMG
Does DTNIQ aggregate ticks? It appears they do based on the reconciliation work I did between Tradestation / Bloomberg / DTN data feeds. Very well could be different in Streaming Quotes vs History, but if it is, that doesn't seem logical. i.e. If I develop models that are tuned to historic tick volume, but then implement them on streaming quotes, it is another variable to have to control for. Not trying to be a pain, just want to understand the data feed.
Kind regards and thanks in advance for the response.
Example: CLG9 - 1/5/09 Bloomberg 17:14:03 2 @ 48.25 17:14:03 3 @ 48.24 17:14:03 1 @ 48.24 Tradestaion 17:14:xx 2 @ 48.25 17:14:xx 3 @ 48.24 17:14:xx 1 @ 48.24 DTNIQ 17:14:03 2 @ 48.25 17:14:03 4 @ 48.24
Was going back over the file and noticed something.....(talk about not being able to see the forest thru the trees).....DTN Feed H,L,O,C..........TS Feed O,H,L,C.....that resolves many of the differences, but there still are far to many differences in the file as well as the missing minutes. (Sheet with revised Delta formulas attached)
CLG09 in TS is actually electronic only, CLG09.P is their pit, and CLG09.C is their combined file, so if anything, the TS CLG09 should contain fewer ticks than +CLG9.
I did some more digging and hit up a friend's Bloomberg machine and believe I found the difference: TS has "EP" ticks in them.....EP on Bloomberg stands for Exchange Generated Price and involves the exchange generating a price when spread orders cross....see reconciliation spreadsheet in the attached zip file for the detailed Bloomberg explanation.
With that in mind, I expect that DTN should be passing those ticks along and including the extended exchange code. They do come into play for highs and lows. It should be the user's choice as to whether they want to ignore them or not.
Your thoughts?
I have seen it occur frequently. Columns are just a visual way of lining up the ticks. When I hit a gap (or a sequence of trades where one might show a volume of 3 and the other have 3 distinct ticks) I shift over to the right and get the rows lined up again. I do that 3 times then shift back left just to keep in the same screen window without having to scroll. Ticks highlighted in green were where a gap occurred....almost 100% Tradesstation had a tick DTN didn't. The one pink one was where I found the missing tick in a subsequent repull of the DTN data.
That being said, please see the attached Excel spreadsheet. It is yesterday, 1/5/09 in one minute intervals as pulled via the following request: "HID,+CLG9,60,2" & vbCrLf. The tradestation dump was just their 1 minute intervals for the same time period......symbol CLG09 in their syntax. It was more expedient than doing a tick by tick, however, I have enclosed the TS tick by tick as a text file for your reference. If I was chasing this down, I'd look at the one minute file and conentrate on the completely missing minutes from the DTN section. You will also note that I have included a Delta column and in every instance where there is a difference (except where DTN is missing a full minute) TS High is higher and TS Low is lower. Seems systemic to me and migh be as simple as me not understanding what +CLG9 really is (I understand it to be a union of the electronic and pit ticks)
Thanks.
Sinon
DTNIQ historical tick data: +CLG9
As a bit of QA / QC on some code I am writing, I ran a compare of "+CLG9" against my Tick Feed from Tradestation and it looks like I am either not pulling the right ("complete") symbol from DTN or there are some ticks missing from the feed (possibly corrections inserted by TS?) Thought it could be timestamp differences but the open and close ticks line up. If they are corrections, they are material inserts, because they expand the high AND low.
The attached pdf lines up the 9:00 AM est minute from DTN and the same minute from Tradestation on a slow day 2008-12-26. The differences are all one sided, with TS having more ticks than DTN. Green ticks are the ones that appear to be missing. (The one tick highlighted in pink is either an error in my process or a missing packet etc in the original download, because I did mange to find it in a subsequent repull when double checking before posting this)
I realize they are only ticks, but when my analytics feed differs from my execution feed, especially in what appears to be ticks at the extremes of the minutes (i.e. highs and lows), I have to understand the difference.
Thanks in advance and apologies if (as is likely) the difference is attributable to my lack of understanding / newbieness (and positng in the wrong section originally)
SMG
Is it possible to have the 8 day limit during 8:30 to 4:30 lifted on weekends so that up to 30 days worth of tick data can be downloaded during said hours on Sat & Sun?
Thanks for the consideration.
SMG
Trying with a Zip of the pdf, since the page doesn't appear to like pdf's?.
As a bit of QA / QC on some code I am writing, I ran a compare of "+CLG9" against my Tick Feed from Tradestation and it looks like I am either not pulling the right ("complete") symbol from DTN or there are some ticks missing from the feed (possibly corrections inserted by TS?) Thought it could be timestamp differences but the open and close ticks line up. If they are corrections, they are material inserts, because they expand the high AND low.
The attached pdf lines up the 9:00 AM est minute from DTN and the same minute from Tradestation on a slow day 2008-12-26. The differences are all one sided, with TS having more ticks than DTN. Green ticks are the ones that appear to be missing. (The one tick highlighted in pink is either an error in my process or a missing packet etc in the original download, because I did mange to find it in a subsequent repull when double checking before posting this)
I realize they are only ticks, but when my analytics feed differs from my execution feed, especially in what appears to be ticks at the extremes of the minutes (i.e. highs and lows), I have to understand the difference.
Thanks in advance and apologies if (as is likely) the difference is attributable to my lack of understanding / newbieness.
SMG
Newbie Qxn.....Is TickID a GUID (globally unique ID) or is it a unique identifier when combined with some other fields i.e. in the case of a futures contract:
Symbol,Contract_Month, Year, TradeDate + TickID guaranteed to be a unique ID across all of dtn?
Thanks in advance,
SMG
|
|