I'm using HTD to request tick data on SPY. I've copied a continuous portion below:

2014-09-23 09:30:00,198.4300,100,4601873,198.4300,198.4300,4305,0,0,C
2014-09-23 09:30:00,198.4400,100,4601973,198.4300,198.4300,4306,0,0,C
2014-09-23 09:30:00,198.4100,333382,4935355,198.4200,198.4300,4318,0,0,C
2014-09-23 09:30:00,198.4300,500,4935855,198.4200,198.4300,4321,0,0,E
2014-09-23 09:30:00,198.4300,500,4936355,198.4200,198.4300,4322,0,0,E
2014-09-23 09:30:00,198.4300,500,4936855,198.4200,198.4300,4323,0,0,E
2014-09-23 09:30:00,198.4300,500,4937355,198.4200,198.4300,4324,0,0,E
2014-09-23 09:30:00,198.4400,100,4937484,198.4200,198.4300,4326,0,0,C
2014-09-23 09:30:00,198.4400,200,4937684,198.4200,198.4300,4327,0,0,C


1. TickID on line 2 is 4306 and on line 3 in 4318. However, the Last Size on line 3 equals the change in the Total Volume between lines 2 & 3. So is this an aggregation of trades? If so, how is it determined that those trades should be aggregated.


2. Between lines 7 & 8 TickID 4325 is missing and the volume is not accounted for. The Total Volume increased by 129 between lines 7 & 8, but the Last Size in line 8 is 100. TickID 4325 is not in the rest of that day's data.

a. I've seen terms "implied trade" / "non-last-qualifying" used elsewhere in this forum, but I'm not sure how/if "implied trade" applies to equities and I really can't find the term "non-last-qualifying" outside of iqfeed. If this is the reason for the unaccounted for volume above, can you direct to a more substantial definition of "non-last-qualifying" (an exchange generated document perhaps)?

b. If I was watching / recording SPY in real-time, would I find these 29 shares with a TickID of 4325?


Much appreciated,
Ryan

Ryan

I have attached a png file to this posting which shows the time stamps/tick ID/info/mkt center/trade conditions.

I hope this answers your question.

Expanding on Loren's answer, tickIDs are not guaranteed to be sequential or even always increasing. They are only guaranteed to be unique per symbol per day. In most cases (whenever available), these are provided by the exchange and are intended to be used only as a unique trade identifier so they can't reliably be used as an indicator of missing data.

Loren's email gives you a more complete picture of the actual trade information available in the newer IQFeed protocols (see the API docs for information on switching to the new protocol). As you can see in that screenshot, Line 3 from your data corresponds to the Opening trade received from NYSE_ARCA. We have verified that the volume of this open trade is what we received from the exchange. Unfortunately, we do not know exactly what that volume represents but I would guess you are correct that it is an aggregation of some kind of orders at NYSE_ARCA. Our market data group has sent a request for more information about this from the exchange but I can't guarantee we will get anymore information from them (we will let you know if we do).

Also in Loren's screenshot, you can see the "Other Trade" of volume 29 which accounts for the missing volume from your data. Trades are generally categorized by the exchanges in 2 categories, Last-Qualifying and Non-Last Qualifying and the different is exactly what it sounds like (A non-Last-qualified trade will not set the "Last" price). At DTN, we go one step further and break up the non-last-qualified trades into 2 categories as well, Extended Trades, and Other Trades. The reasoning for this is that some non-last-qualified trades (specifically FormT trades) account for the vast majority of pre/post market ("Extended") trading and are generally inline with the last qualified trades in terms of pricing and trends. All other non-last-qualified trades ("Other Trades") have wide swings in prices that will cause charts to look bad and throw off analytics.

Here are a couple references to exchange documentation concerning this as you requested:
http://nasdaqtrader.com/content/technicalsupport/specifications/utp/utdfspecification.pdf
7.5.1.1. UTP Trade Condition Matrix

https://cta.nyxdata.com/cta/document/5944/2f76d73065c5d0f9629eb576302adc1da66d1f9f
Page 113

Let us know if you have additional questions.

Loren- Is the app in your screenshot an updated HistoryViewer? I only seem to have the one with the hex codes (pain in the butt). Thanks.

That screenshot is from the DTN.IQ software. But, I'll make a note that adding a name column for trade conditions would be helpful in the history viewer.

Tim

Thanks Tim