Hi, I have recently switched to IQFeed 5.0 protocol with
"S,SELECT UPDATE FIELDS,Most Recent Trade Date,Most Recent Trade TimeMS,Most Recent Trade,Most Recent Trade Size,Total Volume"
This is all working fine except now occasionally I'm getting some spikes in the data which don't look like they are real trades. I'm attaching an example here (PNG file) which I've also verified by looking through IQConnectLog.txt:

Q,@ADU13,07/18/2013,07:10:36.004,0.9187,1,38571,
Q,@ADU13,07/18/2013,07:30:36.418,0.9187,19,39741,

Any ideas what is causing these? Are these errors/blips in the data or some sort of "non-qualified" trade? Is there any systematic way to filter these out?

Thanks!

Hello, looking at tick history (specifically at trade condition) for these 2 trades, they were both implied trades and were non-last-qualifying. Implied trades are generated by the exchange as a result of spread trading.

One of the new features in IQFeed 5.0 is that we now have the ability to send through all trades that the exchange provides along with the necessary information for you to decipher them and include/exclude whatever trades you want to see (or don't want to see).

The Most Recent Trade fields in IQFeed 5.0, include all trades that the exchange sends. We have also added 2 fields that will allow you to identify these trades and exclude them from your processing. The first field is the Most Recent Trade Conditions field (along with the Trade Condition lookup). The second field is the Message Contents field which will give a general grouping of trades C = last-qualified, E = form T, O = Everything not C or E. In this case, these trades would have come through with a trade condition indicating they were implied and a message contents field indicating they were "O".

With all that said, you have a few different options to handle this in your app. You can either add the trade conditions and message contents field to your current fieldset and filter these trades within your app, or you can switch your fieldset to use either the "Last" fields (Last, Last Date, Last TimeMS, Last Size) which will only update with last qualified trades or you can use the Extended Trade fields (Extended Trade, Extended Trade Date, Extended Trade TimeMS, Extended Trade Size) which will update with last-qualified trades + form T trades. If you are only watching futures, these 2 fieldsets will return the same data since FormT trades only apply to equities.

Thanks for this comprehensive reply, very helpful.

So having now had a chance to play with Most Recent Trade Conditions and Message Contents fields, I can see that all the spikes in my future currency data (e.g. @AD and @JY) where due to the occasional group O messages with condition 4D.

However... for crude futures QCL, these same group O messages seem to make up a substantial (~5%) portion of the data which look legitimate and without any spikes. In fact including these group O messages we are now getting much better agreement between IQ feed tick data for crude and the data we can download from TickWrite the next day.

So my question now is, is there any API for programatically finding out which symbols should include group O messages as part of normal data and which ones should exclude them? Or is this a user parameter that one would only know by trial and error?

Unfortunately this would have to be something to implement on your end. All the information we get from the exchange to describe these trades is already sent through to you via the feed.