We have been getting many reports from our users of problems associated with incorrect timestamps for data records in your historical intraday data. We believe this affects the tick data. However it may affect minute data as well.

The problem is there is out of order time stamping in your data base. This causes problems for us. The first problem is that if there is a time stamp for a tick record which is ahead of the time that is for, then all tick records following that record that have a preceding timestamp will get filtered out by our program. This is a special filtering to maintain the integrity of our database. One reason we do this is that if there is an invalid or out of order time stamping in the chart, this will cause our chart drawing tools to not function properly. This filtering manifests as missing data to the user.

Apparently some of your support people have been telling our customers that this is a Sierra Chart problem. It is not. It is a problem that you need to resolve on your side. We do plan to remove this filtering because of this problem. However, when we do remove this filtering users are still going to have problems using drawing tools. It will be appreciated that you promptly correct this issue and acknowledge that there is a problem. If there is not a problem, then could you identify to us why we are having a problem.

We have been telling people that you are a reliable service, but the problems that people are having are just as bad as other services.

A prompt response on this issue is much appreciated.

Thank you,
Anthony

Hello Anthony, are you reporting that the data is bad (incorrect timestamp) or are you saying that because the timestamps are out of order it causes problems for you?

Concerning the out of order trades:
Occasionally, the exchanges send us data out of order. You will see this in the Level 1 streaming data if you examine it closely. The IQFeed history servers do not reorder this data (it is served to your app in the same order it is received from the exchange).

On the other hand, if what you are reporting is a timestamp that is simply completely wrong and showing the wrong date, or something that is severely out of order (more than a few seconds), please give us an example of the data that we can investigate.

Incorrect timestamps definitely do cause problems for us and I would think they would cause problems for anyone. None of the data feeds we work with provide out of order timestamps. Your data feed has never done this until recently. I don't understand what is going on. You really need to provide timestamps that are always in order otherwise is going to be an ongoing problem. How is it that everything was OK before and now it is not? Therefore I don't really understand your answer. You are saying that the exchanges sometimes provide out of order data. However that was never the case in the past based upon what you have transmitted. Unless it was only off a few seconds.

It seems as though the time-stamp is simply incorrect but we have to look more closely.I think the timestamp is way off by more than a few seconds. But even a few seconds off is not very good. If you have been delivering data that has timestamps which are off by a few seconds, then that probably has gone unnoticed. But this problem definitely is vastly more severe than that.

Anthony
Sierra Chart

This is not a new occurance. The exchanges have always (to my knowledge) transmitted quotes in this way. However, in the past, IQFeed did not transmit seconds in the timestamp. As a result, unless the discrepancy occured on the rollover of a minute, it was not noticable (since the trades had the same timestamp).

Assuming we are talking about out of order (chronologically) trades, it is important to clarify that the timestamps themselves are not incorrect (the trade occured at the time indicated).

It is not entirely uncommon for a trade to be out of order by a few seconds (less than 5). It is fairly rare (but not entirely unheard of) for it to be out of order by more than a few seconds (would require investigation by our market data team). If you are talking about timestamps that appear completely wrong (for example: a trade for midnight comming in the middle of the day or a trade with the wrong date), I am more inclined to chalk that up as something in our servers rather than data the exchange sent us bad but would still require investigation.

Whichever occurace we are talking about, at this point, I am not willing to say that this isn't an issue on our end. Right now, I am simply letting you know what you should expect from IQFeed currently. The issue requires more investigation (which requires some examples of the data in question).

Being that the problem is intermittent, I do not have any example now. We also have taken out the filtering, which means it has less of an effect on our software.

If I read these posts correctly, the datafeed sends the actual trades, with the correct time stamps, but they don't always arrive in time stamp order ?

I believe that my graphing code could tolerate this, since it just draws lines between the next two data points. However, this could throw a big wrench in my analysis routines which assume that ticks loaded to a vector are moving forward in time.

I could add a method to reorder the ticks, which would be fine for graphing at the end of the day. Reordering would to be to expensive to apply in realtime though.

I may need to add a filter that drops any back dated ticks before they get added to my tick stream Vector. This approach might also work for SierraChart.

---

Joe paladin