I use IQFeed for my real time data, and Interactive Brokers as my broker.

Today, while trading a stock, I noticed that there were consistent large discrepancies between the quote data provided by IQFeed and IB throughout the day.

The discrepancies were not so much in the bid & offer prices but in their sizes: IQFeed was regularly displaying sizes 1/2 or 1/3 of what IB was reporting.

I called both IQFeed's and IB's technical support today to try and track down this discrepancy.

IB told me that they form the sizes in the intelligent way that anyone doing trading would wish, namely, they aggregate all the sources (if there are multiple ones) at the best bid and offer to show the total liquidity available at those prices.

IQFeed, in contrast, told me that they do NOT aggregate: they just report the size of just one of the sources of the best quote. For example, if the best bid price is $10 and both ARCA and ISLAND have shares at that price, say 500 and 1000 shares respectively, IQFeed will only report a bid size which comes from one of them, say the largest (1000), and does not sum them up (1500) like IB does.

Would someone from IQFeed please respond and confirm whether or not what I was told by your technical support guy is correct?

And if it is correct, why do you do this? This makes your data greatly inferior for trading. Is there any possible way that you could change your system to report accurate information for traders?

And if it is not correct (i.e. you do aggregate sources), then we need to start an investigation to find out why your data was so far off the mark.

What you are told is correct. We provide the BBO from the exchange, which is what the exchange requires us to show. What you may be seeing from IB is an aggregate of ECN quotes as well. We don't carry the ECN feeds on IQFeed.

---

Jay Froscheiser
DTN Market Access, LLC.

>We provide the BBO from the exchange, which is what the exchange requires us to show.

So you just show the NYSE quotes for listed stocks, and NASDAQ quotes for OTC stocks?


>What you may be seeing from IB is an aggregate of ECN quotes as well.

I am sure that it is.


>We don't carry the ECN feeds on IQFeed.

Are there any plans by you guys to do so in the future?

There is now quite a bit of liquidity information that you are missing if you do not carry them, right?

Is the issue here simply that the ECNs want to charge you too much money, or is it just the current state of your technology?

It isn't the state of our technology. It is more a function of ROI. The fees to bring the feeds in (communication lines primarily) and development to implement them isn't currently justified by demand We are always looking at demand for various feeds, and will be implementing new feeds as the demand justifies it. ECN feeds are on the list of things we are looking closely at.

---

Jay Froscheiser
DTN Market Access, LLC.

Jay:

I'm a little confused by your comment "We don't carry the ECN feeds on IQFeed."

I thought Island (INET) , ARCX, & BRUT are all ECNs. Can you exapnd on this a little?

Jazz:

I'm not convinced there is a "right" vs "wrong" way to report this.

Using your example, you might want to ask IB if you placed an order for 1500 shares using SMART routing, if they would split your order and sent 1000 shares to ARCA and 500 shares to ISLAND. I suspect most brokerages would send all 1500 shares to a single exchange for execution. My hunch is IB will tell say you would have to manually send 2 orders.

>I'm not convinced there is a "right" vs "wrong" way to report this.

I am!

I cannot imagine any trader who would NOT want to see all the market liquidity that is available on the market.

In principle, I understand that DTN currently finds it too expensive to include the ECNs (but why does IB not find it so expensive?), but I cannot imagine it ever being good for you as a trader to not have the info available.

The only change that you might want from what IB does is to have even more information (e.g. a breakdown, by exchange, of the liquidity at the BBO).

If you still disagree, please post a counter example and educate me.

In saying this, I am assuming that your broker offers you access to all the exchanges; if your broker, say limits you to NYSE or NASDAQ then perhaps you might only want to see the quote size available there. Actually, even for this case, you probably still want to see the total (since that can strongly hint at the real buying or selling pressure) as well as the size at the exchanges at which you can execute (since that is all the size that you will be able to get off). So this case argues strongly for having even more information than what IB provides.


>Using your example, you might want to ask IB if you placed an order for 1500 shares using SMART routing,
>if they would split your order and sent 1000 shares to ARCA and 500 shares to ISLAND.

That is my understanding of what they do with SMART routing. The whole point of SMART routing at IB is that you do not have to send in multiple orders, they will do all that for you behind the scenes using logic on their servers.

I asked the people at IB why you would ever not want to use SMART order routing, since it is so much better in general. They said that the only people for whom it does not make sense are the people who are doing exchange arbitrage, so they need precise control over what exchange gets their order.


>I suspect most brokerages would send all 1500 shares to a single exchange for execution.

Then they don't have SMART routing. (I have no idea if the concept is unique to IB, but I would be very surprised if so, since it is such an obvious thing that you would want to have. Then again most brokerages have really crappy technology, so maybe they have been cheap and just not invested in this capability.)


>My hunch is IB will tell say you would have to manually send 2 orders.

They have never done that to me.

Ignoring fast markets, where there are complex timing issues (i.e. what you see on your screen is a delayed and inaccurate representation of what might actually be available), I have ALWAYS been able to put in, say, a single limit order at the best bid and pick up all the shares that were offered. Occaisionally, the filling of my shares comes in stages; I have never thought of this before, but maybe it is IB getting reports back from different exchanges at different times.

Who do you use as your broker? I recommend IB actually. They aren't perfect, but are decent.

I'm at office now but will post longer reply later from home.

One question for you. Are you using the API from IB and/or IQ? or just the TWS from IB and DTNIQ software from IQ?

>One question for you. Are you using the API from IB and/or IQ? or just the TWS from IB and DTNIQ software from IQ?

I am using the Java APIs for both, since I am writing a fully automatic trading system in Java.

With IB's Java api, this is NOT a socket tap into their remote servers. Instead, you need to be running a copy of their client software, TWS (Trader WorkStation), and you make a socket connection to that, which in turn internally communicates with IB's remote servers. I am in the middle of writing the software to auto execute my model's orders; it is more of a pain than it should be, because IB could have made their api a lot better.

With IQ/DTN's java api, you are essentially doing the same thing: opening a socket connection to some local software (in this case a windoze dll--bad choice on DTNs part, since now you are stuck with using windoze as your OS, wish they would have written the whole thing in Java, but thats another topic...).

That makes 2 of us, and I think there's another, Skunk, and maybe more. I'm working on an ATS using both APIs under VC++.

With programming skills, you have access to all the tools and info you want/need. With IQ, enable regional quotes. Check the API docs on how to do this. You can then see the BA and corresponding size for each market. With IB you can request mkt data both for SMART and for the individual exchanges you want to monitor closely. Then you can create Level 1 quotes based upon your own criteria.

I use the data feeds from both providers and compare them within my app. Both have strengths and weaknesses. I log all messages from IQ & IB so I can create a playback mode for my app. It's easy with the IQ messages since they come as one string. IQ is a little trickier since they do not include the ticker, but rather the ID in the TickPrice & TickSize events. IB does not report every trade and that is intentional on their part. IQ does report every trade (or at least tries to) and gives you the market center where the trade executed. With that info, you can see where the issues you follow are most active and focus on the regional quotes for the active markets. Check out the thread I started on Bid/Ask crossed situation. I resolved this by "ignoring" the specific market which almost always caused crossed BA (AMEX for QQQQs).

Also, by logging the IQ data, you may be able to see what IB is really doing to execute your orders if you can match your fill(s) to the IQ print data.

This may also contain useful info:

http://www.interactivebrokers.com/download/4Q-2004-IB-Order-Routing-Report.pdf

>That makes 2 of us, and I think there's another, Skunk, and maybe more. I'm working on an ATS using both APIs under VC++.

You live in the NYC area? Ever wanna talk shop?


>With IQ, enable regional quotes. Check the API docs on how to do this.

Thanks--I had actually not encountered that or realized the significance of it when reading their docs myself.

They describe it at
C:\Program Files\DTN\IQFeed\docs\ReceivingRegionalQuotes.html
(for the installation on my computer, at least).

But it does not solve the problem of missing ECN liquidity from IQFeed, it just allows you to precisely see the breakdown of liquidity from exchanges that they do support.


>IB does not report every trade and that is intentional on their part.

Oh really? That is news to me. Does IB document that anywhere?


>IQ does report every trade (or at least tries to) and gives you the market center where the trade executed.

I have set up a lot of consistency checks (e.g. compare the total volume of trades that IQFeed reports versus a sum of all the individual ones that they have sent me), and every day I have megabytes of log file errors generated by problems that I see with IQFeed.

I am not sure what part of this is because the exchanges themselves are really bad in the data that they provide, or whether IQFeed is messing up. I suspect some of both, but am sure that IQFeed is at least partly responsible as I notice lots of other errors (e.g. Q messages that do not follow their own internal format and so are unparseable) which indicate that they do not have high quality software.

Born in NYC. Lived in or near until after college. Moved to LA in 1985. Much better weather except when neighbor's pool is sliding down the hillside. Happy to chat - my email is in my profile. I do make it back to NYC once in a while.

>IB does not report every trade and that is intentional on their part.
IB documents a limit of 50 messages per second, but it seems that sometimes I get more. I recall some discussion of this in their user forum. I believe they prioritize current bid/ask and size over other messages. As you know there is nothing in their feed similar to IQ's "t" (trade) message. For example, let say the last size message was 100. If 5 trades of 100 shares go off at the same price in a row, you might not receive any message except for a total volume print and they just don't match up 1:1 - you can get 1 volume report which is 500 shares higher than the prior, rather than 5 volume report updates. I'm following heavily traded stocks like QQQQ's, INTC, MSFT, etc. If you were trying to create your own streaming ticker you can do this with IQ feed, no problem, but I don't think you could do it with IB feed.

I'm getting about 2 million messages per day from IQ feed. My daily log files are normally 800-1200 MB. I do see some errors when debug logging is set over 0, but all seem to work fine when it is set to 0, however, the message parser provided in the watchlist C++ sample caused me all sorts of problems so I rewrote it.

while(IBmsg.Find(",,") >= 0)
IBmsg.Replace(",,", ", ,");

then I use strtok() to parse the message. I can send you a C++ snippet if that help. I don't do Java, so I'm not sure.

>But it does not solve the problem of missing ECN liquidity from IQFeed, it just allows you to >precisely see the breakdown of liquidity from exchanges that they do support.

I'm confused. Maybe you and or Jay can enlighten me. IB does provide one item I would like to see IB add, market depth, but they limit me to watching 3 tickers. Which ECN's are missing? or is it some aspect of the data provided that you feel is lacking?

What I would really like to see IQ implement is depth of mkt info similar to the book viewer avaliable at http://www.archipelago.com/. While this only shows ARCX and INET books, my analysis of QQQQ's shows about 65-70% of QQQQ's total volume trades on these 2 ECNs (assuming the trade print data from IQ is full and accurate).

>IB does not report every trade and that is intentional on their part.

I just found a good discussion of this here:
http://www.interactivebrokers.com/discus/messages/2/25613.html?1100048704


>IB documents a limit of 50 messages per second, but it seems that sometimes I get more.

Let me know if you know of a solid link discussing this; someone just posted on this topic today:
http://www.interactivebrokers.com/discus/messages/2/27415.html?1109263195


>I do see some errors when debug logging is set over 0, but all seem to work fine when it is set to 0, however, the message parser provided in the watchlist C++ sample caused me all sorts of problems so I rewrote it.

I am not sure what you are refering to about debug = 0 logging in the context of IQFeed. IB does have such a capability with their api, see for example http://chuckcaplan.com/twsapi/index.php/void%20setServerLogLevel%28%29, but that is not what we were talking about.

The errors in my log file are generated by my own program when various consistency tests that I have devised fail. IQ themselves almost never reports an error. (Their server software should be written better so that problems are caught before ever being sent out.)


>>But it does not solve the problem of missing ECN liquidity from IQFeed

>I'm confused. ... Which ECN's are missing?

All of them are missing--look at Jay's first reply to my initial post.


>What I would really like to see IQ implement is depth of mkt info similar to the book viewer avaliable at http://www.archipelago.com/

Yup, that would be nice too.

re: IB 50 message per sec limit - You found some of the info. I think there's more on the Yahoo TWSapi forum. I don't recall exactly where I found it, but I think as you examine the IB data feed closely you'll reach the conclusion that it's a scaled down feed. IQ provides you tick by tick data, while IB has no intention of providing this type of detail. On the flip side, Friday there was some issue preventing IQ (and others) from getting data from CINN (Island/Inet) for over an hour, but IB was showing updated quotes during the period. Since IB is a broker and directs a LOT of orders to this mkt (evidenced by routing report), I suspect they have a different level of service than IQ.

re: Debug level - Look at the IQ API for "IQConnect logging Log Settings for IQConnect ". It's a registry setting. With log level set to 2, I've seen some "corrupted" data, like a last print on Q's of 3742, and it's obvious the decimal is missing. I still need to put some code in my app to check for this, as well as prints that are obvious exchange reporting errors.

re: ECNs - check the IQ API for "Market Center Codes A list of Regional Market Center codes. "

I may be mistaken. If so, someone please let me know.

c = CINN = Island/Inet
p = ARCX = Archipelago
t = NASDAQ which incorporates some of the lesser players like BRUT & BTRADE

If you open a level 2 window with DTNIQ along side the archipelago book viewer you can see them match up.

Jay? You out there today?


I'm a little confused by your comment "We don't carry the ECN feeds on IQFeed."

I thought Island (INET) , ARCX, & BRUT are all ECNs. Can you expand on this a little?

We don't have the direct ECN feeds, but Nasdaqs Level II feed provides BBO from each MMID/ECN. With direct ECN feeds, you would see the full depth from those ECNs instead of their BBO.

---

Jay Froscheiser
DTN Market Access, LLC.

So the regional data for ARCX & CINN is extracted from the Level 2 feed?

Regional quotes are different than level II quotes. The regional quotes are in the level I feed and also provide only the BBO from each of the regional exchanges.

---

Jay Froscheiser
DTN Market Access, LLC.

Here is something more about IB's smart routing that they just sent me in an email today:


Smart Routing Gets Smarter

We are pleased to announce yet another brokerage industry first, IB SmartRouting Auto-RecoverySM which will automatically re-route your order to a new market center should a market center with your order experience a technology malfunction. Unlike other brokers that must wait for an exchange to recover in order to ascertain whether or not your order has been executed, IB will automatically re-submit your order to a functioning exchange and undertake the risk of a double fill on the malfunctioning exchange. Rather than having to experience the uncertainty of a pending order in limbo, you will now know the exact status of your order. Auto-Recovery is yet another reason to choose IB and SmartRouting.