||Sep 4, 2004 11:37 AM
||May 20, 2009 08:52 AM
||Oct 22, 2011 12:31 PM
nsolot has contributed to 273 posts out of 18567 total posts
(1.47%) in 5,067 days (0.05 posts per day).
20 Most recent posts:
Sure, although this is a bit fuzzy as the conversations and forum threads are probably 3 years ago.
I'm been tinkering with an ATS, which auto trades using the feeds from Interactive Brokers and IQFeed, and would place trades through Interactive Brokers (IB). IB is located in Greenwich CT, however it appears the servers are NYC.
I think a number of us concluded that the closer one is to IQserver the better. FWIW, I only use the Nebraska server farm as Level 3 has proven (over a span of several years) to be an unreliable ISP. I'm based in LA, however other users who were located outside the US had latency issues magnitudes worse than what I was seeing. One guy, based in Australia, ended up colocating in Boston and a number of his data feed issues were resolved.
Would be ideal to be able to colocate in same facility with IQ.
I'm still interested in a colo facility between the Nebraska server farm and the east coast. I tried a couple and was not happy with either of them.
I was hoping at one time that DTN would partner with a colo provider in some manner to provide some kind of QOS between DTN farm and colo, but I think there was only one other customer who also expressed interest.
I think we covered this topic a long while back, with Tim, before Steve was running this show.
Level 1 quotes consoldate from various "regional" exchanges. Sometimes a glitch at a regional causes quotes to go stale, and as a result the level 1 qoute gets crossed. I had suggested at one time that DTN add some logic to detect "stale" quotes and discard them from being used in the compostite Level 1 quote, but my sugegstion was not adopted.
The solution I utilize is to work with regional quotes for the active exhange (or two or three) for the issues you follow rather than the Level 1 quotes. The regional exchange quotes rarely get crossed and if they do, they usually resolve instantly.
Hope this helps.
This evening IB software offers up the automatic rollover to the Dec contracts.
Well, I'm a wee bit confused...
Usually, my Interactive Brokers flashes a rollover option, but didn't do that today.
When I look at ES & NQ, today's volume is still slightly higher for the September contracts than December.
It it possible something has changed with regard to roll over of most active contracts? Maybe tomorrow is the day?
I'm getting R messages like this one:
In the market center doc page, CBOE is listed as an option regional quotes market center, but not for equities.
Not sure what's happened...
Had similar problem with 156 farm. Changed my hosts file, and got logged on to 148 farm.
Same thing today. 156.224 & 156.225
Seems the new server platform delivers one set of regional quotes and then stops.
For example, I get
and no more regionals afterwards.
I'm not saying E-CME platform is 100% bulletproof...
just better (more reliable) than E-CBOT.
In either case, you should see almost contant size updates on bid/ask or both. If things go quiet for more than a few seconds, best to assume something has gone wrong.
In my observation E-CBOT has always been less reliable than its counterpart E-CME where the ES & NQ e-mini futures are traded. I recall the two exchanges merged earlier this year, however no plans annouced to move YM to the more stable platform.
For this reason, I've turned my attention away from YM and on the other two. Also more volume and deeper market on ES & NQ.
based on my observations, the issue isn't so much the number of symbols you're watching, but the aggregate number of update messages you recieve in any given time slice.
Active issues (MSFT, INTC, QQQQ, etc.) will send lots of messages.
Low volume stocks will usually send fewer, however keep in mind that significant news can cause a normally low volume stock to spike on any given day, and this can have the effect of slowing down (or other problems) for the entire set of tickers you're watching.
Let me give you an example:
Today at approx 9:34 AM EST IQ reported in the real time feed (not T&S) ES prints at 1475.25 (volume prints were 215235 , 215236, 215238 if you want to pull the T&S data for same).
The Q messages showed bid/ask of 1475.50x1475.75 so these trade prints were under the bid.
The same trades were reported via Interactive brokers feed, however at the same time they showed the bid/ask to be 1475.25x1475.50, so those trade prints (according to them) hit the bid.
While not impossible, I would be highly suspect of any trade prints on ES, NQ, and YM outside the bid/ask as these are fully electronic exchanges with only a single market. Trades on stocks outside the bid ask do not surprise me at all for a number of reasons.
I have previously reported this for ES, NQ, and YM, but since it doesn't happen often it never got a high priority.
It appears that bid ask updates don't always make it into the streaming feed (i.e. Q messages) and as such trade prints appear outside the bid ask. This is confirmed by my second feed which shows a bid/ask change that IQ doesn't, at least in the Q messages.
This is again confirmed by pulling T&S data, and isolating the particular trade by volume print. T&S will report a different and usually correct bid/ask than what was rec'd in the Q message.
I'm using the data feed provided by Interactive Brokers as a second feed. Theirs has some limitations compared to IQ in that it will only allow a limited number of tickers (I think 100, but they keep changing it), and they make no attempt to provide tick by tick data. Theirs is a "snapshot" service which provides bid/ask/last/volume a couple times a second or so for each ticker being watched. There's some downside to this, however their feed doesn't seem to experience delays as they purposely omit data.
Also, they do provide market depth for many securities.
My app is watching the e-minis (ES, NQ, YM), the underlying ETFs, and a handful of individual stoocks from both feeds, and so long as they match I assume data is reliable and accurate.
Another item I just noticed. Nasdaq stocks are getting 4 digits of precision while Listed stocks only two. Is that intentional? Couldn't the trailing zeros be trimmed similar to absolute price change (field 5)?
Additional fields which are inefficient because either the info is duplicative, should be client side calculation, should only be sent once, extended trade info being sent during regular hours, etc.
17 Range float Trading range for the current day (high - low)
19 Open Interest integer IEOptions, Futures, FutureOptions, and SSFutures only
20 Open float The opening price of the day.
21 Close float The closing price of the day.
22 Spread float The difference between Bid and Ask prices
24 Settle float Futures, FutureOptions, and SSFutures only - Is this different than Field 21?
33 Extended trading last - Suppress during normal market hours
34 Expiration date MM/DD/YYYY Deprecated, use field 53 in the Fundamental message
37 Extended trading change - Suppress during normal market hours
38 Extended trading difference - Suppress during normal market hours
39 Price-Earnings Ratio
40 % Off Average Volume
41 Bid Change float Change in Bid since last offer
42 Ask Change float Change in Ask since last offer
43 Change From Open float Change in last since open
46 Market Capitalization float Real-time calculated market cap
45 Volatility float Real-time calculated volatility (Today's High - Today's Low) / Last
49 Days to Expiration string Number of days to contract expiration
50 PVol Integer Previous Day's Volume
51 Regions String The available regions for regional data
62 Settlement Date MM/DD/YYYY The date that the Settle (field 24) is valid for
And Field 46, Market cap, I retreive the following data points from a recent log file (August 9):
First, why is this even a float?
Second, the data is simply wrong. WMT cap is 191 Billion, not Million. XOM is $464 Billion. These should be sent as integer types, and Client app should have to multiple by 1 million, so data sent would be 191797 and 466440. Personally, I don't use this field and wish it could be supressed altogether, but I can see how others might want it.
Sorry Jay, but I have to post a voice of dissent.
"The only solution is more efficient client software and/or faster hardware" is just plain wrong in that it addresses the SYMPTOM, not the PROBLEM.
The problem is that IQConnect and Client apps canot keep up with market data during heavy trading periods due to fundamental inefficiencies related to the design of Q messages. For example:
1. Field 6, percent change should be truncated to 4 decimals. I'd actually take the position this never should have been included in a Q message, but rather it should be a client side calculation.
2. Fields 9 & 10 (daily hi & low) should only be sent when a change occurs, otherwise a single character should be used indicating the prior value should be used. The same concept should be used for many other fields.
3. Field 31, last trade date. Does this need to be sent with every Q message, especially if it traded in the current day?
4. Field 43, change from open. You provide open & last. Should be client side calculation.
5. Field 51 - Regions is a very long string which need not be sent every Q message. Should probably be in the F message anyway. If iqconnect could only send it once for each ticker that would be a significant savings.
I'm sure there are many other items which would make a much more efficient Q message and reduce CPU needed by both apps.
Then there's the matter of duplicate messages and blocks of messages which contain very little useful info. I've provided samples of these to Steve. Examples include blocks of consecutive Q messages which only report bid/ask size changes, which when reported in a series really provide no useful info, but heavily contribute to the problem of loading.
Finally, and Steve & I don't exaclty see eye to eye on this, there are many instances of consecutive trade reports, which I believe are "broken" fills. For example, an order to buy 1,000 shares gets filled in lots of 100,200, 300, etc. IQ clients often receive multiple Q messages which I feel can and should be consolidated into a single Q message provided the last, bid and ask price is the same for all consecutive trade messages.
The SOLUTION is a more efficient message structure combined with eliminating messages not needed by client apps.
Let's hear what others have to say. What's more important? True tick by tick data, or a feed that can stay up to speed during fast market conditions?
Can I get instructions on how to connect to 156 farm only? 148 servers have been a pain in my A$$ due to flakey ISP between me & you.