||Sep 17, 2004 01:48 AM
||Aug 11, 2011 04:55 AM
||Aug 11, 2011 04:55 AM
dhakme has contributed to 150 posts out of 20453 total posts
(0.73%) in 6,338 days (0.02 posts per day).
20 Most recent posts:
If it makes you feel any better we have been experiencing larger feed delays recently (again using timestamps as a reference). Normally these occur during bursts of market activity such as open/close and after major news announcements. Of course, with recent events these have occurred more often. Tick counts have more than doubled over the last few days for the same watchlists on both the NYSE and NASDAQ.
From your trace route it looks like you're in NZ. We originally (until 4-5 years ago) ran our servers from Australia and had a lot of trouble with the feed during busy periods due to the high network latency (250ms ping times). At the time this resulted in data corruption but may manifest itself differently with the later versions of IQFeed. This was resolved when we relocated to the East coast USA.
Also, as IQFeed is single threaded, look at the process CPU usage to see if it's maxing out a single core (25% on a quad core, 50% on a dual core, etc) rather than overall CPU usage.
I've looked at the pings & trace routes and I don't believe they reveal anything but here they are. I also ran the pings for around 15 minutes and had no anomalies.
/home/trader-> ping 188.8.131.52
PING 184.108.40.206 (220.127.116.11) 56(84) bytes of data.
64 bytes from 18.104.22.168: icmp_seq=1 ttl=54 time=45.4 ms
64 bytes from 22.214.171.124: icmp_seq=2 ttl=54 time=45.7 ms
64 bytes from 126.96.36.199: icmp_seq=3 ttl=54 time=46.0 ms
64 bytes from 188.8.131.52: icmp_seq=4 ttl=54 time=45.5 ms
64 bytes from 184.108.40.206: icmp_seq=5 ttl=54 time=45.6 ms
64 bytes from 220.127.116.11: icmp_seq=6 ttl=54 time=45.3 ms
--- 18.104.22.168 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5004ms
rtt min/avg/max/mdev = 45.302/45.626/46.084/0.365 ms
/home/trader-> traceroute 22.214.171.124
traceroute to 126.96.36.199 (188.8.131.52), 30 hops max, 40 byte packets
1 184.108.40.206 (220.127.116.11) 0.845 ms 1.374 ms 1.343 ms
2 18.104.22.168 (22.214.171.124) 0.718 ms 0.710 ms 0.688 ms
3 126.96.36.199 (188.8.131.52) 0.663 ms 0.873 ms 0.851 ms
4 nyrkbprj02-ae1.303.rd.ny.cox.net (184.108.40.206) 0.815 ms 0.792 ms 1.009 ms
5 mtc1dsrj01-ae2.0.rd.om.cox.net (220.127.116.11) 43.692 ms mtc1dsrj02-ae2.0.rd.om.cox.net (18.104.22.168) 43.654 ms 43.653 ms
6 22.214.171.124 (126.96.36.199) 44.304 ms 188.8.131.52 (184.108.40.206) 44.188 ms 220.127.116.11 (18.104.22.168) 44.289 ms
7 22.214.171.124 (126.96.36.199) 44.895 ms 44.969 ms 44.805 ms
8 (188.8.131.52) 45.064 ms 45.021 ms 45.001 ms
9 (184.108.40.206) 80.018 ms 79.985 ms 79.933 ms
10 220.127.116.11 (18.104.22.168) 45.117 ms 45.398 ms 45.061 ms
11 (22.214.171.124) 46.523 ms 46.510 ms 46.484 ms
12 126.96.36.199 (188.8.131.52) 45.275 ms !X 45.434 ms !X 45.305 ms !X
/home/trader-> ping 184.108.40.206
PING 220.127.116.11 (18.104.22.168) 56(84) bytes of data.
64 bytes from 22.214.171.124: icmp_seq=1 ttl=52 time=43.9 ms
64 bytes from 126.96.36.199: icmp_seq=2 ttl=52 time=45.3 ms
64 bytes from 188.8.131.52: icmp_seq=3 ttl=52 time=42.4 ms
64 bytes from 184.108.40.206: icmp_seq=4 ttl=52 time=42.5 ms
64 bytes from 220.127.116.11: icmp_seq=5 ttl=52 time=42.4 ms
64 bytes from 18.104.22.168: icmp_seq=6 ttl=52 time=43.6 ms
64 bytes from 22.214.171.124: icmp_seq=7 ttl=52 time=42.4 ms
--- 126.96.36.199 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6006ms
rtt min/avg/max/mdev = 42.452/43.264/45.336/1.053 ms
/home/trader-> traceroute 188.8.131.52
traceroute to 184.108.40.206 (220.127.116.11), 30 hops max, 40 byte packets
1 18.104.22.168 (22.214.171.124) 0.975 ms 0.939 ms 1.115 ms
2 126.96.36.199 (188.8.131.52) 0.553 ms 0.858 ms 0.846 ms
3 184.108.40.206 (220.127.116.11) 0.806 ms 0.794 ms 0.772 ms
4 vb1010.rar3.nyc-ny.us.xo.net (18.104.22.168) 3.621 ms 2.260 ms 2.882 ms
5 * * *
6 22.214.171.124.ptr.us.xo.net (126.96.36.199) 0.600 ms 0.794 ms 0.871 ms
7 omh-core-02.inet.qwest.net (188.8.131.52) 41.770 ms 41.800 ms 41.776 ms
8 omh-edge-09.inet.qwest.net (184.108.40.206) 41.398 ms 51.287 ms 51.279 ms
9 220.127.116.11 (18.104.22.168) 42.890 ms 43.224 ms 43.202 ms
10 22.214.171.124 (126.96.36.199) 42.816 ms 43.141 ms 43.124 ms
11 solar0.interquote.com (188.8.131.52) 42.411 ms !X 43.386 ms !X 42.559 ms !X
While I appreciate that upgrading the client will eliminate it as the source of the problem, doing so would require some development work at our end which we cannot do in the short term.
However, I should point out that the problem did not exist prior to May 23 and it came on suddenly. Also, as you pointed out, we don't have a problem with the .156 farm which suggests a server side issue. The fact that Craig is also seeing delays supports this.
As a workaround, we will reset our alert threshold to 5 sec which will cause our software to reconnect and hopefully we will reconnect to the .156 farm (from memory we can force a connection to the .156 farm but we won't do that for now).
We do the same thing as Craig. Compare local clock with T message and local clock is regularly synced to a time server.
Since May 23 we have noticed that our data feeds from the IQFeed 66.112.148.x data servers are experiencing data delays of up to 10 seconds throughout the trading day. We use the IQFeed (server side) generated timestamps to determine delay by comparing them with the actual time. This has proven to be a reliable method and the difference is usually less than 500ms throughout the day except during activity spikes.
Unfortunately we only generate alerts when delays exceed 10 seconds so we were a little slow in noticing the problem. Since the delays weren't experienced every day we trawled through our logs and identified the 66.112.148.x farm as the culprit (the 66.112.156.x farm is behaving normally). Additionally we have separate IQ accounts used for NASDAQ and NYSE feeds and the NYSE feed is significantly worse...probably reflecting the larger rate of updates.
Was some kind of change implemented on the weekend of 21/22 May?
I have attached some charts (delay is the pink, measured in ms)
Yesterday (Monday) at around 15:46 EST two servers connected to IQFeed lost their connections. Neither server was able to successfully reconnect. The servers would login to IQFeed but were unable to watch any symbols.
Were there any known problems around that time?
We are running older versions of the IQFeed client as we have not made the necessary changes to support later clients. Could this be a cause?
That's very helpful.
Thank you for your response.
I did try to use IQFeed with WINE some time ago when IQ32.dll was required to launch and had a few problems but didn't persist.
I'll give it another go with the later IQFeed and hopefully will have more luck.
On various threads over the years people have requested API support under Linux or alternatively a Java client that would provide cross-platform support.
As neither of these appears to be forthcoming, a number of people have had success in running the existing client under WINE on linux.
Would anyone out there care to share their experience and possibly provide some pointers and hints?
We have very good linux knowledge and reasonable Windows experience but none with WINE.
Does DTN have any experience with IQFeed under WINE? If so, would you consider an unofficial (ie unsupported) HowTo document?
We are aware of emails sent out by Telvent DTN with the following message:
"We are going to be upgrading our servers in the near future & the version of IQ you are using may not be compatible with the new server sets, and will no longer work."
The emails don't give any details of the impending Server upgrades nor which versions of IQFeed will be compatible. I did reply to one of the emails but got no response. It would be very helpful if you could answer the following:
1) When are the Server upgrades planned?
2) Which versions of IQFeed will be compatible with the new servers?
I believe the source of the problem has been identified and it is on our side.
Please consider this problem closed.
We have a trading server that is dedicated to the Nasdaq exchange.
Over the last few trading sessions our software has detected feed delays for Nasdaq symbols. The detection method compares the IQFeed generated “T” record timestamp with the actual time. Normally the largest difference is a few seconds during the peak open/close periods.
Over the last two sessions our delays have been around 30+ seconds at open/close as well as during the day. We've investigated the following as possible causes:
a) PC resources: CPU & memory usage are reasonably low. No single
process/thread is being constrained
b) Network: We are in a Tier One data center with 100Mbps connections. A second server that uses IQFeed exclusively for NYSE symbols doesn't have the same feed delay
c) IQFeed servers: Both our Nasdaq and NYSE dedicated machines are often connected to the same IQFeed server but the delays are only on Nasdaq.
d) Software anomalies: Both of our servers are running identical software (one is a clone of the other)
Given the above, the only other cause I can see is there is a delay within the DTN Ticker plant or the Nasdaq feed. Can anyone at IQFeed shed any light on this? If there is a problem, when will it be rectified? Our day trading algorithms are adversely affected by any delay beyond 5 seconds.
Can't make any comments regarding VS but I can make a few other suggestions.
When we set up our machines, no x86 CPUs supported virtualization. Our development machines now run Linux as a host and a virtual machine with XP as a guest OS. You can run the windows stuff (DTN) in the XP guest and the rest of your application can run under Linux. Communication is via a simple socket connection.
I'm not trading on Friday so I'll get in touch (via chat) sometime in the morning after market open.
Sorry Steve, I should have explained the chart.
I was actually initially connected to the 156 farm (very left of the chart)and everything was going okay. The data then became very choppy, sometimes falling to a very low rate and then spiking upwards (middle of the chart). I then forced a new connection twice (it chose another 156 server initially) and connected to a 148 server. The data became steady again (right of chart).
After I posted that chart we had some minor trouble with the 148 farm where data would stop completely for a few seconds. Unfortunately this triggered a reconnect to the 156 farm which failed completely! (Hundreds of watches failed, data feed almost non-existent). This happened several times throughout the day and the 156 farm failed in this way every time from then on.
Once again, network traces showed no problems with the network to either farm and using a different ISP didn't help either.
Sorry Steve, but I'm still seeing the same behaviour.
I have switched to using the 148 farm exclusively so I can't confirm if my observed performance issues are still there. I will switch back to using both farms for Tuesday and report back.
I don't think I'm going to be of much help diagnosing this problem. I will send you our two watch lists via email.
It happens to both of our machines and we monitor the data rate on both. If the number of updates gets too low then we automatically reconnect to a new data server. Our software ensures that we alternate data farms when we reconnect For the example I posted our server was watching 1300 NYSE symbols and was connected to the 184.108.40.206 server. The data rate dropped to 11 updates/sec and we reconnected to the 220.127.116.11 server.
It doesn't appear to be symbol specific. We seem to get updates for all of our symbols (I say 'seem' because I am only sure about the symbols we've purchased which is typically 20-30 per day).
Attachment didn't seem to work so here's a link.