Hi all, I have a problem with IQFeed ticks sometimes being delayed by 4-5 minutes by the time they are sent to my application. I have spoken to customer support and the problem was "fixed" after restarting IQFeed, however the support rep was unable to provide a long-term resolution (I've seen this problem multiple times before). "IQFeed sometimes hangs or gets delayed" is not a great explanation when I have an automated trading system depending on your feed.

In particular I looked at the SPY ticker (i.e. SPDR ETF). The price being received by my application seemed several minutes behind the price in my Interactive Brokers. Then I started IQWatchQuote for SPY and I saw the same delayed price and the Bid Time: and Ask Time: fields were ~ 4 mins behind real time (though ticking constantly). The Delayed: value was empty.

I'm running IQFeed version 5.1.1.3 on linux. Yes, I know that officially you don't support linux but I doubt that's the culprit and if I have to I'll run my system on Windows for a couple of days just to show you that the problem happens there too. I have 2 processes that use the IQFeed tick data:
* a tick recorder that records 1800 stock symbols
* the trading application that subscribes only to SPY

The computer is a desktop with 8 CPU cores and 12 GB of RAM. It's nowhere near running out of CPU or memory.

My theories:
* You IQFeed client runs only on a single thread and by the end of the day it has fallen behind on processing quotes for 1800 stocks by several minutes.
* My client is connecting to a bad server on your side. Seems unlikely otherwise more people would have complained.

My questions:
* How can we fix this?
* Why isn't your client complaining loudly about being behind by several minutes? Instead it fails silently and feeds old tick data to my trading application.

Daniel

---

"I hear and I forget. I see and I remember. I do and I understand."
-- Confucius

One possible way to determine the reason for the delay your seeing in IQ Feed is to utilize the Diagnostic Utility.

You can locate your the Diagnostics one of two ways
(1) Locate your DTN icon in your system tray down by your clock on the lower right hand corner.
(2) C:\Program Files (x86)\DTN\IQFeed

You can right click on the DTN icon you will find Support and Launch Diagnostics. Under the diagnostic utility you are looking for client stats.

Under Client stats there is a column listed "KB Queued". If you are seeing something other than 0.00 in this column that indicates that the your 3rd Party platform is not keeping up with the IQ Feed and the software begins queueing data for your software. This could potentially create the artificial delay in the IQ Feed.

If this does not resolve the data delay you are seeing please contact us at support@iqfeed.net with your login information so that we can check with our developers on the server side.

---

Stephen Shockey | Telvent DTN LLC | Trading Markets | United States |Customer Service Technical Support
Phone: +1800.779.7299
Edited by DTN_Stephen on Dec 10, 2014 at 04:21 PM
Edited by DTN_Stephen on Feb 23, 2015 at 12:57 PM

Thanks. I found the client stats diagnostics. I'll look at it next time I see the problem (hopefully I can reproduce it tomorrow). Not much data flowing back right now...

---

"I hear and I forget. I see and I remember. I do and I understand."
-- Confucius

Hello, I moved this thread over to the Developer Support Forum since this is related to your own app instead of a 3rd party app.

With that said, since the included Watchquote application showed the same delay as your application, it is likely not a problem in the outgoing data queue (which is what the Diagnostics application will display). This means that the data is most likely being queued at the server level waiting to be delivered to your application. You can contact developer support while the issue is occurring via chat from the website and we can confirm this is the case (but we can only see it while the issue is occurring).

Assuming that is the case, the two most probable reasons for the data getting queued on the server side is either your CPU is getting saturated and not able to process the data quick enough or the available bandwidth from your machine to our servers is getting saturated (this may be less than your total bandwidth available).

You are correct that the portion of IQFeed that processes streaming quotes is single threaded so this means that the speed of each core in your CPU is much more important than the number of cores. Since you are watching 1800 symbols and I'm assuming they are fairly active (you used SPY as an example), you will want as fast of processor speed as possible to handle that load. Since your machine is an 8Core (is that 8 physical cores? or is it a quad core with Hyper Threading? or is it an 8 Core with HT meaning 16 parallel execution threads?) machine, if during the problem, you see a single core being saturated or possibly even just the overall cpu showing the same amount of saturation that would equate to a single core being saturated, CPU is likely the issue.

To test the bandwidth option, the best indicator we have available is going to be for you to use speedtest.net to run a bandwidth test to the location in Omaha Nebraska and send us the results. This test site is very close to our server's physical location so it should give fairly accurate results. It should be noted however, that this will only test for failure and not necessarily success because their test will use up to 4 parallel downloads to try and maximize the throughput whereas IQFeed, being single threaded, will only use a single connection.
Edited by DTN_Steve_S on Dec 10, 2014 at 10:33 PM

Thanks Steve.

I ran the network bandwidth test, I get 30+ MBits/sec download speed from Omaha, which just about the same as the same test with the closest server (in Los Angeles). There are no other bandwidth intensive processes running on my machine.

The machine has 8 physical cores running at 2.66 GHz (with HT turned on). Since you mentioned that the quote processing in the IQFeed client is single threaded, I agree this seems like the likely problem. I'll look at CPU usage tomorrow and confirm it's taking up 1 CPU core most of the time.
Assuming that this is the actual problem, I can think of 2 possible solutions:
* Run the IQFeed client on multiple machines (possibly virtual machines) so that it will use more than 1 thread. Will your server allow multiple connections?
* Run the client on faster hardware. I have a 3.4 GHz box sitting around ...

I don't suppose using multiple threads is on DTN's todo list? :P I would still prefer if the IQFeed client complained (with popups or otherwise) that it's falling behind by that much, I would have found the problem quite some time ago.

---

"I hear and I forget. I see and I remember. I do and I understand."
-- Confucius

The problem didn't happen today but I'll keep a daily watch. The IQConnect.exe process generally takes up 30% of 1 CPU core when I start up my processes at night (markets aren't really trading then), which I'm guessing is due to it doing lots of polling in a busy loop. During the actual trading day it took up 60-80% of 1 CPU core.

---

"I hear and I forget. I see and I remember. I do and I understand."
-- Confucius

how are you running in on Linux? using Wine?

Hello Andrew,

Yes, Wine does seem to work without issue whenever we have tried it and we have several users who regularly run it there. Though I can't say for sure if that is what illuminatus7 is using.

Tim

Hi guys, yes I'm using Wine. I've recently moved to new hardware and I haven't seen the issue come up again. I still have the objection that IQFeed should have complained loudly when it's falling behind by as much as 5 mins (either via popup windows on the desktop or via error messages on the TCP API).

---

"I hear and I forget. I see and I remember. I do and I understand."
-- Confucius