Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »IQFeed Developer Support »Timestamp messages
Author Topic: Timestamp messages (4 messages, Page 1 of 1)

gelo2000
-Interested User-
Posts: 15
Joined: Jan 28, 2009


Posted: Feb 20, 2009 12:19 PM          Msg. 1 of 4
Hello,

I have noticed that sometimes IQFeed misses minutely timestamp messages (see end of file). Is it a bug or am I missing something?

Thanks
-----------
S,KEY,130
S,SERVER CONNECTED
S,IP,66.112.156.181 60004,66.112.156.213 60002,66.112.156.210 60009,66.112.156.214 60001,66.112.156.213 60005,66.112.156.213 60012,66.112.156.200 60003,66.112.156.214 60015,66.112.156.210 60050,66.112.156.214 60014,66.112.156.212 60016,66.112.156.211 60018
S,CUST,real_time,66.112.156.114,60002,u7XqVVVv,4.5.0.3,0,INDEX DTNNEWS SCROLLNEWS RT_TRADER COMTEX PRIMEZONE INTERNET_WIRE NASDAQ,,1800,QT_API,,
S,KEYOK
T,20090220 13:05:00
T,20090220 13:06:00
T,20090220 13:07:00
T,20090220 13:08:00
T,20090220 13:09:00
#### Misssing timestamp here !!! #####
T,20090220 13:11:00
T,20090220 13:12:00
T,20090220 13:13:00

gelo2000
-Interested User-
Posts: 15
Joined: Jan 28, 2009


Posted: Feb 20, 2009 01:30 PM          Msg. 2 of 4
Apparently it's happening when IQFeed get's overloaded:
* I am listening for 1800 symbols on a fairly new Mac (2.5 GHZ, 2 GB RAM), running IQFeed with wine
* At first it works well, but in a of couple minutes the messages start to lag
* Also only then cpu usage of IQFeed hits 100%
* It might be a feature of IQFeed 4.5 as I have made some (superficial) stress tests with IQFeed 4.4 earlier and there weren't such problems.

Strange.

DTN_Steve_S
-DTN Guru-
Posts: 2096
Joined: Nov 21, 2005


Posted: Feb 20, 2009 03:52 PM          Msg. 3 of 4
Hello, what you seem to be experiencing happens by design (and is not something new).

When the server gets an error on the client socket, it will drop the data rather than queue it for a later send. This is designed this way so that no single client can adversly affect other clients connected to the server.

On the other hand, IQConnect is designed so that it will queue data that gets received from the server indefinately until it gets delivered to the client app or until the connection is dropped.

What this means is that for the most part, the data processing in your app and the operating envionment for IQConnect are responsible for problems such as these. If your app is slow reading data off the socket, then IQConnect is going to be queing data internally. The more time IQConnect has to spend managing it's internal data queues, the less time it has available to read from the server socket and eventually the socket buffer on the serverside will fill up causing errors on the socket and the server to drop data. This is why it takes a few minutes for the problem to occur.

You mentioned that you didn't experience these problems with 4.4.0.3. My initial reaction would be that you simply didn't notice the problem with that version. Also keep in mind that these types of problems are also greatly dependant upon the market conditions so it is possible that your current issue wouldn't have existed during your testing of 4.4.0.3 becuase there was less data comming through the system. Our internal testing indicated that 4.5.0.3 is significantly more efficient at processing the streaming quotes feed than 4.4.0.3. However, this has the side effect of your app getting delivered larger "bursts" of messages and the potential to end up queuing data internally is greater since more data is actually getting delivered to your app in a shorter amount of time.

Of course with your setup, you also have the added (untested) overhead of running IQFeed in wine. I am not sure how much overhead this adds to the system but it should be significant in relation to IQFeed.

Hopefully this explanation gives you a better idea of what is going on. As I mentioned above, the only real control you have as a developer is increasing the efficiency of your data processing within your app which will reduce the load on the IQConnect client running on your computer.

Other than that, it is possible (although not probable) that you are doing something with the feed we hadn't anticipated and simply changing some methodology might help as well. Is there anything unusual about how you are watching these symbols (do you use fieldsets or do you make multiple connections to IQconnect to do so)?

gelo2000
-Interested User-
Posts: 15
Joined: Jan 28, 2009


Posted: Feb 26, 2009 09:49 AM          Msg. 4 of 4
Hello,

I am running IQFeed on a more powerful machine now and there is no problem any more.

Thank You!
 

 

Time: Wed September 18, 2024 6:25 PM CFBB v1.2.0 13 ms.
© AderSoftware 2002-2003