Joined: |
May 20, 2004 11:30 AM |
Last Post: |
Dec 6, 2023 04:58 AM |
Last Visit: |
Dec 6, 2023 05:00 AM |
Website: |
|
Location: |
|
Occupation: |
|
Interests: |
|
Avatar: |
|
|
AIM: |
|
ICQ: |
|
MSN IM: |
|
Yahoo IM: |
|
|
matthewklein has contributed to 42 posts out of 21251 total posts
(0.20%) in 7,509 days (0.01 posts per day).
20 Most recent posts:
The historical data of the HUT symbol is not adjusted according to the last split.
Matthew Klein
GRBK appears to have a wrong name.
You are showing: Green Brick Energy it should be: Green Bridge Partners.
Perhaps you should also check GRBK-A or GRBK-PA which I see on Yahoo.
Thanks.
Matthew Klein
Missing dividend problems continue. For example (and this is just one; there apparently are many) is the latest dividend for symbol TLT. July dividends have been declared and paid, and still there is no update in the fundamental data.
Dividend data was working great for years; now many many dividends are just not being recorded. Is there any plan to resolve this?
Matthew Klein
Are dividend data still being updated within the fundamental data for ETFs?
Just one example (but there are many):
Symbol = DIA The last dividend pay date shown by IQFeed fundamental data is 5/16/2011, but two monthly dividends have come and gone since then, with no dividend information being updated in the fundamental data. Same applies for SPY (one dividend missing) and other ETFs as well.
Has something changed in the way dividend data is being returned?
Matthew Klein
Any news? Any confirmation that it is indeed a system-wide problem and not just me? Any estimate of how long it will take to fix? Markets are now open for electronic futures...
Matthew Klein
All symbols now returning NOT FOUND. I have two accounts, running on two machines, and both have this problem. Guys - what's going on? Why am I the only one reporting this, and why do I need to alert the company via the developer forums? Is anyone home?
Matthew Klein
Now it works fine, thanks.
Matthew Klein
I have been unable to log in to my iqfeed account, either from my home or office computer. Continuously receive: Failure Retrieving Login Information.
Is this a system-wide outage? Am I the only one experiencing it? Does anyone at IQFeed know about this?
Matthew Klein
10:49 ET - And now it's back.
I didn't change anything? Did DTN? (Honest?)
MK
Matthew Klein
10:42 PM Eastern U.S. - Unable to connect to ip list server, repeatedly.
My ISP/colo facility assures me no changes have been made to the firewall, and all ports that were open this morning remain open.
Is this a system wide problem?
If not, how about some clarity on which specific ports / ip addresses I should try to telnet into to test where the initial connection process is failing.
Is anyone but me experiencing these issues?
Matthew
Matthew Klein
My software is able to connect to retrieve historical data, but cannot connect to quote servers:
SERVER STATUS = DISCONNECTED.
I wanted to post here to alert someone at IQFeed.
Is this a planned outage? If so, it would be really great if you could announce it somewhere so that your customers don't spend a couple hours diagnosing the problem, pulling their hair out, etc.
If this is an unplanned outage, can you please investigate? I know it's Easter, but instruments start trading tonight in the overnight sessions, and I want to make sure everything will work.
Thanks
Matthew Klein
On Friday nights, certain Forex pairs post bizarre bid/ask spreads that ruin my attempts at gathering and processing last-trade-prices for forex pairs. It's not simply a case like what we witness when Nasdaq marketmakers post extremely high and low bid and asks when they stop making markets. In the case of these forex pairs, the bizarre prices actually worm their way into your historical data.
Here's an example. If you request a quote for USD/HKD on Friday night (or Saturday morning) you learn that the Hong Kong Dollar last traded at 3.8775.
Last=3.8775 Bid=3.8775 Ask=3.8785
Unfortunately, that is not true. In fact the HKD traded at well over 7 all day (and all decade).
So let's say your an IQFeed Customer and you want to suck it up -- you don't want to complain about these quotes on the IQFeed forums -- you decide to be a Man about it, and figure out a way to code around this little problem. So you decide what you'll do on Friday nights is call for some historical data, and use the closing price of the last trading day as your last price. Here's what you get using daily history bars:
12/20/2005,7.7522,7.7528,7.7516,7.7526,0 12/21/2005,7.7527,7.7527,7.7518,7.7521,0 12/22/2005,7.7522,7.7527,7.7519,7.7524,0 12/23/2005,7.7525,7.7537,3.8775,3.8775,0
You see that according to IQFeed historical data, the Hong Kong Dollar dropped precipitously in one day.
Now, a similar story in minute bars:
12/23/2005 3:09:00 PM,7.7531,7.7531,7.7531,7.7531,0 12/23/2005 3:21:00 PM,7.7533,7.7533,7.7533,7.7533,0 12/23/2005 3:43:00 PM,7.7531,7.7531,7.7531,7.7531,0 12/23/2005 4:03:00 PM,7.7529,7.7531,7.7529,7.7531,0 12/23/2005 4:15:00 PM,7.7529,7.7531,7.7529,7.7531,0 12/23/2005 4:16:00 PM,7.7529,7.7531,7.7529,7.7531,0 12/23/2005 4:52:00 PM,7.7529,7.7529,7.7529,7.7529,0 12/23/2005 5:01:00 PM,7.7534,7.7534,7.7534,7.7534,0 12/23/2005 5:06:00 PM,3.8775,3.8775,3.8775,3.8775,0
The same strange effects seem to occur every week with the Mexican Peso and the Hong Kong Dollar. Weekend prices are just plain wrong, and they seem to work their way into the historical data, too. In other words, one can't rely on the IQFeed data feed to tell you what the last traded price of these currencies was, without implementing additional logic.
I understand that you are simply re-packaging feeds from forex houses. But I would ask only that you lob a request to your vendors (since you presumably have more clout than the individual retail customer like me) that they clean up the way they handle quotes over the weekends. At a minimum, DTN should not allow bad quotes into its own historical database.
Matthew
Matthew Klein
Natalie:
I'm adding some more extensive logging to my app, so I will be able to send you something that is intelligable to you, and helpful. As soon as I have done this (and as soon as the problem occurs again), I will email you the log. (Please let me know the email address where I should send it.)
To answer your question, though, I'm watching exactly 5 symbols when the problem occurs. However, it *seems* (need to confirm this) to happen when those five symbols are very heavily traded, during peak hours, and thus I get a very fast stream of update messages (so, the watch list might include: @YMZ5, @ESZ5, @ER2Z5, etc. during peak trading hours). What seems to happen is that my wSYMBOL requests are ignored by IQFeed when this fast and furious stream of update messages is being shoved into the socket.
Again, I'll try to capture this in a useful log for you. Hopefully I'll have something for you in the next day or two.
Matthew
Matthew Klein
Natalie:
My app will also request recent minute-bar history for symbols that are "of interest." No other IQFeed functionality is used -- no news, no optionchains, etc.
Regarding the history requests: I will sometimes get a failure when requesting history, but I believe this is caused by the too-many-history-requests bug, which is different than the problem I am reporting.
To repeat: the symptom of the problem I am reporting is that new "watch" requests are ignored, and the requested symbol never gets added to the message stream.
Matthew
Matthew Klein
I beginning to suspect perhaps that the problem is caused by the unusual way I use IQFeed.
First, some background. My own app is used for internal trading and trade-tracking purposes, so it doesn't need to keep a lot of symbols on the watch list at any one time. Another way to say this is that I have no need to do a lot of "scanning" of the stock universe, looking for various conditions among a large universe of symbols. Instead, I simply want to be able to pull up an up-to-date quote for a stock/future I am interested in trading. When I have looked at the quote and fundamental data, my app is generally "done" with the symbol, and doesn't need it any longer.
I tell you this to explain why I use IQFeed the way I do. I have read elsewhere on this forum that IQFeed frowns upon users doing a lot of "watching" and then "unwatching" of symbols, since each new request requires more bandwidth than a simple update message.
So when I first designed my app, I tried to be a good software citizen. If I needed a quote, I would start to watch it, pull up the quote, then process it. But I would leave the symbol in my watchlist and so continue to receive update messages for it.
But I noticed some distrubing things. First, the more quotes I had in my watchlist, the longer the latency of getting a new quote for a brand-new symbol. What's more, when I studied the IQFeed Connection Manager, I saw that the Internet Bandwidth used by my app grew quite substantially as the number of symbols in the watchlist grew. (I suppose this is obvious.) There reached a magic point (60, 70, 80 symbols, I think) where the latency for requesting a new quote on a new symbol grew too oppressive ... I suppose because my bandwidth / DTN's CPU cycles were occupied by delivering update messages for symbols I didn't really need.
So I began to experiment with an alternative software design. I would keep the number of symbols in my watch list quite low (say, 5). Thus I would ask for 5 symbols, but when the sixth was needed, I would unwatch the first one. Since I don't need to keep quotes in memory, and since I need symbols only periodically and don't need to watch them over time, this was fine with me. In fact, I believe this set-up was better from DTN's point of view, since my Internet Bandwidth usage was kept quite low (.75 K/sec during the trading day, even during periods where symbols get updated frequently).
The result of this design is: (1) I don't use a lot of DTN bandwidth (so don't get mad at me), and (2) I don't suffer latency when I need a new quote on a new symbol. In general, I can ask for a quote on a symbol, by watching it, and have the update and fundamental messages arrive almost instantly.
However, the implication of this design is that I do indeed do a lot of watching and unwatching of symbols. This might be the reason (I hypothesize) that I am one of the few who has reported a problem in which an attempt to watch a new symbol finally just fails, even though the IQFeed core is still running fine.
So, if you want to emulate my software, I would suggest you simply write a small app which will keep 5 random symbols in a watchlist, and -- when you want to watch a sixth, will unwatch symbol number 1 first. Just have the app ask for a new sixth symbol repeatedly. After some random amount of time (sometimes hours, alas!) the next attempt to watch a symbol is simply ignored, even if the watch request is repeated.
Matthew
Matthew Klein
Jay:
OK. I did some more investigation and have the following to report.
I set an alarm to sound when the socket problem occurs. (That is, when IQFeed will not respond to any more 'watch' requests, but is still delivering update messages for symbols which have already been watched on already-open socket).
When I heard the alarm this morning, I followed Jay's instructions. I telneted into port 5009, and then manually issued 'watch' commands. Lo and behold, the new socket worked fine. New symbols could be watched an unwatched.
Jay and Natalie: any ideas what this means? Any other ways I can help you investigate this?
Finally, I wonder: could I possibly be the *only* person to have this problem? If so, does this indicate that I am a lousy programmer, and I am doing something wrong; or is it merely an indication that I am one of only a few who uses the TCP/IP interface?
Matthew
Matthew Klein
Hmm. For what it is worth, I actually do also make numerous history requests, but for minute data, not daily data. I remember reading in these forums that the problem Natalie has mentioned relates to daily history requests, right?
Let me ask this: is it possible these are both the same problem? Is one of the symptoms of the "too many history requests" problem that IQFeed stops listening to socket 'watch' requests, but it still continues to deliver updated quotes via the socket for items already in the watch list?
I would consider this great news, because it would mean the problem I am reporting is a problem that has already been reported, and has a known (or soon-to-be-known) cause, and is something which IQFeed is already presumably working on.
Matthew
Matthew Klein
I have been hesitant to report the following problem for quite some time, since I assumed it was the fault of my own programming. But in the past few weeks, I have added copious logging to my own application, have attempted to debug it scrupulously, and now I am coming to the conclusion that perhaps the problem lies with IQFeed, not my own app.
A description: my app uses the TCP/IP (sockets) interface to IQFeed. The app runs great for some period of time. Then, at some interval which may be a few hours, or may be ten minutes, it stops working great.
Specifically, what happens is that while my app continues to receive Q, P, and F update lines perfectly well, IQFeed apparently stops listening to watch requests. I know this because symbols which are perfectly good simply don't appear in the message stream, even after multiple attempts to "watch" them. Once this condition starts, it's terminal ... no matter which symbols I attempt to add to my watch list, nothing gets added. The only way to deal with this is to shut everything down and restart it.
To address the first question DTN will ask: I am not bumping up against the symbol limit in my account. I confirm this both via my own internal application's counting scheme, and through the Connection Manager Feed Stats.
I wonder if anyone else out there is using sockets programming w/ IQFeed and facing a similar issue. Has DTN heard of anything similar? If neither answer is yes, then it may very well be a problem in my code, but if so - I can't fathom what it can be - after all, I can sit and watch the incoming message stream continue to roll into my app from IQFeed... but no new symbols get added.
Can anyone help?
Matthew
Matthew Klein
It seems that Emini Crude (@QMX5) prices are rounded to two decimal places, when in fact they should have 3 significant digits.
MK
Matthew Klein
I'm sort of hoping someone at IQFeed might take an interest in this question. And, more than that, answer it.
Ten days have passed since I posted it. If this is not the appropriate place to ask this question, or the answer is simply "unknowable", then please tell me that, as well.
Thanks...
Matthew
Matthew Klein
|
|