I often get an empty string from "socket.recv(1024).decode('latin-1')" when using IQFeed with wine & Python on Linux. If I restart the downloading process, and make the same HTT call, I get back a normal message (sometimes it's "NO_DATA", which is also normal).

Has anybody else experienced this, and knows how to fix it? My script auto-restarts the process whenever this happens, but it's still quite annoying.
Edited by jayqwi on Dec 8, 2022 at 04:48 PM

My app is written in C++ rather than Python, but I've often seen equivalent behavior. I believe this to be due to a bug in Wine. I work around it by closing the socket, opening a new one, and retrying the request. This works always, so far as I can tell.

Coincidentally, I haven't seen this happen in over a week, so I was wondering if a recent Wine update had fixed it. My current Wine version is 7.22 (Staging), the default version for Fedora 36.

I can't imagine why the same HTT request would return data some times and not others, unless it's running afoul of the "8/180" rule.

During regular trading hours (9:30 – 4:00 US Eastern time), tick data is only available for the last 8 days, and API requests will not return anything older during that time. Outside regular trading hours, tick data requests for up to 180 days will be processed.

So if you make the same request inside and outside of those times, and it's for data more than eight days old, that would explain getting different results. Beyond that, I'd need to see the entire request.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

As taa_dtn noted, I think this is most probably a bug in wine. Would be good if there's some workaround.

I've gone through a number of years using wine. I somehow doubt it is a wine issue. My connections have been stable.

I use LXDE and x2go Wine v7.0

stargrazer: How many threads (or processes) do you use for HTT commands? I've noticed that it's mostly when I have multiple workers (I've tried both threads and processes; same problem) that each open a socket to IQFeed and run HTTs. I think it only very rarely (if at all) happens when I have only one worker.

I'm using Ubuntu 18.04.4 LTS with wine-7.0

Are you using the the v6.2 of the client?

I don't necessarily use HTT, but a number of the other history commands, and can do several 1000 symbols without error. I think 8 to 10 threads is the limit. And I've incorporated the time quota into the commands.

The following may provide some insight:
https://github.com/rburkholder/trade-frame/blob/master/lib/TFIQFeed/HistoryBulkQuery.h
https://github.com/rburkholder/trade-frame/blob/master/lib/TFIQFeed/HistoryQuery.h

The biggest recommendation, is if you reuse a connection, be sure if you've processed all messages to completion prior to sending a new command.

I'm running client version 6.2.0.23 currently. I've been seeing this behavior for years, though, with many different versions of the client and a few different versions of the protocol and with two versions of my app. That's why I went to the trouble of implementing the workaround, and among the reasons why I suspect the problem isn't in the app or iqconnect.

In my case, each thread (of 8) issues a request and processes it to completion before issuing another request. Anything unexpected is logged, so I'm pretty sure there's nothing leftover that hasn't been processed. I simply get no response (and I've also tried a wide range of timeouts with no success).

I am exactly in the same situation. I also tried inserting delays between HTT calls to see if it helps, but it didn't. Even reopening the socket doesn't work for me sometimes, and as a last resort I also need to restart wine & IQFeed. All of this is automated, but it's quite annoying having to implement and maintain. Would be nice if there were a simpler workaround. I'm also pretty sure other people had the same problem and discussed it somewhere on these forums, but the search capability does not seem very easy to use.

Actually, it might be the threading thing. Not sure how you are doing it.

Use _only_ one thread per socket. You can have multiple requests per socket by using the tag feature. Only have about 8 - 10 requests submitted at any one time.

==> 1 socket - 1 thread - multiple tagged requests

In actual fact, I'd have to scrunch back through my code. I think I may have multiple sockets open, one thread per socket, with no more than 8 to 10 sockets, running one request in each socket at a time.

Edited by stargrazer on Dec 11, 2022 at 10:37 AM
Edited by stargrazer on Dec 11, 2022 at 10:37 AM
Edited by stargrazer on Dec 11, 2022 at 10:43 AM

Yes, I always use only one thread per socket. So if I have 8 threads, each thread opens its own unique socket. Out of frustration, I also tried using one process per socket (each process using only one thread), but same problem.

Same here; each history-request thread has its own socket. I actually did this in reverse of the order jayqwi used; I started with each requester being a separate process, then switched to separate threads because load balancing is easier that way. I'm using tags, but only to make sure that no stale data arrives from previous requests; the intent is to run each request to completion before starting another.

Probably irrelevant, but while history-requests are being processed I do have at least two other independent processes with their own connections to the Level 1 port.

And I should emphasize that I haven't seen this misbehavior at all this month. Since my software hasn't changed (or even been recompiled), and iqconnect.exe hasn't been changed, I think that points to either Wine or the software running on the servers. I know there was a Wine update last month, so I suspect something got fixed in that release.

If something were interfering with the HTT command (like a bug in Wine), it should be possible to confirm this via the iqconnect.txt log. Turn on "lookup requests", and the log will contain all historical requests that were sent to the IQFeed API. If something is changing the HTT command such that it fails, it will show up here.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist