Hi,
I'm hoping to please ask for some advice from the forum members who are more experienced than me regarding some novice/basic system design questions (as I'm new to building an algorithmic trading system from scratch). I’ve 3 questions (#1-#2 are relevant together, #3 I could post as its own thread if preferred but didn’t want to clog up the forum with lots of novice questions so grouping in one thread).

Context: The end result I'm working towards is a live trading system that uses minute bars to make trading decisions (in future I may go down to N second bars). I've signed up for DTN IQFeed and have consumed historical minute data for my tickers (using request: "HIT,%s,60,20180601 075000,,,093000,160000,1\n" % sym) so I think I’m now in good shape with the 'backfill' piece of work and currently building a backtester.

Q1: I'm now struggling with the design aspect of keeping the minute data updated (every minute) and being able to use it for future paper & live trading. As I understand it, the most common option is to consume live tick data and compile minute bars from this, every minute. Are there any other options people recommend? For example, I wasn't sure if it'd be possible to consume 'historical' minute data, every minute (using the same HIT request format). If so, do you know how frequently the historical data is updated? For example, if I make a request for 11.03am-11.04am data at 11.04am+1sec, would the data be available then? I’ll never be running a HFT system where millisecond data is needed for decision making.

Q2: I can’t discern from the documentation if it’s possible to request and retrieve historical level 2 data, does anyone know if this is possible and what command to use? If it’s not available, then I suppose that dictates needing to consume and compile tick level data? (signed up for Nasdaq level 2 data)

Q3: I’ve read a lot of forum threads regarding how Linux isn’t properly supported and requires wine to launch. I’ve got it working in quite a manual way currently so hoping if I outline my approach, others might be able to point me in the right direction for an automated alternative so that I can cron job turning iqfeed on and off... I’ve setup an AWS EC2 instance with Ubuntu 18.04 LTS, installed vncserver and an Ubuntu desktop GUI that I access via tightvnc, installed wine and iqfeed via the GUI. To launch my iqfeed python file I start vncserver via an ssh terminal, then connect to the GUI via tightvnc, navigate to the wine DTN folder and manually launch iqconnect and iqlinklauncher, clicking ‘startiqlink’ and ‘connect’ on each pop up. Finally, from the ssh terminal I launch the python file. I’ve tried a whole bunch of different methods for launching iqconnect and iqlinklauncher from the terminal in headless mode but each has their own errors - I can state each option attempted and errors faced, but if anyone is able to share the terminal commands they use to launch without a GUI then I can try to recreate and debug via the error prompts (knowing that it’s a solvable problem).

Thanks in advance for any help you can provide
Edited by ChainsawDR on Jan 4, 2019 at 08:57 PM

Q1) IQFeed has a very handy feature called "streaming derivative data", which means you can request a stream of 1-minute data bars. This provides you with both historic and ongoing (streaming data), avoiding the need to construct such bars by yourself. Details: http://www.iqfeed.net/dev/api/docs/Derivatives_Overview.cfm and http://www.iqfeed.net/dev/api/docs/Derivatives_StreamingIntervalBars_TCPIP.cfm

Q2) AFAIK, Level-2 (market-depth) data is only available as live streaming data, not historical.

Q3) You don't need to turn the IQFeed client off - simply disconnecting your program from the client will do this automatically - once the client sees that it has no active connections, it automatically exits. I'll let other answer the part of your question related to launching the client. BTW, I think this part of the question warrants a dedicated thread.

---

Note: I am not a DTN employee; my post reflects my personal opinion and not necessarily DTN's

- Yair
IQML - IQFeed-Matlab connector
https://UndocumentedMatlab.com/IQML

Thank you altmany, that’s very helpful! I’ll post a separate question for #3.

Thanks again