Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"If you are serious about your trading I would not rely on IB data for serious daytrading. Took me a while to justify the cost of IQ Feed and in the end, it's just a 2 point stop on ES. Better safe than sorry" - Comment from Public Forum
"Thank God for your Data Feed as the only Zippers I see are on my pants (LOL), and no more 200 pip spikes to mess up charts." - Comment from Spiro via Email
"Thanks for the great product and support. During this week of high volume trading, my QuoteTracker + IQ Feed setup never missed a beat. Also, thanks for your swiftness in responding to data issues. I was on ******* for a few years before I made the switch over early this year, and wish I had done it a long time ago." - Comment from Ken
"Boy, probably spent a thousand hours trying to get ******* API to work right. And now two hours to have something running with IQFeed. Hmmm, guess I was pretty stupid to fight rather than switch all this time. And have gotten more customer service from you guys already than total from them… in five years." - Comment from Jim
"Can I get another account from you? I am tired of ******* going down so often" - Comment from George
"For anyone considering using DTN.IQ for a data feed, my experience with the quality of data and the tech support has been very positive." - Comment from Public Forum
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"Just a quick one to say I'm very impressed so far :) The documentation for developers is excellent and I've quickly managed to get an app written to do historical downloads. The system is very robust and pretty quick considering the extent of data that's available. The support guys have been very helpful too, in combination with the forums it's been plain sailing so far!" - Comment from Adam
"I ran your IQFeed DDE vs. my broker vs. a level II window for some slow-moving options. I would see the level II quote change, then your feed update instantaneously. My broker's DDE, however, would take as much as 30 seconds to update. I am not chasing milliseconds, but half a minute is unacceptable." - Comment from Rob
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »IQFeed Developer Wish List »Multi-Threaded Decompression / Get Number of Ticks
Author Topic: Multi-Threaded Decompression / Get Number of Ticks (5 messages, Page 1 of 1)

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 19, 2013 05:17 PM          Msg. 1 of 5
Is there any chance we could get multi-threaded decompression in IQConnect? I'm bottlenecking on CPU rather than download bandwidth.

Currently I just pull all my symbols at once, which means I bottleneck on bandwidth which makes me happy. However I would like to pull a single symbol at full network bandwidth.

Another thing that could allow me to implement this on my end is if you add a 'get number of ticks', so I could query how many ticks of history there is for a symbol. Once I get that number I could start at trades/threads offsets and do a multi-threaded pull.

Any chance this could happen soon?

-Brandon
Edited by gamozo on Sep 19, 2013 at 05:22 PM

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Sep 19, 2013 05:22 PM          Msg. 2 of 5
Assuming you are talking about historical data downloads, each socket connection to the lookup port is handled in it's own thread. Therefore, you can simply create multiple socket connections to the feed and balance your requests across the sockets to do a multithreaded pull.

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 19, 2013 05:24 PM          Msg. 3 of 5
Yep. That's what I currently do. However that only works with multiple symbols. It would be nice to have one symbol at full speed. And the only way I could think of doing this is if I knew how many ticks there were historically, with that I could start multiple queries at different starting points. I guess I could do it in chunks of 100k trades or so and just guess for now, but it'd be nice to know how many trades to expect.

DTN_Steve_S
-DTN Guru-
Posts: 2093
Joined: Nov 21, 2005


Posted: Sep 20, 2013 09:12 AM          Msg. 4 of 5
How large of requests are you making? Have you looked into requesting data by timeframe instead of by X number of datapoints?

gamozo
-Interested User-
Posts: 4
Joined: Apr 2, 2013


Posted: Sep 20, 2013 10:09 AM          Msg. 5 of 5
Just tick-level on pretty much all the top 20 most traded futures. I think @ES# has about 55 million datapoints at tick level.

I pull at ticklevel as I postprocess all the data and frequently use tick data for simulated fills rather than n-length bars.

It's not a huge issue, I usually just pull once and then have the files around forever. I'm just requesting the feature if it's easy to add as it would be nice here and there.

-Brandon
 

 

Time: Wed April 17, 2024 10:00 AM CFBB v1.2.0 12 ms.
© AderSoftware 2002-2003