Hello,

I am successfully able to access and read data from the API but need support in order to interpret the information I am receiving back from your servers. I am guessing this is a common request from new users since the data format appears to be somewhat proprietary...

It's totally possible I may have missed something in the documentation so any pointers are much appreciated by anyone in the community here that reads this (thanks in advance).

The return data format I am receiving is in a format I'm not sure how to interpret, which appears to be a custom compression scheme I will need to unwind in my code. Generally speaking, the documentation does not seem to clearly spell out the meaning of the returned data in a way that I am confident such that I'm able to interpret it's meaning. In other words, I'm confused what the data I'm getting back means. In a general sense is there a page in the docs (maybe I missed it) that clearly explains the data format that is returned from the API for all API calls?

Let's use Equity Option Chains as an example. If I request an Equity Option Chain, I expect to receive the data that is associated with an Equity Option Chain (Symbol, Strike, Last, Change, Bid, Ask, Volume, and Open Interest) but instead it appears I'm getting an unexpected compressed subset of this data in a compressed format that I don't fully know what it means:

First, I read over this page:
http://www.iqfeed.net/dev/api/docs/OptionChainsviaTCPIP.cfm

Then, I started requesting puts and calls (pc) Equity Option Chains using the CEO call - in this example, for Intel Corp. (INTC), the API returns unexpected values like this:

'INTC1602U44', 'INTC1602U45', 'INTC1607V28', 'INTC1607V28.5', 'INTC1607V29'

Let's break this down using data from the above example. Hopefully there is a document that clearly outlines things like this?

INTC -- Obviously the first element is the ticker symbol.

16 -- It's unclear to me what the next number is but it appears to be the option chain date measured in day of the month - I only was able to decipher this by examining option chains from Nasdaq's website then trying to visually pattern match.

02 (and 07) -- The next double digit number is unclear. Is it the days until expiration?

U (and V) -- I found the letter codes for months in the docs (explained for sending requests but not clearly outlined for the return data format), regardless, it appears in this case the month's letter depends on if the return value is a call or a put. In this case, it is a put, then it appears the month for U is September and the month for V is October.

44, 45 [...] 28.5, 29 -- this appears to be the strike price for buying the put.

Questions:

1) What does the data mean, please feel free to annotate my above example and please note any differences I haven't mentioned between calls and puts.

2) Where is the rest of the option chain data? ie: Last, Change, Bid, Ask, Volume, and Open Interest ...etc... and maybe implied volatility (but I can probably take care of that one)

Also, I am generating a request id for internal book keeping in my code, it's not clear to me if this is being tacked onto each element of the returned data, but it looks like it is only getting attached to the entire block of returned data for a single request. Just mentioning this in case it makes any difference in the return format other than what I've noted.

I will need concise explanations for all of the return data formats and feed types I'm subscribed to (ie: 'OPRA', 'NYSE', 'NASDAQNDX', 'NASDAQ', 'APCMB', 'CBSMKTW', 'CBOENDX', 'AMEX', 'RT_TRADER', 'COMTEX', 'PRIMEZONE', 'INTERNET_WIRE', 'DTNNEWS', 'SCROLLNEWS', 'EXFUND', 'BENZINGA', 'MN_TRADER') and am having trouble finding the full breakdown of this information in the documentation contained on the website.

Thank you in advance for your help!

-e
Edited by etron on Aug 26, 2016 at 06:57 PM

Hello, the chains request simply returns a comma delimited list of contracts that are part of the requested chain.

The symbols are in OPRA OSI format as documented here on our website:
http://www.iqfeed.net/symbolguide/index.cfm?symbolguide=guide&displayaction=support§ion=guide&web=iqfeed&guide=options&web=IQFeed&type=stock

To get price information for the contracts, you will have to issue a request to subscribe to updates for the symbols on the Level 1 port of IQFeed (see http://www.iqfeed.net/dev/api/docs/LevelIviaTCPIP.cfm )

RequestIDs are populated as the first field in any complete message returned from the feed for a request. In the case of chains, it will typically be all returned in a single message followed by the end of transmission message (which should also have the request ID attached).

Most of the authorization codes you list are exchanges which you have signed up to receive so those should be self explanatory. The rest are premium authorizations for News sources which you can access via the Lookup port in IQFeed (same port you are getting chains data from, you just need to send news requests instead of chains request)

Hope this helps.
Edited by DTN_Steve_S on Aug 26, 2016 at 07:44 PM

This is a great start thanks for the pointers I didn't realize that was OPRA OSI. Looking at my account, I'm already subscribed to RT Equity Options, RT NASDAQ Level 1 and Real Time CBOE so this should be good to request Level 1 data?
I'll give it a shot and post back my results. Thanks again for the tips and quick response.

Update on getting Option Chains! The journey continues. Need some more help please:

So, I tried retrieving Options Chain data by getting the options symbols using CEO (http://www.iqfeed.net/dev/api/docs/OptionChainsviaTCPIP.cfm) and then passing those retrieved options OPRA symbols into a Level 1 watch request using ALL the fields available as outlined here http://www.iqfeed.net/dev/api/docs/LevelIviaTCPIP.cfm but this did not work. Some of the fields had some data but it was mostly NAN or empty return data. Then I found the Derivatives port 9400 and that looked more promising.

So -- thanks to your pointer and link regarding the fact that the option chain symbols are in OPRA OSI format and, with a little help from the developer documentation, mostly these two pages:

Derivatives overview:
http://www.iqfeed.net/dev/api/docs/Derivatives_Overview.cfm

Derivatives Streaming interval bars via TCP/IP:
http://www.iqfeed.net/dev/api/docs/Derivatives_StreamingIntervalBars_TCPIP.cfm

I am able, from a single symbol, to request historical derivatives data passing in an options symbol and date range using:

BW,[Symbol],[Interval],[BeginDate BeginTime],[MaxDaysOfDatapoints],[MaxDatapoints],[BeginFilterTime],[EndFilterTime],[RequestID],[Interval Type],[Reserved],[UpdateInterval]

And getting back a stream of data, in seconds as well as in ticks (tick resolution of 1 was giving me an error I haven't investigated yet so using tick resolution of 2 for now).

With the data returned and by parsing the relevant information from the OPRA OSI symbology [Underlier (Ticker), Type (Put/Call), Expiration (datetime), Strike price]

I am able to retrieve packets of data with the following information for example, with a single option symbol INTC1623U35 I can get a bunch of historical options data in this format (this is an internal data structure but corresponds to a the OPRA data and data returned from the BW request):

'date_time': (numpy.datetime64('2016-09-06'), 38015000000),
'expiration': numpy.datetime64('2016-09-23'),
'price_high': 0.14,
'price_last': 0.14,
'price_low': 0.14,
'price_open': 0.14,
'response_type': 'history',
'strike_price': '35',
'symbol': 'INTC1623U35',
'tick_trades': '1',
'type': 'put',
'underlier': 'INTC',
'volume_cummulative': '10',
'volume_internal': '10'

The problem is, this is that this is not the exact data I'm expecting for Option Chains.
This is the data I was expecting to gather for option chains (historical and real-time):
http://www.nasdaq.com/symbol/intc/option-chain

According to Nasdaq the option chain for the Sept. 23rd expiration INTC put w/ $35 strike price, as of today (granted the data from the stream is historically dated from yesterday)

Strike: 35
Puts / Expires: Sep 23, 2016
Last 0.11
Chg -0.03
Bid 0.10
Ask 0.12
Vol 25
Open Interest: 1057

So, there are some minor but notable discrepancies (and missing data like Open Interest, etc) between expected option chain data, the data found at Nasdaq and what we're seeing... Here are some questions:

1) Where is "Last", "Change", "Bid", "Ask" and "Open Interest" and how can I get the full option chain data?

2) What is the difference between Volume Cumulative and Volume Internal and how should one compute the actual option chain volume number from these two volume representations?

3) No matter the interval I choose (i.e. 2 ticks or 240 seconds, etc) it seems the values for OHLC i.e. "open" "high" "low" and "last" are always exactly the same (in the vast majority of data returned) -- is this right? When requesting tick data, what does this mean (tick data is bid/ask not OHLC isn't it)?

Thanks in advance for all the info.

-e

Edited by etron on Sep 7, 2016 at 09:48 PM

I don't think that the Streaming bars feature is what you are looking for.

Background:
IQFeed provides both realtime tick by tick data (every trade/bid/ask/etc) and Historical data. Our historical data can be retrieved as "tick", "interval", or daily/weekly/monthly bars. The tick history is every trade for the symbol and includes bid/ask price data at the time the trade occurs (no data is stored for bids/asks that occur between trades). The intervalized data provides an OHLC plus interval volume as well as the total daily volume up to that point in the day. We support tick/time/volume interval bars with the most commonly requested being time (1min/5min/15min/etc). The daily/weekly/monthly data is EOD data.

The streaming bars feature merges interval historical data requests with the realtime tick by tick data subscriptions and then streams the new bars as they complete/update. As a result, the data supplied using this feature is limited to the same data that we provide via historical data requests. This is also the reason your interval request of 1tick returned an error. Requesting 1tick isn't an interval, its just tick data which isn't supported by the streaming interval bars feature.

In order to get the type of display that you see on nasdaq.com, you will want to go back to issuing your level 1 watch request and parsing that data for the information you need. You are correct that many of the fields will be empty when you request all fields. This is because our feed is security type agnostic and many of the fields won't apply to options data (just like fields such as strike price wouldn't apply to the underlying equity). You will have to parse the csv string that is returned and only look at the fields you want to see. alternatively, you can just request the specific fields you want (in the order you want) and the feed will deliver a message that is more to your liking. Take a look at this page for more info on selecting fields: http://www.iqfeed.net/dev/api/docs/DynamicFieldsets.cfm

You might also need/want to switch to the most recent protocol (5.2) using the S,SET PROTOCOL,5.2 command prior to setting your fieldset. Some fields are only available in the newer protocols.

Hi Steve,

Ok. Thanks for the quick response. Yes, I'm using protocol version 5.2 thanks.

It sounds like the Level 1 data will be the way to go in order to get real time Options Chains -- but what about getting historical Option Chains that correspond to real-time quotes?

Since the Derivatives tick history seems to not be returning bid asks -- even though in your above reply you said:

"The tick history is every trade for the symbol and includes bid/ask price data at the time the trade occurs"


Questions:

1) What about getting historical option chain data with Bid/Ask Open Interest etc?

2) Can you kindly please reply to this with two examples of using a REQUEST command that should be used to get an option chain from the OPRA OSI symbol and a quick summary of the data returned please (example1: historical and example2: real-time)?


Thank you for your help!

-e

Edited by etron on Sep 7, 2016 at 10:25 PM
Edited by etron on Sep 7, 2016 at 10:31 PM

The derivatives history only supports interval requests which returns OHLC data. It's a bit confusing because you can request an interval of X ticks (which you have), where each message represents an interval of data containing X number of trades. However, this is not the same as a historical tick data request where each message represents a single trade with the additional information about that trade supplied. However, this is still a subset of the available fields in level 1 data since not all fields are stored historically.

To get historical tick data, you will need to issue a historical data request on the lookup port (same port your CEO request is made on). You want to issue a Tick data request which is any of the requests that start with HT(HTX/HTD/HTT). These requests documented here: http://www.iqfeed.net/dev/api/docs/HistoricalviaTCPIP.cfm. All of the data returned by these requests are potentially the same but the 3 different requests allow for different types of filters.

For example: HTX,INTC1609I26,10 will give you the last 10 trades on that option symbol (in this case, the symbol has only traded twice):

2016-09-07 09:43:36.408743,9.60,5,5,8.40,9.60,5131387,C,11,20,
2016-09-01 09:30:01.004561,10.25,5,5,10.25,10.50,380528,C,11,20,

The format of the messages are documented in the above page of the documentation.

Issuing a watch request on the level 1 port of IQFeed for realtime streaming data will result in the following data being returned:
F,INTC1609I26,E,,,10.25,10.25,,,,,,,,,,,,,,,,,,INTC SEP 2016 C 26.000,,,,,,,,,,,,,,,12,2,,,2,14,09/01/2016,09/01/2016,,,,,,09/09/2016,26.000,,,100.00,0,
P,INTC1609I26,9.60,5,09:43:36.408743,11,5,8.30,282,12.55,93,9.60,9.60,9.60,10.25,Cbaohlcv,20,

Note that this is the default fieldset for IQFeed protocol 5.2. You can add/remove fields from this fieldset with the dynamic fieldsets in IQFeed to your liking. In addition to the above messages, if you keep the subscription active, you will receive quote messages (start with a Q,) anytime any of the data updates.