API version 4.1.1.1
Windows 2000
pentium 4, 2.8 Gz, 1 Gig Ram
256kb DSL
I have been getting bad data within the update message
It seems a security symbol is inserted into the Last field among other fields
Field counts of the update message do not vary in size when these errors occur
I have tried increasing socket buffer sizes on the Recv and creating separate threads
to handle incomming data to try eliminate the problem.
Below is a sample of the stream that I have captured to show the problem.

0x5B0E0000 51 2c 41 4d 41 54 2c 46 2c 30 2e 53 54 4b 4c 2c Q,AMAT,F,0.STKL,
0x5B0E0010 30 2e 30 33 2c 30 2e 30 30 31 36 34 39 32 35 38 0.03,0.001649258
0x5B0E0020 2c 35 32 35 35 37 37 39 2c 31 30 30 30 2c 31 38 ,5255779,1000,18
0x5B0E0030 2e 32 35 30 30 2c 31 38 2e 30 30 30 30 2c 31 30 .2500,18.0000,10
0x5B0E0040 2e 35 31 30 30 2c 31 30 2e 35 32 30 30 2c 32 30 .5100,10.5200,20
0x5B0E0050 30 2c 31 30 30 2c 31 37 35 2c 31 37 35 2c 30 2e 0,100,175,175,0.
0x5B0E0060 32 35 2c 31 30 3a 31 34 74 2c 2c 31 38 2e 31 33 25,10:14t,,18.13
0x5B0E0070 30 30 2c 31 38 2e 31 39 30 30 2c 30 2e 30 31 2c 00,18.1900,0.01,
0x5B0E0080 2c 2c 2c 74 2c 4e 2c 2c 2c 2c 30 35 2f 31 31 2f ,,,t,N,,,,05/11/
0x5B0E0090 32 30 30 36 2c 2c 30 2e 53 54 4b 4c 2c 2c 2c 2c 2006,,0.STKL,,,,
0x5B0E00A0 30 2e 30 33 2c 30 2c 32 38 2e 30 2c 2d 30 2e 37 0.03,0,28.0,-0.7
0x5B0E00B0 36 36 37 38 32 39 36 39 2c 2d 37 2e 37 31 2c 2d 66782969,-7.71,-
0x5B0E00C0 37 2e 37 2c 30 2e 30 39 2c 31 2c 30 2e 30 31 33 7.7,0.09,1,0.013
0x5B0E00D0 37 32 31 31 38 36 2c 32 38 39 30 37 33 39 36 2e 721186,28907396.
0x5B0E00E0 35 2c 31 34 2c 34 2c 2c 32 30 32 36 36 35 36 35 5,14,4,,20266565
0x5B0E00F0 2c 41 4d 45 58 2d 42 53 45 2d 43 53 45 2d 43 48 ,AMEX-BSE-CSE-CH
0x5B0E0100 58 2d 50 53 45 2d 4e 4d 53 2c 2c 2c 2c 2c 39 30 X-PSE-NMS,,,,,90
0x5B0E0110 38 34 2c 2c 2c 31 38 2e 31 33 39 31 2c 2c 4e 2c 84,,,18.1391,,N,
0x5B0E0120 0a 51 2c 50 43 41 52 2c 46 2c 37 39 2e 36 32 30 .Q,PCAR,F,79.620
0x5B0E0130 30 2c 2d 30 2e 36 34 2c 2d 30 2e 30 30 37 39 37 0,-0.64,-0.00797
0x5B0E0140 34 30 38 34 2c 31 36 31 33 38 36 2c 33 30 30 2c 4084,161386,300,
0x5B0E0150 38 30 2e 33 30 30 30 2c 37 39 2e 31 30 30 30 2c 80.3000,79.1000,
0x5B0E0160 37 39 2e 35 38 30 30 2c 37 39 2e 36 34 30 30 2c 79.5800,79.6400,
0x5B0E0170 35 30 30 2c 33 30 30 2c 2c 31 37 35 2c 31 2e 32 500,300,,175,1.2
0x5B0E0180 2c 31 30 3a 31 34 62 2c 2c 37 39 2e 39 32 30 30 ,10:14b,,79.9200
0x5B0E0190 2c 38 30 2e 32 36 30 30 2c 30 2e 30 36 2c 2c 2c ,80.2600,0.06,,,
0x5B0E01A0 2c 74 2c 4e 2c 2c 2c 2c 30 35 2f 31 31 2f 32 30 ,t,N,,,,05/11/20
0x5B0E01B0 30 36 2c 2c 37 39 2e 36 32 30 30 2c 2c 2c 2c 2d 06,,79.6200,,,,-
0x5B0E01C0 30 2e 36 34 2c 30 2c 31 31 2e 34 2c 2d 30 2e 38 0.64,0,11.4,-0.8
0x5B0E01D0 34 38 38 38 39 35 31 33 2c 30 2e 2c 30 2e 2c 2d 48889513,0.,0.,-
0x5B0E01E0 30 2e 33 2c 31 2c 30 2e 30 31 35 30 37 31 35 39 0.3,1,0.01507159
0x5B0E01F0 2c 31 33 32 37 34 31 36 36 2e 37 38 30 30 30 30 ,13274166.780000
0x5B0E0200 30 30 31 2c 31 34 2c 34 2c 2c 31 35 37 36 31 36 001,14,4,,157616
0x5B0E0210 36 2c 41 4d 45 58 2d 42 53 45 2d 43 53 45 2d 43 6,AMEX-BSE-CSE-C
0x5B0E0220 48 58 2d 50 53 45 2d 4e 4d 53 2c 2c 2c 2c 2c 39 HX-PSE-NMS,,,,,9
0x5B0E0230 37 30 2c 2c 2c 37 39 2e 34 36 32 30 2c 2c 4e 2c 70,,,79.4620,,N,
0x5B0E0240 0a 51 2c 52 49 4d 4d 2c 46 2c 37 34 2e 33 38 30 .Q,RIMM,F,74.380
0x5B0E0250 30 2c 30 2e 37 2c 30 2e 30 30 39 35 30 30 35 34 0,0.7,0.00950054
0x5B0E0260 33 2c 31 32 31 33 30 34 38 2c 31 30 30 2c 37 35 3,1213048,100,75
0x5B0E0270 2e 30 30 30 30 2c 37 33 2e 37 38 30 30 2c 37 34 .0000,73.7800,74
0x5B0E0280 2e 34 32 30 30 2c 37 34 2e 34 36 30 30 2c 35 30 .4200,74.4600,50
0x5B0E0290 30 2c 31 30 30 2c 2c 31 37 33 2c 31 2e 32 32 2c 0,100,,173,1.22,
0x5B0E02A0 31 30 3a 31 34 62 2c 2c 37 34 2e 36 33 30 30 2c 10:14b,,74.6300,
0x5B0E02B0 37 33 2e 36 38 30 30 2c 30 2e 30 34 2c 2c 2c 2c 73.6800,0.04,,,,
0x5B0E02C0 70 2c 4e 2c 2c 2c 2c 30 35 2f 31 31 2f 32 30 30 p,N,,,,05/11/200
0x5B0E02D0 36 2c 2c 37 34 2e 33 38 30 30 2c 2c 2c 2c 30 2e 6,,74.3800,,,,0.
0x5B0E02E0 37 2c 30 2c 34 30 2e 33 2c 2d 30 2e 36 38 31 36 7,0,40.3,-0.6816
0x5B0E02F0 39 38 32 34 32 2c 30 2e 30 33 2c 30 2e 30 37 2c 98242,0.03,0.07,
0x5B0E0300 2d 30 2e 32 35 2c 31 2c 30 2e 30 31 36 34 30 32 -0.25,1,0.016402
0x5B0E0310 32 35 39 2c 31 33 37 36 38 37 37 39 2e 33 31 39 259,13768779.319
0x5B0E0320 39 39 39 39 39 38 2c 31 34 2c 34 2c 2c 35 39 39 999998,14,4,,599
0x5B0E0330 38 34 34 32 2c 42 53 45 2d 43 53 45 2d 43 48 58 8442,BSE-CSE-CHX
0x5B0E0340 2d 50 53 45 2d 4e 4d 53 2c 2c 2c 2c 2c 35 38 38 -PSE-NMS,,,,,588
0x5B0E0350 39 2c 2c 2c 37 34 2e 35 31 35 30 2c 2c 4e 2c 0a 9,,,74.5150,,N,.
0x5B0E0360 51 2c 53 54 4b 4c 2c 46 2c 31 30 2e 35 31 30 30 Q,STKL,F,10.5100
0x5B0E0370 2c 2d 30 2e 38 32 2c 2d 30 2e 30 37 32 33 37 34 ,-0.82,-0.072374
0x5B0E0380 32 32 38 2c 36 34 33 36 35 39 2c 32 30 30 2c 31 228,643659,200,1
0x5B0E0390 31 2e 34 38 30 30 2c 31 30 2e 35 33 30 30 2c 31 1.4800,10.5300,1
0x5B0E03A0 30 2e 35 31 30 30 2c 31 30 2e 35 32 30 30 2c 32 0.5100,10.5200,2
0x5B0E03B0 30 30 2c 32 30 30 2c 2c 31 37 35 2c 30 2e 39 35 00,200,,175,0.95
0x5B0E03C0 2c 31 30 3a 31 34 62 2c 2c 31 31 2e 34 30 30 30 ,10:14b,,11.4000
0x5B0E03D0 2c 31 31 2e 33 33 30 30 2c 30 2e 30 31 2c 2c 2c ,11.3300,0.01,,,
0x5B0E03E0 2c 70 2c 4e 2c 2c 2c 2c 30 35 2f 31 31 2f 32 30 ,p,N,,,,05/11/20
0x5B0E03F0 30 36 2c 2c 31 30 2e 35 31 30 30 2c 2c 2c 2c 2d 06,,10.5100,,,,-
0x5B0E0400 30 2e 38 32 2c 30 2c 36 32 2e 39 2c 2d 30 2e 36 0.82,0,62.9,-0.6
0x5B0E0410 37 36 33 39 30 36 34 39 2c 30 2e 2c 30 2e 2c 2d 76390649,0.,0.,-
0x5B0E0420 30 2e 38 39 2c 31 2c 30 2e 30 39 30 33 39 30 31 0.89,1,0.0903901
0x5B0E0430 30 35 2c 35 39 33 38 39 39 2e 30 38 2c 31 34 2c 05,593899.08,14,
0x5B0E0440 34 2c 2c 31 37 39 30 32 38 37 2c 42 53 45 2d 43 4,,1790287,BSE-C
0x5B0E0450 53 45 2d 50 53 45 2d 4e 4d 53 2c 2c 2c 2c 2c 31 SE-PSE-NMS,,,,,1
0x5B0E0460 35 32 37 2c 2c 2c 31 30 2e 39 39 30 32 2c 2c 4e 527,,,10.9902,,N
0x5B0E0470 2c 0a 51 2c 50 47 2c 44 2c 35 35 2e 37 31 2c 2d ,.Q,PG,D,55.71,-

---

http://mm95.com

here are the mesages you posted with the last partial message removed.

can you give us more information about your app and the envornment it is running when you see the problem?

Can you see any pattern to when it is happening?
How many symbols are you watching?
Which symbols?
Does your app store this data into a database or a log file?
What language is your app programmed in?
Would you be able to provide us an example app (preferably with source) that duplicates the problem so we can test this internally?

Every bit of information you can provide will help us in finding the source of this problem.

I am watching about 1300 symbols.
The problem seems to occur when the market is very active.
The symbols are the most active (biggest volume movers).
No log file, the stream you see was derived from my debugger MS Visual Studio directly from the connected socket.
The MM95 application is programed in assembly language.
I cant release the sorce code.
The MM95 program is accessible from the MM95 website http://mm95.com.
Log in: Gene
Password: 4356

---

http://mm95.com

I updated the subscribtion date the MM95 app is now active (Sorry)

Roger Hanson

---

http://mm95.com
Edited by MM95 on May 11, 2006 at 01:33 PM
Edited by MM95 on May 11, 2006 at 02:01 PM

Roger, I have sent you an email to the email address registered on these boards as well as to the email address registered on your developer account.

Also, just so you know, this is a public message board (it is not restricted to developers). Just figured I would offer that information in case you didnt want the account information in your previous post to be public.

Roger,

Your 256kbps doesn't seem adequate to watch 1300 symbols. Naturally it depends on what you're watching but my experience suggests at least 512kbps is required for NASDAQ and around 300-350kbps for NYSE (during busy periods) for 1300 symbols.

Lack of bandwidth, CPU and high latency can all cause data corruption. What is your ping time to the IQ servers? I can tell you that 45ms results in virtually no corruption while you will get thousands of corrupt packets at 300ms.

One more thing, once you start getting corruption for a specific symbol it will continue even after the cause (eg burst of traffic) has abated. You need to unwatch and rewatch the symbol.

cheers,
Dennis

However, I watch only one symbol. I thought it was my software, but I too get random corruptions, perhaps an hour or two into the trading day. I'm going to see if I can catch it in the act and send in what I have.

Thanks Dennis,
i usually ping around 50MS.
It seems this problem also occures with my customers that are watching about 500 symbols with good bandwidth.
I will test your idea on other connections and see if it duplicates.
Thanks for the info.

Roger

---

http://mm95.com