Hi I have been using your feed for perhaps five years it has worked great.

However it seems over the summer a bug crept into the history command return info. This is subtle but when I'm processing ten thousand or so records it shows up and crashes my software!

There is an excess field of "0" sometimes between the symbol and the date/time field. I will put an example below showing weekly data. I first saw this with the daily data.

Also for some reason this historical data now shows the current TIME along with the date, which used to be all zeros. There is no need to have the TIME, but I used to look for the zero field "00:00:00" and nuke it. Now it is when the data is retrieved TOD.

Anyway, if you look at the following weekly you will see ACTA 20140829 field has excess zero:

ACTA,20140530 10:48:49,19.97,18.96,19.46,19.32,480515,0,
ACTA,20140606 10:48:49,19.52,17.95,19.39,19.21,655178,0,
ACTA,20140613 10:48:49,20.1400,18.9000,19.1300,19.7700,485706,0,
ACTA,20140620 10:48:49,20.8100,19.6300,19.8200,20.5400,850081,0,
ACTA,20140627 10:48:49,21.000,20.375,20.580,20.720,991286,0,
ACTA,20140703 10:48:49,21.120,20.562,20.690,20.970,601200,0,
ACTA,20140711 10:48:49,21.210,19.050,20.880,19.370,534954,0,
ACTA,20140718 10:48:49,19.77,17.77,19.64,18.06,794759,0,
ACTA,20140725 10:48:49,18.2500,17.3100,17.9300,17.3800,500255,0,
ACTA,20140801 10:48:49,17.380,16.120,17.380,16.870,1273759,0,
ACTA,20140808 10:48:49,17.81,16.38,16.96,17.64,918948,0,
ACTA,20140815 10:48:49,18.40,17.59,17.81,17.80,501624,0,
ACTA,20140822 10:48:49,18.680,17.750,18.010,18.100,473548,0,
ACTA,20140829 10:48:49,18.370,17.230,18.220,17.350,474072,0,
ACTA,0,20140829 10:48:49,18.370,17.230,18.220,17.350,474072,0,
ACTA,20140905 10:48:49,20.4900,16.5000,17.4700,16.6200,429699,0,
ACTA,20140912 10:48:49,18.35,14.03,16.53,17.31,587925,0,
ACTA,20140919 10:48:49,18.0800,15.6200,17.0900,15.9900,870417,0,
ACTA,20140926 10:48:49,16.7600,13.2400,13.2400,16.4300,1270043,0,
ACTA,20141003 10:48:49,16.7800,15.3200,16.2300,16.5600,983706,0,
ACTG,20140926 10:48:50,17.2200,15.7700,16.9700,15.9000,1350590,0,
ACTG,20141003 10:48:50,16.1040,14.9400,15.7200,15.4900,1569218,0,
ACU,20140926 10:48:51,17.1800,16.5000,17.1437,16.5000,32192,0,
ACU,20141003 10:48:51,16.9650,16.4100,16.4100,16.9001,15358,0,
ACXM,20140926 10:48:51,18.0000,17.0100,17.9700,17.1350,3083174,0,
ACXM,20141003 10:48:51,17.2700,16.2000,16.8900,17.0550,2853072,0,
ADBE,20140926 10:48:57,69.2500,65.7900,66.7000,68.3550,18337959,0,
ADBE,20141003 10:48:57,69.5000,66.1500,67.6900,68.4000,17733084,0,
ADC,20140926 10:48:58,28.6500,27.2000,28.1400,27.5900,327636,0,
ADC,20141003 10:48:58,27.9500,27.0000,27.3400,27.0900,371740,0,
ADEP,20140926 10:48:58,8.3900,7.5301,7.9300,8.3700,536684,0,

Hello,

Is there any chance you could tell us the exact request that was made on ACTA? Each request is individual to itself, and creates its own unique connection, so if you have the request that could certainly help us track it.

As to the time, these are deprecated in the newer protocols and are no longer sent in the more recent design. So it may be worth your time to consider upgrading to the 5.1 protocol to see if that resolves the other issue you are having as well.

Tim

I don't know the exact request, it is calculated based on my local storage of the data requesting the back data for each of about 3800 symbols. But I consistently do the same thing, making requests on these symbols from 26 separate threads. Only the symbol and start date varies.

I put in debugging info to show me when this zero field shows up and it is not often, this looked to be the only symbol in this pass. When I first saw it there were quite a few more.

But basically somewhere in your code it either inserts this excess zero field or it doesn't. I can ignore when you generate this excess field but your code should not generate it at random. Nothing in my request should generate an extra field in a single record, either the field is there or not in all data returned.

It is something new that broke over the summer, as this code has been running fine for as I said five or so years.

I don't believe it is in my side although of course have been wrong before.. :)

Jim,

We are not seeing anything that would allow for this to happen without something happening in the request. Could you please enable logging on your historical requests and the next time this happens, send us a copy of the log, along with the return that is in error? You can enable logging via the diagnostic app, which is available via the system tray(right click), under support.

Tim

So what would I have to do in the request that would cause a single field to be added in some records?

Meaning what would cause a return record set to have all records except one to return expected number of fields?

I can enable logging in your app and I can store all the commands and socket returns in mine, but doubtful it is more significant than answering the question above.

My expectation is the request is like all the other requests, but somehow occasionally you populate an excess zero field. If it were me looking at the code to see how an excess field could get in there would be my first stop... :)

The two lines I use to build commands are:

strCmd.Format("HWX,%s,%d,1,%d\r\n", strSymbol, nWeeks, nCur );

strCmd.Format("HDT,%s,%d,%d,,1,%d\r\n", strSymbol, lStart, lToday, nCur );


If you want send me the logic that generates the return I will look at it. Have been programming forty years can probably figure it out. :)

In the meantime I'll monitor my data and look for the errors. But I have a workaround.

This originally was hard to track down because it actually blew out the atoi() Microsoft function and caused the stack to corrupt and crash asynchronously later on in a dialog message handler. Very ugly. Took a while to narrow it down..

Hi Jim,

Given your code, strCmd.Format("HWX,%s,%d,1,%d\r\n", strSymbol, nWeeks, nCur );

And the return generated.

ACTA,20140530 10:48:49,19.97,18.96,19.46,19.32,480515,0,

What is nCur in this case? It appears in your example you are sending a request ID of the symbol name, hence why we precede each line with, in the example ACTA. But the formatting example you gave shows nCur as a digit instead of the expected string.

Is there another place where the string is getting manipulated perhaps? We don't have a reason why one line should be different than the rest, hence why we want to understand the requests being sent and the errors they are creating, so any further examples or logs would be helpful for us to try and recreate this issue.

Thanks for any help you can provide,

Tim

Thanks for being on top of this Tim.

The nCur is my index into my array of stocks, it comes back with the data, which I parse off the front. Then this ordinal is compared to the ordinal requested to match the data. Subsequently the lines with this ordinal are prefaced with the matching Symbol name and are put into files one for each letter of the alphabet. Once all the data is downloaded and stored into these files other code parses the files and updates individual stock files.

So basically the nCur gets me back to my data making sure the values match on return info. I think at one point had multiple requests in the same thread to match up but converted this all to linear processing of requests in multiple threads is the reason did it this way. So I have one thread for "A", one for "B", etc..

I just added into this logic to look for the zero situation when it first gets read so can add the raw line right off the socket read to the debug output. Will see if anything bad shows up.

BTW my real-time logic is reading all the tape data for up to 1800 symbols at one point and it has worked great. This is the first time I've seen any issues with your software. (i think, could be me!) On the other hand I spent an entire summer trying to get eSignal API working on their end about a decade ago. They never seemed to test it or want to fix it. Friends joked they must have hired a drunk summer intern.... Your offering is very solid.

Good morning,

Thanks, we are all very committed to the product and trying to help provide you with a stable platform, so it is great to have some feedback to confirm what we do.

We will be awaiting to hear what comes of the additional logging, but if nothing appears obvious on the next occurrence, just send it on to us and we can do some additional testing with the request in question to see if we can re-create something here.

Tim