Advise for this XML feed i am looking at

Archive
#1

I am trying to get this XML feed from this link.

NOTE: this forum is not properly creating the below link in its entirety. So copy and pate it in sections.

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20("AAPL")&env=store://datatables.org/alltableswithkeys

You will notice that the web page for this link opens a nice XML tagged list.
However if you run Wire Shark, and view the returned data is looks nothing like what you see on that web page. Its like compressed or something.
So my question is, what do i need to look up to be able to convert that to the XML information i am looking at on the web page ?

Things to note. I am not using .net to read this data. I came there because many of you have more knowledge that expands the .net/mf world, and i was hoping someone would shed some light on what is going on here.

#2

I used Fiddler to look at the data and it looks ok to me. (I removed some of the data).

The difference I see between Fiddler and Chrome, is that Chrome formatted the data. In the Fiddler raw view (below) the XML data is all on one line.

Not sure what you are looking to do with the data (language, usage of the data, etc) - if you provide that, then perhaps there’d be some suggestions.

#3

Yes, that is my point, when you view it in the browser its ok. But the data being sent to the browser is formatted somehow. I think the browser is decompressing that data and then displaying it. The server reply is 1667 bytes. the first part is legible but all the data after that is not. That data is i feel encoded somehow and the browser is decoding it then displaying it as XML.

AFter this all the rest of the data is just illegible as if you were looking at a raw .zip file data.

#4

Just noticed this in the header —

“Content-Encoding: gzip”

So the data is compressed as you mentioned. I surprised that the “Raw” setting in Fiddler uncompressed it.

#5

Ewww, that sucks for me. I have to code my own gzip converter ;(

#6

Being overly technical, you need to write a decompression routine, and not a converter.

Maybe someone else has written one in your non .NET environment?

#7

isn’t calling it a converter still the same?
I can convert it from gzip to xml, or a can convert it from xml to gzip.
in the light of this context we know that i need to convert it from gzip to XML.

#8

Loosely, sure why not. For me, I wouldn’t but that’s just me, so don’t put to much stock in that :smiley:

GZip is a compression scheme and not a data format (as XML is) - it can compress xml, text, binary (not very well), etc …

#9

No way would i do that :wink: . just seeing if my lingo was jacked up.

just doing some googleing for some C source that does the conversion. Hope fully its not that large.
The real @ #%&@ of it, is that i have a side project that i need to so this all in basic! Now that is going to be a royal hemorrhoid to convert to do.

#10

[quote=“jdal”]

No way would i do that :wink: . just seeing if my lingo was jacked up. [/quote]

I think I’ve found some middle ground in our lingo discussion … :wink:

You need to write code that converts a compressed GZip byte stream to an uncompressed byte stream, which just so happens to be in XML format.

[quote]just doing some googleing for some C source that does the conversion. Hope fully its not that large.
The real @ #%&@ of it, is that i have a side project that i need to so this all in basic! Now that is going to be a royal hemorrhoid to convert to do.[/quote]

Your project is in basic … VB.NET? VB6? QBasic? TurboBasic?

#11

actually none of these. the closest one its akin to is Qbasic.

#12

FWIW, i found a simpler solution.
If one looks at the GET message sent to the server is this:

[code
]GET /v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22AAPL%22)&env=store://datatables.org/alltableswithkeys&Accept=text/plain HTTP/1.1
Host: query.yahooapis.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive



Is one simply changes the "Accept-Encoding: gzip, deflate" to "Accept-Encoding: gzipdeflate" this will make the server not respond with gzip data, and life is now normal again ;)
#13

[quote=“jdal”]Is one simply changes the “Accept-Encoding: gzip, deflate” to “Accept-Encoding: gzipdeflate” this will make the server not respond with gzip data, and life is now normal again :wink:
[/quote]
I was just going to suggest messing with the request header to turn off compression.

Text compresses very, very nicely so doing this means the response size is going to grow considerably - dunno if you care about that.

Also, it’s probably not that “gzipdeflate” is the magic word. It’s more likely that the server doesn’t understand it so it falls back to plain old text. You might also try leaving that header out or using “text” or “jdalWantsText” or something that’ll make more sense if you revisit this code in a year…

#14

ok, thanks for the tip. I grabbed that “gzipdeflate” from the Get information from NASDAQ RSS feeds.
So i just figured it was a correct way to tell the server not to gzip it

#15

…or it could be that I’m full of it and it’s a valid encoding.

#16

Much easier! And less resource intensive on your side.

BTW - MSFT stock is quite a bit cheaper :wink:

#17

Speaking of MS, i love the promo for the new Surface device here
http://www.microsoft.com/surface/en/us/default.aspx

Looking for more information on it, i found this youtube video of MS demoing it and it locked up. Jeez, MS just cant get a break.