45037

need help with bizarre java.net.HttpURLConnection behavior

I am attempting to download a jpg using HttpURLConnection and am encountering a very peculiar bug.

Here's the url: http://www.vh1.com/sitewide/promoimages/shows/m/my_antonio/video/supertrailer/seg_1/320x240.jpg

if you open it in a browser you will see the image.

However, when I use HttpURLConnection I don't get the image... What I get is a 301 which, quite strangely, redirects to http://wap.vh1.com

so

con.setInstanceFollowRedirects(true); //additional stream code here to go and get the stuff found in con

proceeds to go ahead and download the text from wap.vh1.com, rather than the jpg that you see in the browser.

I'm guessing that there is some header wackiness that's causing this, but I haven't the faintest idea what the host is expecting to see in order to redirect me to the same place as where it's redirecting the browser (and curl and wget and everything else I can think to point at it).

I'm just about ready to shoot myself, so, if you help me you will be preventing my 6 year old daughter from going fatherless.

Answer1:

The site redirects you based on user-agent. Add this before you open the connection,

conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.15) Gecko/2009101600 Firefox/3.0.15");

Answer2:

It seems like the server interprets your request as coming from a mobile device, possibly based on the User-Agent header. That's why your redirected to the mobile site. Try setting the User-Agent explicitly.

Answer3:

For more flexibility you can utilize the http commons libraries, which have great debugging support for the wire through log4j ...

Also, user agents and more request parameters can be set easily.

For more information, see their tutorial.

Answer4:

While I can't help you with your specific problem, here's what I would do:

    <li>

    Download wireshark, sniff the HTTP request sent by your java application

    </li> <li>

    Copy/Paste the request, and run it with telnet (or a tool such as WFetch)

    </li> <li>

    Fiddle with the request headers and see if behavior changes.

    </li> </ul>

    (I'd suspect the site screens the request based on the User agent header or something similar)

    Answer5:

    The java.net package doesn't support lot of the needed features out of the box (like automatically saving and sending cookies). Use Apache's httpClient instead

Recommend

  • HTTP Error 404: Not Found when using wget to download a link
  • Jena Fuseki Server command not found
  • Configuring mod_proxy for Apache to reject wrong domain requests
  • Redirecting codeigniter project to https with htaccess: never complete
  • Create event for arrow keys in extjs
  • Link in Google search results is not redirected to expected page as per .htaccess
  • Any command to show how many applications are currently running under Tomcat?
  • How to Grab SSL Certificate in OpenSSL
  • Why is this jQuery reference '$(“”)' instead of '$(“”)'?
  • doing database write after the response
  • Name hiding in constructor initialization list
  • Executing a function that adds columns and populates them dependig on other columns in Pandas
  • Return list of all users in BOX Enterprise Account
  • Why does _GET in PHP wrongly decodes slash?
  • How do I get the post request with express js?
  • Excel VBA Intersect
  • IllegalArgumentException: The servlets named [HelloWorlds] and [Hello] are both mapped to the url-pa
  • How to get the index of element in the List in c#
  • Android NFC read Tags issue. Activity starts each time on data received
  • openssl handshake failed
  • How to resolve docker host names (/etc/hosts) in containers
  • Express JS Display Data By ID
  • how to set to NULL all the filestream varbinary(max) fields?
  • Ionic storage “get” returns null only on the second call within method
  • Double dispatch in Java example
  • How to implement Deep Linking in Roku SG application?
  • pip in virtualenv gets ConnectTimeoutError
  • Element.tagName for python not working
  • What does 'Language neutral' mean with regard to MAKELANGID?
  • Excel's Macro-Recorder usage
  • How to use carriage return with multiple line?
  • Record samples being played with OpenAL
  • Breeze - Deleted Items nav properties bug
  • Highlight one bar in a series in highcharts?
  • Android screen density dpi vs ppi
  • javaw.exe and eclipse startup problems
  • How would I use PHP exceptions to define a redirect?
  • How to extract text from Word files using C#?
  • Weird JavaScript statement, what does it mean?
  • Rails 2: use form_for to build a form covering multiple objects of the same class