22523

HtmlUnit commenting out lines of facebook page

Question:

I am trying to simulate the login process to my facebook page using HtmlUnit (and I do have good reasons to do the same). Here is my java code for the same:

public static void main(String[] args) throws IOException { //tried to experiment with the browser types also. But to the same result //even using no param constructor does not help. WebClient webClient=new WebClient(BrowserVersion.CHROME); HtmlPage page1=webClient.getPage("https://www.facebook.com/bhramakarserver"); HtmlForm loginForm=(HtmlForm)page1.getElementById("login_form"); HtmlTextInput username=(HtmlTextInput)page1.getElementById("email"); HtmlPasswordInput password=(HtmlPasswordInput)page1.getElementById("pass"); username.setValueAttribute("myFbUsername"); password.setValueAttribute("myFbPassword"); HtmlElement button = (HtmlElement) page1.createElement("button"); button.setAttribute("type", "submit"); // append the button to the form loginForm.appendChild(button); page1=button.click(); //page1.executeJavaScript("window.scrollBy(0,6000)"); does not work System.out.println(page1.asXml()); HtmlSpan postContentSpan=(HtmlSpan)page1.getByXPath("//span[@class='userContent']").get(0); System.out.println(postContentSpan.asXml()); }

When I run this, I get the following error:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:604) at java.util.ArrayList.get(ArrayList.java:382) at com.rahulserver.fbhighlight.Main.main(Main.java:35) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

So clearly the pathogenic line is

HtmlSpan postContentSpan=(HtmlSpan)page1.getByXPath("//span[@class='userContent']").get(0);

The xpath is returning null. I posted <a href="https://stackoverflow.com/q/21331081/1291122" rel="nofollow">this</a> question related to it and go the answer that that the code containing the above xpath is commented out,hence is returning null.

So why is that happening and how do I make it work? As the page loads on scrolling down further,as is usual with facebook, I tried to simulate the process using

page1.executeJavaScript("window.scrollBy(0,6000)");

But yet it does not work and I get the same result. Here is the generated html file's pastebin link:<a href="http://pastebin.com/MfXsYSJQ" rel="nofollow">http://pastebin.com/MfXsYSJQ</a>.

I am sure that someone on SO would be able to come up with an out-of-the box answer to it...

.

Answer1:

The issue arise due to Browser you are using,a lso need to add the AJAX support and javascript wait.Change the Browser and need to add some more lines which are as below :

WebClient webClient=new WebClient(BrowserVersion.FIREFOX_3_6); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); webClient.getOptions().setThrowExceptionOnFailingStatusCode(false); webClient.getOptions().setThrowExceptionOnScriptError(false); webClient.waitForBackgroundJavaScript(50000);

The FireFox 3.6 is deprecated but it is better that however application runs.

Feel free to select as correct answer if it fulfill ur pblm.

Answer2:

The below code is running on my system.Please find the code

import com.gargoylesoftware.htmlunit.BrowserVersion; import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController; import com.gargoylesoftware.htmlunit.WebClient; import com.gargoylesoftware.htmlunit.html.HtmlElement; import com.gargoylesoftware.htmlunit.html.HtmlForm; import com.gargoylesoftware.htmlunit.html.HtmlPage; import com.gargoylesoftware.htmlunit.html.HtmlPasswordInput; import com.gargoylesoftware.htmlunit.html.HtmlSpan; import com.gargoylesoftware.htmlunit.html.HtmlTextInput; import java.io.IOException; public class App { public static void main(String[] args) throws IOException { WebClient webClient=new WebClient(BrowserVersion.FIREFOX_3_6); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); webClient.getOptions().setThrowExceptionOnFailingStatusCode(false); webClient.getOptions().setThrowExceptionOnScriptError(false); webClient.waitForBackgroundJavaScript(50000); HtmlPage page1=webClient.getPage("https://www.facebook.com/bhramakarserver"); HtmlForm loginForm=(HtmlForm)page1.getElementById("login_form"); HtmlTextInput username=(HtmlTextInput)page1.getElementById("email"); HtmlPasswordInput password=(HtmlPasswordInput)page1.getElementById("pass"); username.setValueAttribute("username"); password.setValueAttribute("password"); HtmlElement button = (HtmlElement) page1.createElement("button"); button.setAttribute("type", "submit"); // append the button to the form loginForm.appendChild(button); page1=button.click(); HtmlSpan postContentSpan=(HtmlSpan)page1.getByXPath("//span[@class='userContent']").get(0); System.out.println("The content is "+postContentSpan.asXml()); } }

Recommend

  • Scroll horizontalScrollView left-right
  • JavaScript - “Scroll Down” Button - Scroll From Landing Page to Top of the Header
  • scrollTop() not working in Chrome browser
  • Gaining access to the SVG DOM in Chrome and Safari through Javascript
  • Session management in GWT client side
  • Creating a layer of gradient within an SVG path dynamically
  • extjs4 catching the scroll event on panel
  • Aptana 3 remove bundle (jquery)
  • How to save a record and immediately use its GUID
  • How can I run DataNucleus Bytecode Enhancer from SBT?
  • Why am I getting a “405 Method not allowed” error on “Put” operations through a WCF Resful service?
  • quiver not drawing arrows just lots of blue, matlab
  • AppleScript : find open tab in safari by name and open it
  • How do I display a dialog that asks the user multi-choice questıon using tkInter?
  • Angular2 Response for preflight is invalid (redirect) from some GET requests
  • Ajax Upload File: $_FILES is empty but files exists in request header
  • Run multiple queries from 1 SQL file showing result in multiple tables
  • Button click event not firing in jQuery
  • wxPython: displaying multiple widgets in same frame
  • chrome.tabs.executeScript only fires when the Developer Console is open
  • JSON response opens as a file, but I can't access it with JavaScript
  • Alert pop up with LWUIT
  • FileReader+canvas image loading problem
  • Disabling Alt-F4 on a Win Forms NotifyIcon
  • req.body is undefined - nodejs
  • Why doesn't :active or :focus work on text links in webkit? (safari & chrome)
  • Change an a tag attribute in JavaScript based on screen width
  • jQuery tmpl and DataLink beta
  • VB.net deserialize, JSON Conversion from type 'Dictionary(Of String,Object)' to type '
  • How to delete a row from a dynamic generate table using jquery?
  • SQL merge duplicate rows and join values that are different
  • AngularJs get employee from factory
  • Proper way to use connect-multiparty with express.js?
  • How can I get HTML syntax highlighting in my editor for CakePHP?
  • How to set the response of a form post action to a iframe source?
  • How do I configure my settings file to work with unit tests?
  • How to CLICK on IE download dialog box i.e.(Open, Save, Save As…)
  • IndexOutOfRangeException on multidimensional array despite using GetLength check
  • Is there any way to bind data to data.frame by some index?
  • Binding checkboxes to object values in AngularJs