69309

Using the Mechanize gem with the Nokogirl gem?

Question:

I'm trying to scrape a website that requires authentication to get an element on a page with an id of #cellTotal.

Right now, using Mechanize I have logged into the page I want to access, but using basic Nokogiri functions like:

@selector = page.css("#cellTotal").text

Gives me this error:

undefined method `css' for #<Mechanize::Page:0x61234f8>

Here is what I have so far:

agent = Mechanize.new agent.get("example.com") agent.page.forms[0]["username_field"] = "username" agent.page.forms[0]["password_field"] = "password" agent.page.forms[0].submit @selector = agent.page.css("#cellTotal").text

How can I select an element on this page?

Answer1:

You can use page.parser to gain access to the underlying Nokogiri object.

<a href="http://mechanize.rubyforge.org/Mechanize/Page.html#method-i-parser" rel="nofollow">http://mechanize.rubyforge.org/Mechanize/Page.html#method-i-parser</a>

require 'mechanize' agent = Mechanize.new agent.get("http://stackoverflow.com/questions/23064821/using-the-mechanize-gem-with-the-nokogirl-gem/") agent.page.parser.class # => Nokogiri::HTML::Document agent.page.parser.css("#answer-23065003 .user-details a").text # => "akatakritos"

Recommend

  • How do I prevent a popup form from taking focus from the parent
  • Prevent OutOfMemoryException With AForge and Graphics.FromImage.DrawImage
  • Explicit instantiation declaration with extern
  • CSS “inherit” not working as expected with focus state
  • ASP.NET Session ending abruptly
  • Application.AddMessageFilter - how to read exactly what keys were pushed?
  • Compare variables PHP
  • Python Django Getting user input
  • C# - automatic delegate type from method
  • Loading and viewing images from imagelist
  • Replicating and differentiating portions of a form
  • Sum and Average of a series of numbers inputed to a text field
  • Spring MVC redirect with custom http headers
  • How to make the tableview response pan gesture in ZUUIRevealController
  • How to replace TouchesBegan with UIGestureRecognizer
  • how to remove a div with same ids but display='block' and display='none' in JAVa
  • Where can I get Microsoft.DirectX.dll?
  • Getting error 'Cannot read property 'document' of undefined' while importing exp
  • Jquery popup on mouse over of calendar control
  • how to upload multiple files in c# windows application
  • Adjust width of select element according to selected option's width
  • How to handle images sent by a mobile device?
  • Accessing IRQ description array within a module and displaying action names
  • Modifying destination and filename of gulp-svg-sprite
  • Javascript Callbacks with Object constructor
  • Where to put my custom functions in Wordpress?
  • Trying to switch camera back to front but getting exception
  • Free memory of cv::Mat loaded using FileStorage API
  • CSS Applying specific rule for a specific monitor resolution with only CSS is posible?
  • Hits per day in Google Big Query
  • Angular 2 constructor injection vs direct access
  • FormattedException instead of throw new Exception(string.Format(…)) in .NET
  • Change div Background jquery
  • Linking SubReports Without LinkChild/LinkMaster
  • XCode 8, some methods disappeared ? ex: layoutAttributesClass() -> AnyClass
  • Programmatically clearing map cache
  • costura.fody for a dll that references another dll
  • Observable and ngFor in Angular 2
  • UserPrincipal.Current returns apppool on IIS
  • java string with new operator and a literal