What I need to write is a code snippet that would go to a website e.g. www.google.com find the search box put in the phrase and retrieve HTML code of results page/pages. Is it possible to achieve this in Java?
Accessing website search box in Java
1.9k views Asked by BroPl AtThere are 5 answers
Most probably you should be able to pass a parameter to the url (have a look at the google url after issuing a search, there are plenty of parameters) or use a post request (if the site supports it, check for an API description).
If you read the URL directly from Java (e.g. using the URL
class) you'll get the returned HTMl as is.
The first tool I thought of was Selenium. It is primarily a web testing framework, but can be used to automate a browser for the kind of operation you're suggesting.
http://seleniumhq.org/docs/03_webdriver.html#getting-started-with-selenium-webdriver
e.g. www.google.com
For Google, use the JSON/Atom Custom Search API. It is the only (legal) way to access Google search.
HttpUnit can also be used. It's a well documented, open source and easy to use unit test framework.
Yes, use something like HttpClient, although there are other similar options.