Passing results from mechanize to BeautifulSoup

524 views Asked by At

I get an when i try to mix mechanize and BeautifulSoup in the following code:

from BeautifulSoup import BeautifulSoup
import urllib2
import re
import mechanize

br=mechanize.Browser()
br.set_handle_robots(True)
br.open('http://tel.search.ch/')
br.select_form(nr=0)
br.form["was"] = "siemens"
br.submit()
content = br.response
soup = BeautifulSoup(content)

for a in soup.findAll('a',href=True):
if re.findall('title', a['href']):
    print "URL:", a['href']

br.close()

The code from the beginning till br.submit() works fine with mechanize and the for loop with BeautifulSoup too. But I don't know how to pass the results from br.submit() into BeautifulSoup. The 2 lines:

content = br.response
soup = BeautifulSoup(content)

are apparently wrong. I get an error for soup = BeautifulSoup(content):

TypeError: expected string or buffer

Can anyone help?

1

There are 1 answers

0
C R On

Try changing

content = br.response

to

content = br.response().read()

In this way content now has html that can be passed to BeautifulSoup.