The code reaches out to a message board and indexes/reports the top topics. Using WING IDE, it works fine and reports no errors. However, when ran via command prompt, it will error out saying it can't properly encode a character. This is the first time I've seen this and haven't found a good resource to fix it.

Being that it runs fine in WING, I'm unsure what else to add to the code that would prevent this issue from happening in the command prompt.

import requests

from bs4 import BeautifulSoup

url = raw_input("Enter the board URL: ")

print "\n"

#send the HTTP request
response = requests.get(url)

if response.status_code == 200:

    #pull the content
    html_content = response.content

    #send the page to BeautifulSoup
    html_doc = BeautifulSoup(html_content, "html.parser")

    #extract topic data
    topic_spider = html_doc.find_all("span",{"class":"subject"})
    data = []
    for topic in topic_spider:
        if topic.text!='':
    topiclist = list(dict.fromkeys(data))
    for item in topiclist:
        print ('[*] ' + item)

WING runs this just fine with no errors. Via CMD, the following result occurs after several successful results:

[*] Parenting (successful result)
Traceback (most recent call last):
  File "D:\xxxx\", line 29, in <module>
    print ('[*] ' + item)
  File "C:\Python27\lib\encodings\", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019' in position 31: character maps to <undefined>

3 Answers

ch4rl1e97 On Best Solutions

I note two things.

One, you use print statements like this

print ('[*] ' + item)

which indicates you're using python 3.x

Second, however, your cmd output uses python 2.7. That appears to be your problem. Try python3 on the commandline instead of python, as that is what it defaults to when you have both installed.

See if this solves it before anything else.

Trupti Shetty On

Make sure the python environment in CMD and Wing is the same. Set the environment variables that's in Wing IDE in CMD.

Kareem On

Looks like your code is written in python 3 but your default is set to python 2.

When running your code in CMD just add python3 rather than just python