{"id":1072,"hash":"cbed6aedef62fe95c19a9b117e59d3b7a1d7a562b81eed2a81ceb07d3ff87966","pattern":"UnicodeEncodeError: &#39;ascii&#39; codec can&#39;t encode character u&#39;\\xa0&#39; in position 20: ordinal not in range(128)","full_message":"I'm having problems dealing with unicode characters from text fetched from different web pages (on different sites). I am using BeautifulSoup. \n\nThe problem is that the error is not always reproducible; it sometimes works with some pages, and sometimes, it barfs by throwing a UnicodeEncodeError. I have tried just about everything I can think of, and yet I have not found anything that works consistently without throwing some kind of Unicode-related error.\n\nOne of the sections of code that is causing problems is shown below:\n\nagent_telno = agent.find('div', 'agent_contact_number')\nagent_telno = '' if agent_telno is None else agent_telno.contents[0]\np.agent_info = str(agent_contact + ' ' + agent_telno).strip()\n\nHere is a stack trace produced on SOME strings when the snippet above is run:\n\nTraceback (most recent call last):\n  File \"foobar.py\", line 792, in <module>\n    p.agent_info = str(agent_contact + ' ' + agent_telno).strip()\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\xa0' in position 20: ordinal not in range(128)\n\nI suspect that this is because some pages (or more specifically, pages from some of the sites) may be encoded, whilst others may be unencoded. All the sites are based in the UK and provide data meant for UK consumption - so there are no issues relating to internalization or dealing with text written in anything other than English.\n\nDoes anyone have any ideas as to how to solve this so that I can CONSISTENTLY fix this problem?","ecosystem":"pypi","package_name":"unicode","package_version":null,"solution":"Read the Python Unicode HOWTO. This error is the very first example.\n\nDo not use str() to convert from unicode to encoded text / bytes.\n\nInstead, use .encode() to encode the string:\n\np.agent_info = u' '.join((agent_contact, agent_telno)).encode('utf-8').strip()\n\nor work entirely in unicode.","confidence":0.95,"source":"stackoverflow","source_url":"https://stackoverflow.com/questions/9942594/unicodeencodeerror-ascii-codec-cant-encode-character-u-xa0-in-position-20","votes":1519,"created_at":"2026-04-19T04:52:18.508278+00:00","updated_at":"2026-04-19T04:52:18.508278+00:00"}