"cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer
Full error message
What I want to achieve
To scrape an website using AWS Lambda and save the data on S3.
The issues I'm having
When I execute Lambda, the following error message appears.
{ "errorMessage": "Unable to import module 'lambda_function': cannot
import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'
(/opt/python/urllib3/util/ssl_.py)", "errorType":
"Runtime.ImportModuleError", "requestId":
"fb66bea9-cbad-4bd3-bd4d-6125454e21be", "stackTrace": [] }
Code
The minimum Lambda code is as follows.
import requests
import boto3
def lambda_handler(event, context):
s3 = boto3.client('s3')
upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')
return event
An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
The bucket horserace-dx exists
The folder raw exists
The role of the Lambda is properly set. It can read from and write to S3
The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.
What I did so far
I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user
So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.Solutionsource: stackoverflow \u2197
Execute the following commands. pip install requests==2.25.0 -t ./python --no-user pip install beautifulsoup4 -t ./python --no-user pip install pytz -t ./python --no-user On PyPI, download the following whl files from the pages of numpy and pandas numpy-1.24.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl pandas-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl Unzip the files and move the contents to the python folder. Zip the python folder and upload it to AWS Lambda Layer. Set the layer to the Lambda. Then the code runs without errors.
API access
Get this solution programmatically \u2014 free, no authentication.
curl https://depscope.dev/api/error/6a2598acd3f7b139a1b60917fe64fe86c78a3f8505b0da5329526e8cd1b0b421hash \u00b7 6a2598acd3f7b139a1b60917fe64fe86c78a3f8505b0da5329526e8cd1b0b421