Asyncio RuntimeError: Event Loop is Closed
Full error message
I'm trying to make a bunch of requests (~1000) using Asyncio and the aiohttp library, but I am running into a problem that I can't find much info on.
When I run this code with 10 urls, it runs just fine. When I run it with 100+ urls, it breaks and gives me RuntimeError: Event loop is closed error.
import asyncio
import aiohttp
@asyncio.coroutine
def get_status(url):
code = '000'
try:
res = yield from asyncio.wait_for(aiohttp.request('GET', url), 4)
code = res.status
res.close()
except Exception as e:
print(e)
print(code)
if __name__ == "__main__":
urls = ['https://google.com/'] * 100
coros = [asyncio.Task(get_status(url)) for url in urls]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(coros))
loop.close()
The stack trace can be found here.
Any help or insight would be greatly appreciated as I've been banging my head over this for a few hours now. Obviously this would suggest that an event loop has been closed that should still be open, but I don't see how that is possible.Solutionsource: stackoverflow \u2197
You're right, loop.getaddrinfo uses a ThreadPoolExecutor to run socket.getaddrinfo in a thread. You're using asyncio.wait_for with a timeout, which means res = yield from asyncio.wait_for... will raise a asyncio.TimeoutError after 4 seconds. Then the get_status coroutines return None and the loop stops. If a job finishes after that, it will try to schedule a callback in the event loop and raises an exception since it is already closed.
API access
Get this solution programmatically \u2014 free, no authentication.
curl https://depscope.dev/api/error/a0619c3d8359744a54763b28ede2a15f34812ee7fcf439389b41ec3e461faf7fhash \u00b7 a0619c3d8359744a54763b28ede2a15f34812ee7fcf439389b41ec3e461faf7f