pypipython-3.x95% confidence\u2191 219

"asyncio.run() cannot be called from a running event loop" when using Jupyter Notebook

Full error message
I would like to use asyncio to get webpage html.

I run the following code in jupyter notebook:

import aiofiles
import aiohttp
from aiohttp import ClientSession

async def get_info(url, session):
    resp = await session.request(method="GET", url=url)
    resp.raise_for_status()
    html = await resp.text(encoding='GB18030')
    with open('test_asyncio.html', 'w', encoding='utf-8-sig') as f:
        f.write(html)
    return html
    
async def main(urls):
    async with ClientSession() as session:
        tasks = [get_info(url, session) for url in urls]
        return await asyncio.gather(*tasks)

if __name__ == "__main__":
    url = ['http://huanyuntianxiazh.fang.com/house/1010123799/housedetail.htm', 'http://zhaoshangyonghefu010.fang.com/house/1010126863/housedetail.htm']
    result = asyncio.run(main(url))

However, it returns RuntimeError: asyncio.run() cannot be called from a running event loop

What is the problem?

How to solve it?

The asyncio.run() documentation says: This function cannot be called when another asyncio event loop is running in the same thread. In your case, jupyter (IPython ≥ 7.0) is already running an event loop: You can now use async/await at the top level in the IPython terminal and in the notebook, it should — in most of the cases — “just work”. Update IPython to version 7+, IPykernel to version 5+, and you’re off to the races. Therefore you don't need to start the event loop yourself and can instead call await main(url) directly, even if your code lies outside any asynchronous function. Modern Jupyter lab/notebook Use the following for newer versions of Jupyter (IPython ≥ 7.0): async def main(): print(1) await main() Python or older IPython If you are using Python ≥ 3.7 or IPython < 7.0, use the following: import asyncio async def main(): print(1) asyncio.run(main()) That's also that form you should use if you are running this in a python REPL or in an independent script (a bot, a web scrapper, etc.). If you are using an older version of python (< 3.7), the API to run asynchronous code was a bit less elegant: import asyncio async def main(): print(1) loop = asyncio.get_event_loop() loop.run_until_complete(hello_world()) Using await in your code In your case, you can call await main(url) as follows: url = ['url1', 'url2'] result = await main(url) for text in result: pass # text contains your html (text) response This change to recent versions of IPython makes notebook code simpler and more intuitive for beginers. Further notices Few remarks that might help you in different use cases. Jupyter vs. IPython caution There is a slight difference on how Jupyter uses the loop compared to IPython. [...] IPykernel having a persistent asyncio loop running, while Terminal IPython starts and stops a loop for each code block. This can lead to unexpected issues. Google Colab In the past, Google colab required you to do more complex loop manipulations like presented in some other answers here. Now plain await main() should just work like in IPython ≥ 7.0 (tested on Colab version 2023/08/18). Python REPL You can also run the python REPL using the asyncio concurrent context. As explained in asyncio's documentation: $ python -m asyncio asyncio REPL ... Use "await" directly instead of "asyncio.run()". >>> import asyncio >>> await asyncio.sleep(10, result='hello') 'hello' The asyncio REPL should be available for python ≥ 3.8.1. When does asyncio.run matters and why? Older versions of IPython were running in a synchronous context, which is why calling asyncio.run was mandatory. The asyncio.run function allows to run asynchronous code from a synchronous context by doing the following: starts an event loop, runs the async function passed as argument in this (new) event loop, stops the event loop once the function returned In more technical terms (notice how the argument function is called a coroutine): This function runs the passed coroutine, taking care of managing the asyncio event loop, finalizing asynchronous generators, and closing the threadpool. What happen when using await in synchronous context? If you happen to use await in a synchronous context you would get the one of the following errors: SyntaxError: 'await' outside function SyntaxError: 'await' outside async function In that case that means you need to use asyncio.run(main()) instead of await main().

API access

Get this solution programmatically \u2014 free, no authentication.

curl https://depscope.dev/api/error/0bf9a3eb9843c54c2ac384fcb2f13bff0cb956235227ec48b79abffbfb904f75
hash \u00b7 0bf9a3eb9843c54c2ac384fcb2f13bff0cb956235227ec48b79abffbfb904f75
&quot;asyncio.run() cannot be called from a running event lo… — DepScope fix | DepScope