"Unclosed connection" warning, but close method was awaited

[artkom911]

🐞 Describe the bug
Despite context manager using, after some amount of new sessions, getting warning of Unclosed connection

💡 To Reproduce

import asyncio
import aiohttp


async def main(i=100):
    for j in range(i):
        async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(limit=200)) as session:
            await session.get('http://127.0.0.1:6000')
        print(j, session.closed)
    print('done {}'.format(i))


if __name__ == "__main__":
    asyncio.run(main())

just run script above

💡 Expected behavior
I understand that creating a new session each time is not a good idea, still, I'm not expecting any warnings in this case

📋 Logs/tracebacks

0 True
..
99 True
done 100

Unclosed connection
client_connection: Connection<ConnectionKey(host='127.0.0.1', port=6000, is_ssl=False, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
Unclosed connection
client_connection: Connection<ConnectionKey(host='127.0.0.1', port=6000, is_ssl=False, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
Unclosed connection
client_connection: Connection<ConnectionKey(host='127.0.0.1', port=6000, is_ssl=False, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
Unclosed connection
client_connection: Connection<ConnectionKey(host='127.0.0.1', port=6000, is_ssl=False, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

One more thing is that there may be situations when there are no warnings or their amount changes from time to time.

📋 Your version of the Python

$ python --version
Python 3.9.0

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Version: 3.7.2

$ python -m pip show multidict
Version: 5.0.2

$ python -m pip show yarl
Version: 1.6.3

📋 Additional context
client

[asvetlov]

This problem will be fixed by aiohttp 4.0 only, sorry.

[Sunda001]

uhm when will aiohttp 4.0 be released?

[kk7ds]

Any update on this? I'm suffering this problem as well. Attempting to close the session before exiting the context manager doesn't help, and I get other problems if I try to keep the session around longer:

https://stackoverflow.com/questions/54462271/what-does-cannot-write-to-closing-transport-mean/54463560

Is there some workaround I can use to squelch the constant warnings and/or when is aiohttp 4.0 coming?

[Dreamsorcerer]

Any update on this? I'm suffering this problem as well. Attempting to close the session before exiting the context manager doesn't help, and I get other problems if I try to keep the session around longer:

I suspect there is a better solution than creating a new session every time, though I'm not familiar with the issue.

Is there some workaround I can use to squelch the constant warnings and/or when is aiohttp 4.0 coming?

Ignore them? https://docs.python.org/3/library/warnings.html#warnings.filterwarnings

[kk7ds]

This bug is specifically that using one session per request does not solve the problem. My earlier reply indicates other problems if you go the other way and keep a session for multiple requests.

They're logs, not warnings, AFAIK so not easy to ignore. I found another reference yesterday to someone trying to filter them like that and failing.

FWIW, I think I've found that if I call close() on the response itself, it will alleviate the problem. i.e.:

async with aiohttp.ClientSession() as s:
    r = await s.get('http://...')
    .... do stuff
    r.close()

[Dreamsorcerer]

They're logs, not warnings, AFAIK so not easy to ignore. I found another reference yesterday to someone trying to filter them like that and failing.

Ah no, the code should create a warning, but it also calls the exception handler (which is what you see). So, just ignore that in the exception handler:

def exc_handler(loop, context):
    if context["message"] == "Unclosed connection":
        return
    loop.default_exception_handler(context)

async def main():
    asyncio.get_running_loop().set_exception_handler(exc_handler)

[kk7ds]

Sure, but that's a huge hammer (hack), it may conflict with other default handler overrides, and you lose the reporting of actual problems in that case. What I wanted was a way to appease whatever is getting upset at otherwise totally proper behavior. I believe closing the response does that, without sacrificing the reporting of actual problems.

[Dreamsorcerer]

it may conflict with other default handler overrides

There is only 1 exception handler for a loop.

and you lose the reporting of actual problems in that case.

Well, you could try play around with some kind of context manager that only suppresses these in particular places in your code.

What I wanted was a way to appease whatever is getting upset at otherwise totally proper behavior. I believe closing the response does that, without sacrificing the reporting of actual problems.

The proper solution is apparently in master, but can't be backported to 3.x. You asked for a workaround to suppress the warnings, so that's what I gave you. If there was a clean, precise way to fix it in 3.x, it would already have been implemented in aiohttp.

[braveltd]

This problem will be fixed by aiohttp 4.0 only, sorry.

That's not good ((( 📛