Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong type hint for asyncio.base_events.Server sockets attribute #5535

Closed
ramalho opened this issue May 26, 2021 · 13 comments · Fixed by #5538
Closed

Wrong type hint for asyncio.base_events.Server sockets attribute #5535

ramalho opened this issue May 26, 2021 · 13 comments · Fixed by #5538

Comments

@ramalho
Copy link
Contributor

ramalho commented May 26, 2021

The fix for issue #2364 ("AbstractServer" has no attribute "sockets") added this declaration:

class AbstractServer:
    sockets: Optional[List[socket]]

However, in the Python 3.9 implementation of asyncio.base_events.Server, sockets is a property defined like this:

    @property
    def sockets(self):
        if self._sockets is None:
             return ()
        return tuple(trsock.TransportSocket(s) for s in self._sockets)

So we have two problems:

  1. The value of sockets is never None, it's always a tuple.
  2. It is really a tuple[TransportSocket, ...].

There is example code in the asyncio docs and test code in the standard library that assume sockets is a sequence of sockets.

@ymyzk contributed the patch for #2364 noting "I noticed that this class has some changes in Python 3.7, so we need to update it again in the near future."

The documentation for asyncio.Server.sockets states:

Changed in version 3.7: Prior to Python 3.7 Server.sockets used to return an internal list of server sockets directly. In 3.7 a copy of that list is returned.

As I mentioned, the actual value returned is a tuple of zero or more asyncio.Server.TransportSocket objects.

@ramalho
Copy link
Contributor Author

ramalho commented May 26, 2021

I added type hints to the asyncio example titled TCP echo server using streams from the official documentation. Mypy wrongly complains that Value of type "Optional[List[socket]]" is not indexable pointing at line 23, which is:

    addr = server.sockets[0].getsockname()

@ramalho ramalho changed the title Wrong signature for asyncio.base_events.Server sockets Wrong type hint for asyncio.base_events.Server sockets attribute May 26, 2021
@srittau
Copy link
Collaborator

srittau commented May 26, 2021

Quick check:

  • 3.6.10: simple attribute, can be None
  • 3.7.10: property, returns list, never None
  • 3.8.3: property, returns tuple, never None

@srittau
Copy link
Collaborator

srittau commented May 26, 2021

Also, socket is defined on AbstractServer in typeshed, not on Server. On all checked versions, AbstractServer doesn't defined socket at all.

@ramalho
Copy link
Contributor Author

ramalho commented May 26, 2021

It's worth noting that asyncio.trsock.TransportSocket is explicitly not a subtype of socket.socket. Its docstring states:

All potentially disruptive operations (like "socket.close()") are banned.

srittau added a commit to srittau/typeshed that referenced this issue May 27, 2021
* Move socket from AbstractServer to Server.
* Fix Server.socket type on Python 3.7+.
* Use Iterable instead of list for socket argument.

Closes: python#5535
srittau added a commit that referenced this issue May 27, 2021
* Move socket from AbstractServer to Server.
* Fix Server.socket type on Python 3.7+.
* Use Iterable instead of list for socket argument.

Closes: #5535
@ramalho
Copy link
Contributor Author

ramalho commented May 27, 2021

Thank you @srittau for addressing this so quickly!

I saw you annotated the sockets attribute as Iterable.

However, both examples that I mentioned from the standard library use it as a Sequence like this:

    addr = server.sockets[0].getsockname()

The examples are:

@srittau
Copy link
Collaborator

srittau commented May 27, 2021

@ramalho The attributes/properties are annotated as list or Tuple. Only the argument on 3.7+ is annotated as Iterable and that should work, as the implementation uses list(self._sockets) or tuple(self._sockets).

@ramalho
Copy link
Contributor Author

ramalho commented May 27, 2021

I am sorry but don't understand your answer, @srittau. Yes, the annotation is compatible with the implementations, but the Iterable type is too general.

The documentation for asyncio.Server.sockets in Python 3.7 and later states that the attribute returns a list.

Changed in version 3.7: Prior to Python 3.7 Server.sockets used to return an internal list of server sockets directly. In 3.7 a copy of that list is returned.

Users expect to be able to index a list, and both examples from CPython itself (docs and tests) do this:

    addr = server.sockets[0].getsockname()

Therefore, the Iterable annotation will cause Mypy to flag user's code that uses sockets as a Sequence according to the Python ≥ 3.7 documentation and examples provided in the distribution.

@srittau
Copy link
Collaborator

srittau commented May 27, 2021

But the sockets attribute isn't annotated with Iterable. It's annotated with (depending on the version) Optional[list[_socket]], list[_socket], or Tuple[_socket, ...].

@layday
Copy link
Contributor

layday commented Jun 3, 2021

FYI, the code sample from the asyncio docs still does not type check because asyncio.create_server returns an instance of AbstractServer which does not implement sockets, and you can't even use an assertion prior to accessing sockets anymore.

@JelleZijlstra
Copy link
Member

assert isinstance(server, Server) should work, right?

Also, should we make create_server() return a Server instead of an AbstractServer in the stub?

@layday
Copy link
Contributor

layday commented Jun 3, 2021

assert isinstance(server, Server) should work, right?

That works but Server is not exposed in __init__ and would have to be imported from base_events. It's a bit much for a type-only assertion.

Also, should we make create_server() return a Server instead of an AbstractServer in the stub?

That'd fix it for the top-level start_server function. asyncio.get_{event,running}_loop() would have to return a BaseEventLoop for the loop's create_server method to return a Server as well.

@srittau
Copy link
Collaborator

srittau commented Jun 3, 2021

Cf. #5566. loop.create_server() is documented to return Server, not AbstractServer anyway.

@layday
Copy link
Contributor

layday commented Jun 3, 2021

Great, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants