Skip to content

Commit

Permalink
Fixed mishandling of large buffers in BaseSocket.sendall()
Browse files Browse the repository at this point in the history
Fixes #38.
  • Loading branch information
agronholm committed Feb 2, 2019
1 parent 629ec0b commit c772edb
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 5 deletions.
10 changes: 5 additions & 5 deletions anyio/_networking.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,20 +143,20 @@ async def sendto(self, data: bytes, addr, *, flags: int = 0) -> int:
return self._raw_socket.sendto(data, flags, addr)

async def sendall(self, data: bytes, *, flags: int = 0) -> None:
to_send = len(data)
while to_send > 0:
offset = 0
total = len(data)
buffer = memoryview(data)
while offset < total:
await self._check_cancelled()
try:
sent = self._raw_socket.send(data, flags)
offset += self._raw_socket.send(buffer[offset:], flags)
except (BlockingIOError, ssl.SSLWantWriteError):
await self._wait_writable()
except ssl.SSLWantReadError:
await self._wait_readable()
except ssl.SSLEOFError:
self._raw_socket.close()
raise
else:
to_send -= sent

async def start_tls(self, context: ssl.SSLContext,
server_hostname: Optional[str] = None,
Expand Down
4 changes: 4 additions & 0 deletions docs/versionhistory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ Version history

This library adheres to `Semantic Versioning <http://semver.org/>`_.

**UNRELEASED**

- Fixed mishandling of large buffers by ``BaseSocket.sendall()``

**1.0.0b1**

- Initial release
17 changes: 17 additions & 0 deletions tests/test_networking.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,23 @@ async def server():

assert response == b'blahbleh'

@pytest.mark.anyio
async def test_send_large_buffer(self):
async def server():
async with await stream_server.accept() as stream:
await stream.send_all(buffer)

buffer = b'\xff' * 1024 # should exceed the maximum kernel send buffer size

This comment has been minimized.

Copy link
@njsmith

njsmith Feb 2, 2019

Collaborator

1024 bytes is nowhere near the maximum kernel send buffer size. On my laptop, a new socketpair accepts >200 KB in its first call to send. (And modern kernels like to dynamically resize buffers, so that another kernel might accept more or less.) You probably want, like, multiple megabytes here, and possibly to mess with SO_SNDBUF as well.

This comment has been minimized.

Copy link
@agronholm

agronholm Feb 2, 2019

Author Owner

The original test used 10 MB but I trimmed it down – too much. I intended to use 1 MB.

This comment has been minimized.

Copy link
@agronholm

agronholm Feb 2, 2019

Author Owner

Fixed.

async with create_task_group() as tg:
async with await create_tcp_server(interface='localhost') as stream_server:
await tg.spawn(server)
async with await connect_tcp('localhost', stream_server.port) as client:
response = await client.receive_exactly(len(buffer))
with pytest.raises(IncompleteRead):
await client.receive_exactly(1)

assert response == buffer

@pytest.mark.parametrize('method_name, params', [
('receive_until', [b'\n', 100]),
('receive_exactly', [5])
Expand Down

0 comments on commit c772edb

Please sign in to comment.