Repository navigation
Replace unix socket package - #430
Conversation
|
Botocore and the replacement for unixsocket have an unavoidable dependency conflict for Python 3.9. Options for resolving this are to drop support for Python 3.9, freeze requests/urllib/requests-unixsocket at their current version, or replace requests-unixsocket with something else |
|
I'd vote to drop support for Unix Sockets. Originally I was thinking that this might be a performance enhancement for clients and servers on the same host, but it's marginal if anything. Also, I think there can be problems when two clients are using the same socket connection. If we go this way, the ruanll.sh script should change the --no-docker option to use TCP rather than sockets. |
|
Actually we use unix sockets in hsds_app.py, and I prefer not to update the code to use tcp sockets. |
jreadey
left a comment
jreadey
left a comment
There was a problem hiding this comment.
Let's go with this. I'll submit a PR to remove Python 3.9 support
Nothing in the tree references either file - no workflow, script, or doc - and they contradict pyproject.toml on nearly every axis they share: numpy <2.0.0 against >=2.0.0, aiohttp ==3.10.2 against ==3.9.4, python_version 3.10 against requires-python >=3.11, and requests-unixsocket2 from GitLab, which HDFGroup#430 replaced with requests-unixsocket. They also omit h5json entirely. An environment built from them would not match what HSDS declares it needs, so they are worse than absent.
The cap dates to #363 (2024-05), where 'requests <= 2.31.0' accompanied 'urllib3 < 2.0.0' as the workaround for requests-unixsocket 0.3.x, which raised 'TypeError: HTTPConnection.request() got an unexpected keyword argument chunked' under urllib3 2.x. #430 removed the reason for it: replacing the socket package let urllib3 move to >= 2.4.0 - but the cap was raised to 2.32.4 rather than dropped, 2.32.4 being simply the current release that week. It has bounded the dependency ever since without a live constraint behind it. requests-unixsocket 0.4.1 asks only for requests >= 1.1, and a real request over a unix socket with requests 2.33.0 and urllib3 2.7.0 returns 200, so the pairing the cap protected is fine. Drop the bound and move the pinned requirement to 2.33.0 so CI exercises it - the socket tests are the ones that would notice a regression here.
The original package is incompatible with requests 2.32.0+.
Important
Replace deprecated
requests-unixsocketwithrequests-unixsocket2and update related dependencies for compatibility withrequests2.32.0+.requests-unixsocketwithrequests-unixsocket2inPipfile,pyproject.toml, andrequirements.txt.requestsversion constraint to<=2.32.4inPipfile,pyproject.toml, andrequirements.txt.urllib3version constraint to>=2.4.0,<3.0inPipfile,pyproject.toml, andrequirements.txt.aiobotocoreversion constraint to>=2.13.0inPipfile,pyproject.toml, andrequirements.txt.requestsversion in.github/workflows/python-package.yml.This description was created by
for c071fa4. You can customize this summary. It will automatically update as commits are pushed.