Skip to content

Speedup bulk transfer - #247

Merged
pahaz merged 2 commits into
pahaz:masterfrom
alex3d:speedup-bulk-transfer
Oct 28, 2022
Merged

pahaz merged 2 commits into
pahaz:masterfrom
alex3d:speedup-bulk-transfer

Conversation

@alex3d

@alex3d alex3d commented Oct 20, 2021

Copy link
Copy Markdown
Contributor

This PR would greatly speedup bulk transfer:

  • increase network buffer from 1kb to 16kb
  • do not hexlify all transferred data in case the TRACE log level is disabled

Without these patches (1gb.dat is 1 GByte of random data):

$ python3 ./sshtunnel.py -U user -P password -L :8081 -R 127.0.0.1:80 -p 22 host
$ curl http://127.0.0.1:8081/1gb.dat -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1024M  100 1024M    0     0  10.0M      0  0:01:41  0:01:41 --:--:-- 10.2M

With increased network buffers:

$ curl http://127.0.0.1:8081/1gb.dat -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1024M  100 1024M    0     0  32.7M      0  0:00:31  0:00:31 --:--:-- 32.4M

With increased buffers and hexlify disabled:

$ curl http://127.0.0.1:8081/1gb.dat -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1024M  100 1024M    0     0  59.3M      0  0:00:17  0:00:17 --:--:-- 62.1M

@pahaz
pahaz merged commit d2167a2 into pahaz:master Oct 28, 2022
@pahaz

pahaz commented May 14, 2023

Copy link
Copy Markdown
Owner

Welcome to the CONTRIBUTION list: aa7342e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants