Skip to content

*: support compiling tidb to wasm - #13069

Closed
lucklove wants to merge 18 commits into
masterfrom
feature/wasm
Closed

lucklove wants to merge 18 commits into
masterfrom
feature/wasm

Conversation

@lucklove

@lucklove lucklove commented Nov 1, 2019

Copy link
Copy Markdown
Contributor

Signed-off-by: lucklove gnu.crazier@gmail.com

What problem does this PR solve?

This PR make it possible to compile tidb to wasm and use tidb in browser.

What is changed and how it works?

  • add a term to interact with tidb core directly instead of mysql protol
  • add wasm platform support for some low-level operation source file
  • add a web interface to interact with term
  • add polyfill for golang's missing fs operation in wasm

Check List

Tests

  • Manual test

Code changes

Side effects

Related changes

Release note

@lucklove
lucklove requested a review from a team as a code owner November 1, 2019 08:31
@ghost
ghost requested review from SunRunAway and wshwsh12 and removed request for a team November 1, 2019 08:31
@lucklove lucklove changed the title feature: support compile to wasm feature: support compile to wasm (WIP) Nov 1, 2019
Signed-off-by: lucklove <gnu.crazier@gmail.com>
@lucklove

lucklove commented Nov 1, 2019

Copy link
Copy Markdown
Contributor Author

This PR works now, but need a little work to do until I remove "WIP".

cd wasm
GOOS=js GOARCH=wasm go build -o main.css
python -m SimpleHTTPServer 8000

Comment thread util/mathutil/mathutil_js.go Outdated
const MaxInt = int(MaxUint >> 1)
const MinInt = -MaxInt - 1

func MaxUint64(x, y uint64) uint64 {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add comments for the export functions and vars

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, thanks for reminding

@zz-jason zz-jason changed the title feature: support compile to wasm (WIP) *: support compiling tidb to wasm (WIP) Nov 4, 2019
Signed-off-by: lucklove <gnu.crazier@gmail.com>
Comment thread go.mod Outdated
@codecov

codecov Bot commented Nov 6, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #13069 into master will increase coverage by 0.0008%.
The diff coverage is 0%.

@@               Coverage Diff                @@
##             master     #13069        +/-   ##
================================================
+ Coverage   80.1771%   80.1779%   +0.0008%     
================================================
  Files           483        482         -1     
  Lines        121516     120911       -605     
================================================
- Hits          97428      96944       -484     
+ Misses        16330      16234        -96     
+ Partials       7758       7733        -25

Signed-off-by: lucklove <gnu.crazier@gmail.com>
@syrusakbary

Copy link
Copy Markdown

GOOS=js GOARCH=wasm go build -o main.css

I just inspected https://play.tidb.io and was a bit weird seeing a 13Mb css file main.css.
When inspecting, it was actually a wasm file: https://play.tidb.io/main.css

Any reason on naming it main.css rather than main.wasm ?

Comment thread Makefile
Comment thread Makefile Outdated
Comment thread .gitignore
Comment thread wasm/index.html.tpl

@Deardrops Deardrops left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please declare the minimum browser support version for running TiDB wasm in PR description.
If needed, add some check logic when load the index.html page, prompt some information about unsupported browser.

Comment thread wasm/index.html.tpl Outdated
Comment thread wasm/index.html.tpl Outdated
@lucklove

lucklove commented Nov 7, 2019

Copy link
Copy Markdown
Contributor Author

@syrusakbary thanks for comment.
We use .css instead of .wasm because many CDNs don't compress .wasm file automatically and unfortunately our CDN is one of them. In fact, the size of main.css is not 13MB, it's about 80MB, after compression, it's 13MB.

@syrusakbary

syrusakbary commented Nov 7, 2019 •

Copy link
Copy Markdown

We use .css instead of .wasm because many CDNs don't compress .wasm file automatically and unfortunately our CDN is one of them. In fact, the size of main.css is not 13MB, it's about 80MB, after compression, it's 13MB.

Oh no! Thanks for the info :)
Note: It might be worth to add that in a comment somewhere in the code (in the Makefile?) for future readers

@lucklove

lucklove commented Nov 7, 2019

Copy link
Copy Markdown
Contributor Author

@syrusakbary thanks for suggestion. I'll add a comment to makefile

@lucklove

lucklove commented Nov 7, 2019

Copy link
Copy Markdown
Contributor Author

known issue: golang1.13 can't produce large wasm correctly
golang/go#34395

Signed-off-by: lucklove <gnu.crazier@gmail.com>
Signed-off-by: lucklove <gnu.crazier@gmail.com>
5kbpers and others added 3 commits November 28, 2019 16:35
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: lucklove <gnu.crazier@gmail.com>
Signed-off-by: lucklove <gnu.crazier@gmail.com>
@lucklove
lucklove requested a review from a team as a code owner December 3, 2019 05:57
@ghost
ghost requested review from alivxxx and francis0407 and removed request for a team December 3, 2019 05:57
@alivxxx
alivxxx removed their request for review December 3, 2019 09:44
@francis0407
francis0407 removed their request for review December 3, 2019 10:20
5kbpers added 2 commits December 10, 2019 21:21
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
@lucklove lucklove changed the title *: support compiling tidb to wasm (can review, but don't merge) *: support compiling tidb to wasm (can review) Dec 11, 2019
@5kbpers 5kbpers changed the title *: support compiling tidb to wasm (can review) *: support compiling tidb to wasm Dec 12, 2019
5kbpers added 5 commits December 13, 2019 10:42
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
@5kbpers

5kbpers commented Dec 13, 2019

Copy link
Copy Markdown
Contributor

This PR is ready to merge, PTAL, thx! @zz-jason @siddontang @ngaut

Comment thread go.mod

replace github.com/pingcap/check => github.com/tiancaiamao/check v0.0.0-20191119042138-8e73d07b629d

replace github.com/coreos/go-systemd => github.com/5kbpers/go-systemd v0.0.0-20191209150347-994f05092cc6

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: here we forked go-systemd for building WASM.

Signed-off-by: lucklove <gnu.crazier@gmail.com>
@lucklove

lucklove commented Dec 25, 2019 •

Copy link
Copy Markdown
Contributor Author

We have fixed TiDB's Wasm compatibility problem and moved this feature to a standalone repo, closing this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants