Skip to content

Add Zipformer recipe for GigaSpeech - #1254

Merged
csukuangfj merged 16 commits into
k2-fsa:masterfrom
yfyeung:phone2
Oct 21, 2023
Merged

csukuangfj merged 16 commits into
k2-fsa:masterfrom
yfyeung:phone2

Conversation

@yfyeung

@yfyeung yfyeung commented Sep 14, 2023 •

Copy link
Copy Markdown
Collaborator

@yfyeung yfyeung changed the title Add GigaSpeech Zipformer Add Zipformer recipe for GigaSpeech Sep 15, 2023
@desh2608

desh2608 commented Oct 2, 2023

Copy link
Copy Markdown
Collaborator

@yfyeung do you have a pretrained model you could share?

@yfyeung

yfyeung commented Oct 17, 2023 •

Copy link
Copy Markdown
Collaborator Author

@yfyeung do you have a pretrained model you could share?

@desh2608 Sure, check https://huggingface.co/yfyeung/icefall-asr-gigaspeech-zipformer-2023-10-17

Comment thread README.md Outdated

We provide two models for this recipe: [Conformer CTC model][GigaSpeech_conformer_ctc]
and [Pruned stateless RNN-T: Conformer encoder + Embedding decoder + k2 pruned RNN-T loss][GigaSpeech_pruned_transducer_stateless2].
We provide three models for this recipe: [Zipformer]

@csukuangfj csukuangfj Oct 18, 2023 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a link for [Zipformer].
Screenshot 2023-10-18 at 09 39 40

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I removed it. It's redundant.

@csukuangfj csukuangfj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

The PR gives the state-of-the-art WER for gigapseech in icefall!

Left some minor comments.

@@ -0,0 +1,444 @@
# Copyright 2021 Piotr Żelasko

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to replace it with a symlnik?

@yfyeung yfyeung Oct 18, 2023 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have modified this file. In the past, GigaSpeech XL splits need to be merged. This one uses lhotse.mux.

@@ -0,0 +1,436 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

@@ -0,0 +1,775 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

@@ -0,0 +1,522 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed some comments like huggingface link in it.

@@ -0,0 +1,280 @@
#!/usr/bin/env python3
# Copyright 2021-2023 Xiaomi Corporation (Author: Fangjun Kuang, Zengwei Yao)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

@@ -0,0 +1,436 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

@@ -0,0 +1,273 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

@@ -0,0 +1,240 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you replace it with a symlink?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

- normal-scaled model, number of model parameters: 65549011, i.e., 65.55 M

You can find a pretrained model, training logs, decoding logs, and decoding results at:
<https://huggingface.co/yfyeung/icefall-asr-gigaspeech-zipformer-2023-10-17>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you upload the tensorboard log to
https://wandb.ai/site
and post a link to it here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, the link has not been posted. please leave a message when you think this PR is ready to merge.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I have posted that.

- normal-scaled model, number of model parameters: 65549011, i.e., 65.55 M

You can find a pretrained model, training logs, decoding logs, and decoding results at:
<https://huggingface.co/yfyeung/icefall-asr-gigaspeech-zipformer-2023-10-17>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a CI test for your model?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I have added.

@yfyeung
yfyeung requested a review from csukuangfj October 20, 2023 14:24
@yfyeung yfyeung added the zipformer For the upgraded zipformer recipes label Oct 20, 2023
@yfyeung yfyeung added zipformer For the upgraded zipformer recipes and removed zipformer For the upgraded zipformer recipes labels Oct 20, 2023
@yfyeung

yfyeung commented Oct 20, 2023

Copy link
Copy Markdown
Collaborator Author

Please re-review this PR. @csukuangfj

@csukuangfj csukuangfj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@csukuangfj
csukuangfj merged commit 416852e into k2-fsa:master Oct 21, 2023
@yfyeung
yfyeung deleted the phone2 branch October 21, 2023 12:46
@yuyun2000

Copy link
Copy Markdown

I want a brand new streaming English model

baileyeet pushed a commit to reazon-research/icefall that referenced this pull request Jul 16, 2025
Co-authored-by: Yifan Yang <yifanyeung@qq.com>
Co-authored-by: yfy62 <yfy62@d3-hpc-sjtu-test-005.cm.cluster>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

zipformer For the upgraded zipformer recipes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants