With the help of nltk.tokenize.TabTokenizer() method, we are able to extract the tokens from string of words on the basis of tabs between them by using tokenize.TabTokenizer() method.
Syntax :
tokenize.TabTokenizer()
Return : Return the tokens of words.
Example #1 :
In this example we can see that by using tokenize.TabTokenizer() method, we are able to extract the tokens from stream to words having tabs between them.
# import TabTokenizer() method from nltk from nltk.tokenize import TabTokenizer # Create a reference variable for Class TabTokenizer tk = TabTokenizer() # Create a string input gfg = "Geeksfor\tGeeks..\t.$$&* \nis\t for geeks" # Use tokenize method geek = tk.tokenize(gfg) print(geek) |
Output :
[‘Geeksfor’, ‘Geeks..’, ‘.$$&* \nis’, ‘ for geeks’]
Example #2 :
# import TabTokenizer() method from nltk from nltk.tokenize import TabTokenizer # Create a reference variable for Class TabTokenizer tk = TabTokenizer() # Create a string input gfg = "The price\t of burger \tin BurgerKing is Rs.36.\n" # Use tokenize method geek = tk.tokenize(gfg) print(geek) |
Output :
[‘The price’, ‘ of burger ‘, ‘in BurgerKing is Rs.36.\n’]
Recommended Posts:
- Python NLTK | nltk.tokenize.SpaceTokenizer()
- Python NLTK | nltk.tokenize.StanfordTokenizer()
- Python NLTK | nltk.tokenizer.word_tokenize()
- Python NLTK | nltk.TweetTokenizer()
- Python NLTK | nltk.tokenize.mwe()
- Python NLTK | nltk.WhitespaceTokenizer
- Python NLTK | nltk.tokenize.LineTokenizer
- Python NLTK | nltk.tokenize.SExprTokenizer()
- Python | NLTK nltk.tokenize.ConditionalFreqDist()
- Tokenize text using NLTK in python
- Removing stop words with NLTK in Python
- How to get synonyms/antonyms from NLTK WordNet in Python?
- Part of Speech Tagging with Stop words using NLTK in python
- Python | Stemming words with NLTK
- Python | Lemmatization with NLTK
- Python | Gender Identification by name using NLTK
- Python NLTK | tokenize.regexp()
- Python NLTK | tokenize.WordPunctTokenizer()
- Creating a Basic hardcoded ChatBot using Python-NLTK
- Important differences between Python 2.x and Python 3.x with examples
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.
