How many sentences are generated after tokenization

Assignment Help Programming Languages

Reference no: EM131666074

Assignment

Please follow the steps and answer the questions:

Load the necessary packages first:
library(tm)
library(openNLP)
library(textstem)

1. Import the twitter dataset using readreadline() function.

2. Inspect the first 10 tweets from the data.

3. We will specifically look at a tweet at index number 8. Assign a variable name to this tweet. For example: tweet8.

4. Check the data type of tweet8. Before changingthe tweet8 to String type, firstly allow tweet8 to go through stemming. How many words in this sentence has been stemmed? What are the original form and base form respectively?

5. Remove the stop words from stemmed tweet8. Compare the original tweet8 and transformed tweet8. What are the stop words removed?

6. Reassign the 8th tweet to tweet8. Lemmatize the tweet8. What are the words has been lemmatized? What are the original forms and base forms respectively?

7. Change tweet8 to String type.

8. Use sentence tokenization function to segment tweet8. How many sentences are generated after tokenization?

9. Use word tokenization function to divide the words from the sentences. How many words have been generated? Display the words and sentences.

10. Use part of speech tagging function to assign POS tag to each word. Check the word and POS frequency. How many words have been assigned POS tags "VBD" (verb past tense)? What are the words being assigned with POS tags "VBD"?

11. Use name entity recognition function to detect name entities from this tweet. Does this function detect any name entities?

12. Use parsing function to parse this tweet. How many verb phrases (VP) are there? What components compose the last verb phrase? (If your parser does not work, you could skip this question)

Submission:

Create a R script file and write the R commands for each question. Write down the answers to the questions such as "How many words in this sentence has been stemmed?" as R script comments in the same R script file. Submit the file on blackboard.

Attachment:- Lab-Tweets.rar

Reference no: EM131666074

Questions Cloud

Discuss computer was correctly figured for the program : Seller alleges that it was Buyer's responsibility to check whether his computer was correctly figured for the program

How is plastic made into a sustainable product in panama : How is plastic made into a sustainable product in Panama? What companies have utilized it to create either cleaner water,

Marriott and starwood hotels-resorts worldwide merger : Read this article and then find an article about the Marriott and Starwood Hotels & Resorts Worldwide merger.

Write a report on Exporting Wine to the UK : Write a report on given topic. Topic - Exporting Wine to the UK. Filter 4 - Target Markets and Corporate Factors. Likely Target Market in the Allocated Country

How many sentences are generated after tokenization : Use sentence tokenization function to segment tweet8. How many sentences are generated after tokenization? How many words have been generated?

Determining the non-profit organization : Which of these do you think are the most vital or most useful in a non-profit organization? Why?

Calculate customer-level operating income for customers : All customers except E ordered units in the same order size. Customer E's order quantity varied, so E got a discount part of the time but not all the time.

What role do the international monetary fund : What role do the International Monetary Fund (IMF) and the World Bank play? And what are some of the potential benefits or harms of these organizations?

Economic order quantity if price is not consideration : M. P. VanOyen Manufacturing has gone out on bid for a regulator component. What is the economic order quantity if price is not a? consideration?

User Account

All Pages