Build a noisy channel spell correction model

Reference no: EM132273942

Assignment

Upload a Jupyter notebook with solutions and results for each of the following tasks. So 1 Jupyter notebook with well separated out code, solutions/ results and explanations for each of the below are expected.

1. Build unigram, bigram and trigram language models for English language using text 7 in nltk.book. Add Laplace smoothing (add-1 and add-k); Try Unknown handling; Use Stupid back-off; Build in simple and conditional Interpolation. Compute the perplexities of all these variations of the language model. Evaluate all these models against text 5, text 8 and text 9 in nltk.book. Summarize and explain your results. Do use appropriate pruning for all your models.

2. Build a noisy channel spell correction model for real and non-real words for English language, using text7 in nltk.book and using Damerau Levenshtein distance. Using that identify incorrect spellings in text5 and text9. Train, test and tune your model. Summarize and explain your results.

3. Build a Naïve Bayes model to identify incorrect spellings using the data below. Tune and evaluate the model. Run it against text5 and text9. Compare it to the noisy channel spell corrector built in the previous problem. Explain your results.

Attachment:- Assignment.rar

Reference no: EM132273942

Questions Cloud

Identify the behaviors and actions you might see in boards : Considering the increased focus on values such as sustainability, environmental rights, human rights, and social responsibility, identify the behaviors.

The security dilemma produces at the very least what outcome : Which of the following is an international system suggested by Henry Kissinger, in which all states accept other states' right to exist?

What important values are absent in your opinion : Find an organization with a mission and/or vision statement. What values do you see reflected in the statement? What important values are absent in your opinion

Identify your top three terminal and instrumental values : Utilize the "Rokeach Values Survey" to help you determine your values hierarchy, which will assist you in the development of your personal model of leadership.

Build a noisy channel spell correction model : Build a noisy channel spell correction model for real and non-real words for English language, using text7 in nltk.book and using Damerau Levenshtein distance

Discuss the six qualities that relate to presidential job : Briefly discuss the six qualities that relate to presidential job performance. What qualities are most important to a successful presidency?

Is there one best government system in the world : Is there "one best" government system in the world? If you believe there is, what would it be? If there isn't, how do we coexist under different government.

Is this tenet of buddhism compatible with the western notion : According to Buddhism, "suffering is caused by the desire to hold onto things". Therefore, the only way to alleviate suffering is to not form an attachment.

Do you think nazism can happen again : There can be no rational , humane defense of Nazism so how did Nazism dominate Germany for 12 years? Do you think Nazism can happen again? Why or why not?

User Account

All Pages