K-nearest neighbor for text classification, Computer Engineering

Assignment Help:

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 


Related Discussions:- K-nearest neighbor for text classification

What is the purpose of guard bits, What is the purpose of guard bits used i...

What is the purpose of guard bits used in floating point operations? The guard bits are the extra bits which are used to retain the in-between steps to enhance the accuracy in

Explain the fork-join, Explain the fork-join? fork-join: It is prim...

Explain the fork-join? fork-join: It is primitives in a programming language of higher level for implementing interacting processes. The syntax is as given below: fork

Illustrate the ftp commands, Q. Illustrate the FTP commands? FTP includ...

Q. Illustrate the FTP commands? FTP includes many commands however only few are used to retrieve a file. A user is required to understand the three basic commands to connect to

How is the connectivity established in verilog, How is the connectivity est...

How is the connectivity established in Verilog when connecting wires of different widths? When connecting wires or ports of different widths, connections are right-justified, S

Mathematical simulation and modelling applications, Mathematical Simulation...

Mathematical Simulation and Modelling Applications The tasks include mathematical simulation and modelling need lots of parallel processing. Three fundamental formalisms in mat

Requirements of decision support system, Q. Requirements of decision suppor...

Q. Requirements of decision support system? a) Fast computation A decision maker is able to perform a large number of computations very quickly and that too at a low cost

Replacement policy - cache memories , Replacement policy: On a particul...

Replacement policy: On a particular cache miss we require to evict a line to build room for the new line "In an A-way set associative cache, we have A option of which block

Define a console-program to make an array of integers, Define a console-pro...

Define a console-program to make an array of integers. Initialize the array and assign the values in array. Describe a method to pass the array and alter array elements. show array

Explain time slot interchange, Discuss the basic structure and principle of...

Discuss the basic structure and principle of operation of Time Slot Interchange (TSI) switch with the help of a neat diagram. Principle of time slot interchange  Time

Regular expression pattern in a wsdl document, Probelm: (a) Show the at...

Probelm: (a) Show the attributes used by Regular Expression Pattern in a WSDL document. (b) Describe the three standard wire formats for transmitting Web Service requests a

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd