K-nearest neighbor for text classification, Computer Engineering

Assignment Help:

Assignment 2: K-nearest neighbor for text classification.

The goal of text classification is to identify the topic for a piece of text (news article, web-blog, etc.). Text classification has obvious utility in the age of information overload, and it has become a popular turf for applying machine learning algorithms. In this project, you will have the opportunity to implement k-nearest neighbor and apply it to text classification on the well known Reuter news collection.

1.       Download the dataset from my website, which is created from the original collection and contains a training file, a test file, the topics, and the format for train/test.

2.       Implement the k-nearest neighbor algorithm for text classification. Your goal is to predict the topic for each news article in the test set. Try the following distance or similarity measures with their corresponding representations.

a.        Hamming distance: each document is represented as a boolean vector, where each bit represents whether the corresponding word appears in the document.

b.       Euclidean distance: each document is represented as a numeric vector, where each number represents how many times the corresponding word appears in the document (it could be zero).

c.         Cosine similarity with TF-IDF weights (a popular metric in information retrieval): each document is represented by a numeric vector as in (b). However, now each number is the TF-IDF weight for the corresponding word (as defined below). The similarity between two documents is the dot product of their corresponding vectors, divided by the product of their norms.

3.        Let w be a word, d be a document, and N(d,w) be the number of occurrences of w in d (i.e., the number in the vector in (b)). TF stands for term frequency, and TF(d,w)=N(d,w)/W(d), where W(d) is the total number of words in d. IDF stands for inverted document frequency, and IDF(d,w)=log(D/C(w)), where D is the total number of documents, and C(w) is the total number of documents that contains the word w; the base for the logarithm is irrelevant, you can use e or 2. The TF-IDF weight for w in d is TF(d,w)*IDF(d,w); this is the number you should put in the vector in (c). TF-IDF is a clever heuristic to take into account of the "information content" that each word conveys, so that frequent words like "the" is discounted and document-specific ones are amplified. You can find more details about it online or in standard IR text.

4.       You should try k = 1, k = 3 and k = 5 with each of the representations above. Notice that with a distance measure, the k-nearest neighborhoods are the ones with the smallest distance from the test point, whereas with a similarity measure, they are the ones with the highest similarity scores.

 

 


Related Discussions:- K-nearest neighbor for text classification

BCD ADDER, create a BCD adder combinational ckt. that adds 2 digit BCD inpu...

create a BCD adder combinational ckt. that adds 2 digit BCD inputs

Operations from events, Operations from events During analysis, events...

Operations from events During analysis, events which are sent to target objects. An operation on these object are presented as labels on transitions and should not be explicit

Real life decoding, We have not considered the time element in our decoding...

We have not considered the time element in our decoding. An important time for decoding is the time from the address strobe (AS) to when the data is required in a read.   Time

Is it possible to QuantifySoftware Engineering Risks using S, 1. Introducti...

1. Introduction to Software Metrics in Software Engineering.? 2. Role of direct and indirect measures in software process management? 3. Stance taken and justification? 4. Conclusi

Explain importance of different types of distributing frames, Explain impor...

Explain importance of the different types of distributing frames used in exchanges. Here MDF = main distribution frame; MF = main feeder; FP = feeder point; BF = branch f

Serial port communication, The 68HC11F1 has two types of serial ports namel...

The 68HC11F1 has two types of serial ports namely asynchronous (SCI) and synchronous (SPI). The most common form of communication device used in control is the SCI as this provides

What is open addressing, Open addressing:  The easiest way to resolve a co...

Open addressing:  The easiest way to resolve a collision is to begin with the hash address and do a sequential search by the table for an empty location. The idea is to place the

Explain how the different access methods work, Question: (a) Primary an...

Question: (a) Primary and secondary memory differs in their way they access data: (i) Mention the four generic access methods usually present in a computer system. (ii) E

Characteristics of large register file, Characteristics of large-register-f...

Characteristics of large-register-file and cache organizations Large Register File Cache Hold local variables for almost all functio

Storage devices, Logic manufactures have produced 'large scale' logic (LSI)...

Logic manufactures have produced 'large scale' logic (LSI) blocks to achieve complex functions; a typical set of logic functions is that of data storage. There are two main types o

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd