Create algorithm for similarities between documents, Data Structure & Algorithms

Assignment Help:

Here is a diagram showing similarities between documents; this is an actual set of physics lab assignments from a large university.

2442_similarities between documents.png

Each node (square) in the graph is a document. Each edge (line) connects two documents based on the number of 6-word phrases they have in common. To reduce noise, only documents sharing more than 200 6-word phrases are shown. The red square is the lab manual, and the brown squares are two sample lab reports that were distributed in class. (Apparently many students 'borrowed' heavily from these documents.) But if we look carefully, we notice that several papers share a large number of phrases in common with each other and with no other documents. For example, a pair at the top left share 718, a pair at the top right show 882, and a pair on the lower left share 863 6-word phrases in common. It's likely that those people turned in essentially the same lab report or copied from each other. Your program will read through a directory full of files and identify pairs of files that share more than 200 6-word phrases, as a means of detecting copied work.

It's important to understand what we mean by "6-word phrases." It's not just looking at 6 words, then the next 6, etc.; after all, even plagiarists are smarter than that. A 6-word phrase is a word and the following 5 words, for each word with at least 5 words after it. So, for example, the text:

Now is the time for all good men to come to the aid of their country.

Contains the 6-word phrases:

Now is the time for all

is the time for all good

the time for all good men

time for all good men to

...and so on.

Thus, a single extended passage can generate many duplicates. On the other hand, a sentence that happens to begin with "Now is the time..." will be less likely to generate more than a few 'hits' as duplicate phrases. For our purposes, upper- or lower-case doesn't matter, nor does punctuation.


Related Discussions:- Create algorithm for similarities between documents

Omega notation, The ?-Notation (Lower Bound) This notation provides a l...

The ?-Notation (Lower Bound) This notation provides a lower bound for a function to within a constant factor. We write f(n) = ?(g(n)), if there are positive constants n 0 and

What is Polyphase Sort, One of the best known methods for external sorting ...

One of the best known methods for external sorting on tapes is the polyphase sort. Principle: The basic strategy of this sort is to distribute ordered initial runs of predetermi

Algorithm to merge the lists together, Q. Let X = (X1, X2, X3,....Xn) and Y...

Q. Let X = (X1, X2, X3,....Xn) and Y= (Y1, Y2, Y3,....Xm) be the two linked lists respectively. Write down an algorithm to merge the lists together to get the linked list Z such th

Algorithm for insertion into the circular queue, Algorithm for insertion of...

Algorithm for insertion of any element into the circular queue: Step-1: If "rear" of the queue is pointing at the last position then go to step-2 or else Step-3 Step-2: make

Recursive function, The location of a node in a binary search tree is defin...

The location of a node in a binary search tree is defined as a string such as LLRRL, which represents the node that you find by starting at the root, and traversing Left, traverse

HUFFMAN CODING, Ask question 1. Indicate whether each of the following prop...

Ask question 1. Indicate whether each of the following properties is true for every Huffman code. a. The codewords of the two least frequent symbols have the same length. b. The

Graph search using iterative deepening, Prove that uniform cost search and ...

Prove that uniform cost search and breadth- first search with constant steps are optimal when used with the Graph-Search algorithm (see Figure). Show a state space with varying ste

Algorithm for binary search, Q. Write down the algorithm for binary search....

Q. Write down the algorithm for binary search. Which are the conditions under which sequential search of a list is preferred over the binary search?

B-tree of minimum degree t can maximum pointers in a node, A B-tree of mini...

A B-tree of minimum degree t can maximum pointers in a node T pointers in a node.

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd