Create algorithm for similarities between documents, Data Structure & Algorithms

Assignment Help:

Here is a diagram showing similarities between documents; this is an actual set of physics lab assignments from a large university.

2442_similarities between documents.png

Each node (square) in the graph is a document. Each edge (line) connects two documents based on the number of 6-word phrases they have in common. To reduce noise, only documents sharing more than 200 6-word phrases are shown. The red square is the lab manual, and the brown squares are two sample lab reports that were distributed in class. (Apparently many students 'borrowed' heavily from these documents.) But if we look carefully, we notice that several papers share a large number of phrases in common with each other and with no other documents. For example, a pair at the top left share 718, a pair at the top right show 882, and a pair on the lower left share 863 6-word phrases in common. It's likely that those people turned in essentially the same lab report or copied from each other. Your program will read through a directory full of files and identify pairs of files that share more than 200 6-word phrases, as a means of detecting copied work.

It's important to understand what we mean by "6-word phrases." It's not just looking at 6 words, then the next 6, etc.; after all, even plagiarists are smarter than that. A 6-word phrase is a word and the following 5 words, for each word with at least 5 words after it. So, for example, the text:

Now is the time for all good men to come to the aid of their country.

Contains the 6-word phrases:

Now is the time for all

is the time for all good

the time for all good men

time for all good men to

...and so on.

Thus, a single extended passage can generate many duplicates. On the other hand, a sentence that happens to begin with "Now is the time..." will be less likely to generate more than a few 'hits' as duplicate phrases. For our purposes, upper- or lower-case doesn't matter, nor does punctuation.


Related Discussions:- Create algorithm for similarities between documents

Comparisons between linear and binary search, Comparative Study of Linear a...

Comparative Study of Linear and Binary Search Binary search is lots quicker than linear search. Some comparisons are following: NUMBER OF ARRAY ELEMENTS EXAMINED array

Graph connectivity, A connected graph is a graph wherein path exists among ...

A connected graph is a graph wherein path exists among every pair of vertices. A strongly connected graph is a directed graph wherein every pair of distinct vertices is connecte

Terminology used for files structures, Given are the definitions of some im...

Given are the definitions of some important terms: 1) Field: This is an elementary data item characterized by its size, length and type. For instance, Name

What is ruby, What is Ruby Ruby has numerous simple types, including nu...

What is Ruby Ruby has numerous simple types, including numeric classes such as Integer, Fixnum, Bignum, Float, Big Decimal, Rational, and Complex, textual classes like String,

Kruskal algorithm for minimum spanning, Implementations of Kruskal's algori...

Implementations of Kruskal's algorithm for Minimum Spanning Tree. You are implementing Kruskal's algorithm here. Please implement the array-based Union-Find data structure.

Algorithmic implementation of multiple stacks, So far, we now have been con...

So far, we now have been concerned only with the representation of single stack. What happens while a data representation is required for several stacks? Let us consider an array X

Random searching, write a program that find,search&replace a text string

write a program that find,search&replace a text string

Data structure- tree, Tree is dynamic data structures. Trees can expand & c...

Tree is dynamic data structures. Trees can expand & contract as the program executes and are implemented via pointers. A tree deallocates memory whereas an element is deleted.

Explain depth-first traversal, Depth-first traversal A depth-first t...

Depth-first traversal A depth-first traversal of a tree visit a node and then recursively visits the subtrees of that node. Likewise, depth-first traversal of a graph visits

Linear array, representation of linear array

representation of linear array

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd