Reference no: EM132741295 , Length: word count:2500
KF7032 Big Data and Cloud Computing - Northumbria University
Aims
The aim of this assignment is to introduce a practical application of Big Data and Cloud Computing using a realistic big data problem. Students will implement a solution using an industry leading Cloud computing provider together with the distributed processing environment Apache Spark. This will involve the selection of problem appropriate Machine Learning algorithms and methods.
Learning Outcome 1: Apply big data analytic algorithms, including those for visualization and cloud computing techniques to multi-terabyte datasets.
Learning Outcome 2: Critically assess data analytic and machine learning algorithms to identify those that satisfy given big data problem requirements
Learning Outcome 3: Critically evaluate and select appropriate big data analytic algorithms to solve a given problem, considering the processing time available and other aspects of the problem.
Learning Outcome 4: Design and develop advanced big data applications that integrate with third party cloud computing services Personal Values Attributes (Global / Cultural awareness, Ethics, Curiosity) (PVA):
Learning Outcome 5: Critically assess the relationship between knowledge and the ethical and social interpretation of primary research using big data.
Definitions
Portfolio Assignment: A collection of pieces of work
Individual Work: Work carried out by one person only
Group Work: Work carried out collaboratively seeking to improve each other's elements
Peer Review: Critical analysis and subsequent grading of a social equal's work
Semi-Formative: Training tasks assigned course credit to reward and ensure engagement.
Big Data Product: Weapons and Drugs
In the television documentary "Ross Kemp and the Armed Police" broadcast 6th September 2018 by ITV, multiple claims were made regarding violent crime in the UK.
These claims were:
1. Violent Crime is increasing
2. There are more firearms incidents per head in Birmingham than anywhere else in the UK
3. Crimes involving firearms are closely associated with drugs offences
In this assignment you will investigate these claims using real, publicly available data sets that will be made available to you and placed in Amazon S3. These include, but are not limited to:
1. Street Level Crime Data published by the UK Home Office. This dataset contains 19 million data rows giving a crime type, together with their location as a latitude and longitude.
2. Land Registry Price Paid Data: This gives the postcode of a property, the property type from an enumeration of D (Detached), S (Semi-Detached), T (Terraced), F (Flats/Maisonettes) and the price paid.
3. Postcode Data: This data set is based on material provided by the Ordinance Survey. It gives a latitude and longitude to every postcode. This is useful as it relates between the Land Registry Price Paid dataset postcode, and the original crime dataset
latitude/longitude.
Specifics
1. Process the data prepared for you using Apache Spark.
2. Filter the dataset so that crimes refer to relevant events only.
3. Using appropriate visualization methods, statistics, and machine learning, determine whether the claims made by Ross Kemp were true, false, or could not be determined.
4. Explain the reasoning behind your code so that it is clear what each block is intended to achieve, and why.
5. Report critically on the advantages, disadvantages, and limitations of the methods used.
6. Your submission will be a Jupyter Notebook containing both code (typically Python), and explanatory text (Markdown) limited to 2500 words (plus references).
Attachment:- Big Data and Cloud Computing.rar