Reference no: EM133027768

MITS6005 Big Data - Victorian Institute of Technology

Question 1
An insurance company implementeda Big Data solution to store and process data and applications. The company collects structured and unstructured data about their customers and products.

a) Discuss three phases of MapReduce framework to determine the number of customer claims from database.

b) Mention at least two disadvantages of MapReduce and discuss an alternative solution to replaceMapReduce in Hadoop ecosystem for the above problem.

Question 2
Australia is currently leading the global mining section. One of the Australian mining companies is specialized in the extraction and processing of minerals. The company employs over 70000 workers worldwide. A HR and payroll system is used to store the data about the employees and manage their salaries and leaves. One analytical system is used to process satellite images about the mining lands. IoT sensors are used to monitor for any defect in the mining machineries to avoid downtime and operational hazards. You are hired to lead the IT team at the company and identify the appropriate IT solutions.

a) Discuss the database solution that is required for this company based on the data types? Justify your answer.

b) Based on different sources of the data, discuss one big data analytics opportunity from the case study and identify one technology from Hadoop ecosystem can be used to support this process.

Question 3
Big Data is defined by at least three dimensions known as 3Vs: volume, variety, and velocity. Hadoop cluster is a computational cluster designed for storing and analyzing huge amounts of unstructured data in distributed computing environment. A healthcare center is using Hadoop platform to store and process the data which are related to doctors, staff, and patients.

a) Discuss the advantages of using Hadoop cluster in health data applications. Provide examples in your explanation.

b) HDFS is a storage system in Hadoop platform. The healthcare center has a Hadoop cluster, and there is a file about appointments of size 812 MB stored in HDFS (Hadoop 2.x) using default block size configuration. Calculate the number of blocks that needs to be generated for the given size and find each block's size to be stored in the Hadoop.

Question 4

Australian Bank reported an increase in the number of fraudulent credit card transaction recently. Several big data solutions are implemented to store and process the data including data marts and data lakes. However, it is still challenging to detect the fraudulent activities at the right time.

a) Discuss the appropriate Big Data technology that can be used to process the data transactions without latencies?

b) Discuss one application with examples where real-time data processing is required.

Attachment:- Big Data.rar