Reference no: EM132487084
SIT742 Modern Data Science - Deakin University
ASSESSMENT TASK ONE
DATA EXPLORATION: DATA SCIENTISTS SURVEY
Background
In 2017, Kaggle (a data science community and competition platform) conducted a survey on a large range of users registered as the data scientist in their platform. The survey data are broadly covered the skill set of the data scientists, the demographic of the data scientists, the feedback of the platform and many other information.
Task Description
We provide one Jupyter notebook 2020SIT742Task1.ipynb at GitHub-SIT742, together with three data files at the data subfolder:
MCQResponses.csv The csv file contains participants' answers to multiple choice ques- tions. Each column contains the answers of one respondent to a specific question.
ConversionRates.csv Currency conversion rates to USD.
JobPostings.csv Data scientists job advertising in US with job descriptions, from JobPikr.
You are required to develop a data exploration report by completing the provided Jupyter notebook to finish some required analysis, with the exploration data analytics skills as well as visualization skills. Details requirements can be found in the provided notebook, and you need follow the notebook requirements to complete the coding and include the results into the report SIT742T1Report.pdf.
Data Exploration
For a data scientist, after obtaining the dataset, the first most crucial task is to obtain a good understanding of the data he or she is dealing with. This includes: examining the data attributes (or equivalently, data fields), seeing what they look like, what is the data type for each field, and from this information, determining suitable numerical/visual descriptions.
In this part of this assessment task, you need to complete the provided notebook coding parts and finish the required analysis in the attributes such as ‘education', ‘salary' and related demographic information (70%).
Text analysis
For the job advertisement data JobPostings.csv, you are required to write Python code to remove the stop-words, and to extract the high frequency words used in job advertisements.
advertisement information (30%). After that, you can do one self-defined text analysis task to get insight into those
ASSESSMENT TASK TWO DATA ANALYTICS: FIFA 2019
Background
Recently, Kaggle (a data science community and competition platform) released one data set FIFA19, which consists of 18K+ FIFA 19 player with around 90 attributes extracted from FIFA database. Here, we redistribute this data set for this assessment task:
2020T2Data.csv The file contains detailed information about each FIFA 19 player.
Task Description
We provide one Jupyter notebook 2020SIT742Task2.ipynb together with 2020T2Data.csv at the data subfolder.
You are required to analyse this dataset using Jupyter notebook with Spark packages including spark.sql and pyspark.ml.
FIFA19 Data Analytics
following 3 kinds of analysis : To systematically investigate this dataset, your Jupyter notebook should complete the
Part 1 - Exploratory Data Analysis data visualization and understanding.
Part 2 - Clustering Analysis Identify the inherent clusters among players, and for each cluster, identify its profile.
Part 3 - Classification Analysis Build classifiers to predict the ‘position_group' of the player. You are also required to evaluate the performance of at least 3 models using cross-validation.
Project Report
write a report SIT742T2Report.pdf with 1000 1500 words, which should include the Based on your implementation as required in Jupyter notebook, you are required to following information:
(1) The required report ‘Section 1' to ‘Section 3' (results and analysis) as specified in the notebook.
such as any rising star? any omni player? etc. (10%) (2) In the report's ‘Section 4', discuss any findings you can reveal from this data set,
(3) In the report's ‘Section 5', reflect the project group activities, such as the task during this project. (10%) distribution and contributions from each group members, and what you have learnt.
Attachment:- Modern Data Science.rar
Attachment:- Data.rar