Reference no: EM133249898
Problem 1
Input Data File: dept_salary.txt (attached to this assignment)
The first column of the data file has the id of a dept and the second column a salary amount. Write a map-reduce program to get the MINIMUM salary per department. Please submit python mapper and reducer similar to ones given in lab along with detailed instructions on how to use.
Expected files to be submitted:
1. Mapper in python
2. Reducer in python
3. A file of instructions explaining how to use the mapper, reducer, and the data file
Rubric of the Grades:
- Correct Mapper code,
- Correct Reducer code,
- Executable MapReduce job,
- Producing correct output,
- Complete set of instructions showing all steps that needs to be run to produce a result (10 pts),
- Screen snapshot provided and matching to the execution of the submitted mapper and reducer codes
Problem 2
Input Data File: shakespeare_100.txt (download it from this link)
In the Word count example we output all words with their count. What if we only want the words with top 20 counts (This is very common problem in the industry). Think about using multiple map-reduce steps. Please submit python mapper and reducer similar to ones given in lab along with detailed instructions on how to use. If using multiple map-reduce steps, explanation of the steps should be included in your documentation
Expected files to be submitted:
1. Mapper(s) in python
2. Reducer(s) in python
3. A file of instructions explaining how to use the mapper(s), reducer(s), and the data file
Rubric of the Grades:
- Correct Mapper code(s). If multiple mappers the total grade will be split between them,
- Correct Reducer code(s). If multiple reducers the total grade will be split between them,
- Executable MapReduce job(s). If multiple execution steps the total grade will be split between them,
- Producing correct output(s). If multiple execution steps the total grade will be split between them,
- Complete set of instructions showing all steps that needs to be run to produce a result,
- Screen snapshot provided and matching to the execution(s) of the submitted mapper(s) and/or reducer(s) codes