Reference no: EM133091411

Aim:

To develop a greater understanding of how to design and carry out research projects, by critiquing both the experimental design, data analysis and the ethical practices described in a series of hypothetical projects.

Exercise

Read the following scenarios describing hypothetical research projects.

For each scenario provide a short criticism of the research project. You should clearly identify at least four aspects of the project which you believe are incorrect or poor. These may relate to the research question, the experimental design, the analysis and reporting, and/or any ethical or academic misconduct issues. For each criticism which you identify, clearly describe the problem and state how it should have been addressed to improve the quality of the research.

Note: There will be one or more questions in a similar vein on the final exam, so make use of this opportunity to practice and learn how to perform this sort of critical assessment of a research project.

Scenario 1

Dr Jones is an academic in the School of Information Technology at Fictitious University, where he teaches the first-year Introduction to Programming course. He has developed a hypothesis that students who undertake pair programming will learn more than students who work by themselves on the assignment tasks. He obtained ethics clearance to test this hypothesis, and asked students in his class to nominate whether they wanted to work as a pair or individually.

At the end of semester, all students undertook the same exam but students who worked in pairs were given exam books with red covers and students who worked individually were given books with blue covers. Dr Jones marked the exams, and then analysed the results by calculating the mode for each group of students. He has published a paper in which he concludes "the mode of 64 for paired students is higher than the mode of 63 for unpaired students, and so we can conclude that working in pairs was beneficial to students' learning and should be the approach used for all students in future."

Scenario 2

The technology company Goober is developing autonomous (self-driving) cars, and wants to assess their safety. To do this for a month last year they had their cars drive around the suburban streets near their main campus, between the hours of 10am and 4pm each day. The cars were completely autonomous with no human in the vehicle either as a driver or passenger, and they automatically recorded the time spent driving, the distance travelled, the average speed, the number of accidents encountered, and the number of people (including pedestrians and people in other vehicles) injured in any accident.
Two weeks into the experiment an accident occurred. Goober discovered an error in their code which was corrected - as the error no longer existed the data for this accident was removed from their records.
During the experiment the cars also recorded video footage of other cars on the road. Goober used image-recognition software to extract details of license-plates from this footage, linked this to data gathered from people's Goober-owned social media pages, and called up the owners of these cars to ask them whether they had noticed the self-driving cars, and to ask their opinion about how well the cars drove.

Goober has compiled their results into a report which has been sent to the state government. In the report they state "Our self-driving vehicles drove an average of 4000 hours per accident. This is substantially lower than the average of 1 accident per 2000 hours reported across all traffic in the United States last year, thereby demonstrating the safety of our vehicles. In addition, when accidents occurred we observed a mean of only 0.9 injuries per accident, compared to the national average of 1.2 injured persons per accident. The overall results of these experiments provide a compelling case for removing all restrictions of the use of self-driving vehicles."

Scenario 3

The administration at Fictitious University are interested in examining the effect of two new email programs on staff productivity. Having applied for and being granted ethics approval, they emailed all university staff to ask for volunteers - the positive replies were compiled into a list as they arrived. The first 100 staff to reply were assigned to use the WowMail software, and the next 100 staff were assigned to use the ezMail software. This software was installed, and over the next month the time taken by staff to respond to emails was monitored.

At the end of the month these results were compiled into two datasets, corresponding to the two different software systems. At this point the staff in charge of the project discovered that the monitoring system had failed for 40 of the staff in the ezMail group. The staff decided to replace these missing results by randomly choosing 40 records from other people in the ezMail group and adding a small amount of noise to the results. This process was not described in their final report, which stated "The mean response time of staff using WowMail was 27 minutes, and the mean response time for staff using ezMail was 32 minutes. Therefore we can conclude that WowMail improves staff productivity, and we recommend that the University invest $5,000,000 in purchasing this software."