Reference no: EM132721542

CM50267 Software Technologies for Data Science

Project: Traffic Recording Application

Introduction

In this coursework you will undertake the implementation of a simple traffic capturing application. The application is a web-based system that consists of two parts. The web server written in python and a browser based client that presents an interface.

You are provided with:
1. A complete browser based front-end for the app that makes use of HTML, CSS and Javascript. You will not need to modify this code. Or to understand how it operates, other than how it interacts with the server. This is described in the section Web Application Structure and will be covered in a lecture.

2. A skeleton backend framework for the app written in Python that provides the core web server functionality. You are required to:

1. Create a suitable sqlite3 database to hold all the required data.

2. Extend the skeleton code to complete the app functionality by adding code as required. Where it is expected you will extend/modify the code is indicated by the use of ‘##' in comments. You may alter the code elsewhere if you wish.

3. Create two additional programs able to extract summarised data from the database as CSV files. Along with two support programs to allow these to be tested.

The coursework has been broken into a series of tasks. Marks have been allocated to each task. This document describes the behaviour required of the app and the support programs. Tasks will be assessed against this specification. Partial marks may be awarded for functionality present in incomplete tasks. You will use python for program development and using SQL to interact with an sqlite database. You should write the SQL queries yourself and not rely on a library such as pandas to abstract it for you.

Tasks
The marks allocated to each task are shown. The task description provides the expected behaviour the software must have to achieve those marks. Partial credit within a task may be awarded.

Task 1 Setup
Create an SQL database using sqlite that will be used to hold all data needed by the application. No data should be hardcoded into the application or held between server requests within the application. The sole exception to this is the vehicle type and occupancy level definitions. You are advised to consider all the tasks in designing the database as later tasks may require columns in tables initially created for the earlier tasks and for these to be filled in as part of the behaviour of the earlier task.

Task 2 Parameter Handling
Validating that all parts of an incoming request are present is an important first step in a web application backend. For example, currently if a user leaves the username or password fields blank, an error will occur within the Python program as the parameter variable in which the incoming values are stored is not populated when the input is blank. For this task you will need to add appropriate handling for this case. As an example of what is required, the parameters[‘command'] handling already deals with the same issue.

You also need to defend against deliberate or accidentally bad inputs. What is known as an ‘injection' attack. This applies to all of the tasks and you risk not getting all the marks for task 3 thru 7 if they do not defend themselves against bad inputs. Note that the scripted tests allow input to be generated that is not in the form the front-end client may usually generate. This is part of what you need to defend against.

Task 3 User Login
In the skeleton code, only one user ‘test' is supported and they can login with any non-empty password. This is dealt with in handle_login_request(). Add an appropriate table in the database that includes the entries: username and password. Both should be strings. The password should be hashed for security rather than stored in plaintext. You may find hashlib a useful python module for this task. Pre-populate the database with 10 users (test1, test2,...) with corresponding passwords (password1,password2,...). Allow a user to be logged in only once at any given time. An attempt to login while already logged in should be rejected with a suitable error message. A magic session token should be generated that will be passed to the client via cookies and will be returned with each /action request. This will be used to validate access to other actions. A reminder that in this and other tasks you should defend against malicious inputs. Multiple users may be logged in at once.

Task 4 User Logout
When a user chooses to log out, the session should be ended. A record should be kept in the database of the start and end times of each user session. Extend handle_logout_request() to support this. This action should result in the user being re-directed to the login page.

Task 5 Traffic Adding
Extend the handle_add_request() function so that it records the vehicle in the database. The table used should include entries for the location, type, occupancy and time of the recording as a minimum. It should also be possible to identify which user created the record and which other records they created during the same login session. The response must include an update of ‘message' and ‘total', even if the input is invalid or a user is not logged in.

Task 6 Traffic Correction

Extend the handle_undo_request() function so that it records the need to undo the vehicle in the database. The table used should include entries for the location, type, occupancy and time of the recording. It should also be possible to identify which user created the record and which other records they created during the same session. You can choose to have add and undo records in the same or different tables. Undo entries should not cause add entries to be removed only prevent them being counted in the statistics generated. Vehicles must exist to be undone. The response must include an update of ‘message' and ‘total' even if the input is invalid or a user is not logged in.

Task 7 Online Summary
Extend the handle_summary_request() function so that it returns the correct traffic statistics for the current session rather than the current sum_* placeholder value in the code.

Task 8 Offline Summary
Create a program (separate from the web app) called task8_out.py that is able to query the database and indicated the number of each type of vehicle and occupancy at each location during a period specified by a start date and time and an end date and time. Invalid dates and times should be intentionally rejected. The output should be delivered as a csv file with a row per location with the format as described in Figure 1.
Create a second program (separate from the web app) called task8_in.py that consumes a csv file containing entries of the form described in Figure 2 and updates your database to include these traffic observation records. The updated records should be correctly reflected when you run the first program of this task. You do not need to maintain user/session details beyond what is required to make this task possible for your given database schema.
They will not be extracted. You can, for example, assume that all entries were made by the same user in a single session.

Task 9 User Hours
Create a third python program (separate from the web app) called task9_out.py that is able to query the database and indicate the total hours each user was involved in counting in order to allow pay to be determined. It should allow a date to be provided in the form yyyymmdd and provide the totals for that date, the week ending on that date and the preceding month ending on that date. The output should be delivered as a csv file with a row per user of the format described in Figure 3.
Create a fourth python program (separate from the web app) called task9_in.py that is able to read a CSV file of the form described in Figure 4 and updates your database to reflect the user counting sessions specified.

Task 10 Code Quality
Lint tools are designed to assess the overall quality of code. pylint performs this task for the python language. It checks things like variable naming conventions, unused variables and various other practices that can lead to developing code that contains errors. Use the pylint program to assess the standard of your code. For each point above 4 that pylint gives you, you will receive a mark, up to a limit of 4 marks.
The test will be performed as part of lab 6. You will be asked to apply it to the server.py and the task 8 and 9 code, but only the score from server.py will be used to determine the mark.

Task 11 Report
Write a brief report that describes the application you have written. It should include:

Details the structure of your database.
A description of how your code manages the add and undo feature of the application. The errors and malicious input behaviour it seeks to prevent.
It should be no longer that four pages in length.

Lab 6 Review Exercise
Lab 6 is run in combination with the project. The purpose of this Lab is to give you experience in a phase of development that is frequently sacrificed to meet other priorities, testing and quality control.
When you submit your work to moodle, a copy will be made available to one of your fellow students. You will receive details of another students work. An ‘Application Acceptance Test' form will be released. You must complete the tasks on that form using the code you have received and then upload the completed form to the Lab 6 Assignment on moodle prior to the deadline for the Lab.
If you complete this process, you will receive full marks for the lab. If you fail to complete the tasks you will receive partial credit for the tasks you have completed. You should also upload a test report for your own code.
The testing results will be available to the member of staff that marks your work. But they are advisory only they will not map directly to your final mark for the project tasks.
If any student's code is not tested by another student, it will be tested by the unit leader/tutors. This will result in a delay to the marking but will not impact the mark achieved.
The tests will include:
1. Running pylint on the server code.
2. Running a test script of the style found in TrafficApp_Test.ipynb. The script will have more tests and will not be released until after the project deadline.
3. Running the in and then out program for task 8 using a supplied test file and checking against an expected output.
4. Running the in and then out program for task 9 using a supplied test file and checking against an expected output.

Attachment:- Traffic Recording Application.rar