Reference no: EM132369010

Data analytics for intrusion detection

Purpose of the assessment (with ULO Mapping) This assignment assesses the following Unit Learning Outcomes; students should be able to demonstrate their achievements in them.

c) Evaluate intelligent security solutions based on data analytics
d) Analyse and interpret results from descriptive and predictive data analysis

Assignment Overview

For this assignment, you will analyses and evaluate one of the publicly available Network Intrusion datasets given in Table 1.

Your task is to complete and make a research report based on the following: 1- Discuss all the attacks on your selected public intrusion dataset.

2- Perform intrusion detection using the available data analytic techniques using WEKA or other platforms.

3- In consultation with your lecturer, choose at least three data analytic techniques for network intrusion detection and prepare a technical report. In the report, evaluate the performance of data analytic techniques in intrusion detection using comparative analysis.

4- Recommend the security solution using the selected data analytic technique.

Follow the marking guide to prepare your report.

Dataset	Attacks
UNSW- NB15	analysis, backdoors, DoS, exploits, fuzzers, generic, reconnaissance, shellcode, worms
NSL- KDD	DoS, remote-to-local, user-to-root, probing
KDD CUP 99	DoS, remote-to-local, user-to-root, probing
CIC DoS	Application layer DoS attacks (executed through ddossim, Goldeneye, hulk, RUDY, Slowhttptest, Slowloris)

Table 1 Network Intrusion Dataset

Section 1: Data Analytic Tools and Techniques

In this section, your task is to complete and write a report on the following:

1. Install/deploy the data analytic platform of your choice (on Win8 VM on VirtualBox).

2. Demonstrate the use of at least two data analytic techniques (e.g. decision tree, clustering or other techniques) - you are free to use any sample testing data to demonstrate your skills and knowledge.

3. Lab demonstration: Must explain how each tool technique works in your lab prior to week 11. Data can be anything including Iris dataset.

Section 2: Evaluation of the Penetration Test (PT) of the given Dataset of UNSW in Table1

1. Select from UNSW example of the dataset, cvs, pcap and bro files to evaluate the result of the penetration test as explained below

2. For csv files you need to generate statics to identify the total number of attacks related to DOS, Exploits, generic, reconnaissance, shellcode, and worms and display the result in a graph and shows the percentage of attacks compared to normal traffic. (need to submit the excel csv file you analyzed with your report)

3. Use Wireshark to open the cap file and generate report with different statistics related to: Resolved address
DNS, http Packet length TCP Throughput

4. Use bro file and analyse results and write report on the type of traffic generated. Then, convert Bro Logs to Flows, where you can convert the Bro logs into IPFIX (using IPFIX utility) by defining your own elements and templates, then create bro report by filtering and thresholds to watch for specific events or patterns

Section 3: Data Analytic for Network Intrusion Detection (using Weka if possible)

Perform the following tasks and write a full report on your outcomes:

1. Convert the benchmark data suitable for the data analytic tools and platform of your choice. Explain the differences in the available data format for data analytics.

2. Select the features with rationale (external reference or your own reasoning).

3. Create training and testing data samples.

4. Evaluate and select the data analytic techniques for testing.

5. Classify the network intrusion given the sample data.

6. Evaluate the performance of intrusion detection using the available tools and technologies (e.g. confusion matrix).

7. Identify the limitation of overfitting.

8. Evaluate and analyse the use of ensemble tools.

9. Recommend the data analytic solution for the network intrusion detection.

10. Discuss future research work given time and resources