Reference no: EM132501133

ECE 228 Machine Learning for Physical Applications - University of California

Classifiers and Autoencoding

In this Python programming homework we hope to provide you with an opportunity to become comfortable with linear, convolutional, dropout layers and general construction of a model in a deep learning framework. We make use of the Keras Tensorflow2 framework for its ease of use to those new to deep learning frameworks. This homework allows use of PyTorch with the requirement that the coder adheres to the layout of the notebook.

Meaning, substituting the PyTorch class defining a network is placed where the corresponding Tensorflow2 code is located. Please print the network after initialization.

For ease we would like that you answer the questions in markdown cells directly in the notebook. The questions are located at the top of the notebook so you do not have to search. Answers should be a short paragraph (except for 2.3) with any requested plots generated from the code below.

Overfitting

In this section you will construct a classifier for the Fashion MNIST dataset and see an example of model overfitting.

Deliverables: A working model according to the design specified, Answers to questions.

Dropout
The goal here is to provide one of a few possible improvements to deal with overfitting issues. Dropout is also useful as a "stress test" of your model. During training it randomly disconnects a percentage of connections between layers. This forces a network not to rely too heavily on any one connection, but to distribute information properly throughout the network.
Deliverables: A working model according to the design specified, Answers to questions.

Vanilla Autoencoder
Autoencoders are a simple but useful model. Similar to how a one-hot encoding transforms data into a useful form for a particular network. Autoencoders provide powerful nonlinear encoding of data by squeezing the data through a "bottleneck" called the latent space. Which in a sense filters the data down to a sparse basis or representation of the data. The goal is to find a smallest latent space which gives a highly accurate reconstruction of the data.

Deliverables: A working model according to the design specified and latent space representation.

Convolutional Autoencoder
A more powerful and complex form of autoencoder which can combine local information to create complex abstract features across 2D image patches. Using max pooling provides invariant features while only using convolution layers to reduce feature map size yields equivariant features. Deliverables: A working model according to the design specified and latent space representation.

Anomaly Detection
In this section you are given a dataset of vibration data coming from a machine pump. The data has 8 channels, one for each acoustic sensor location. There is background noise. Your goal is to construct an autoencoder that can detect the anomaly data. You are free to approach this as you see fit with the only stipulation being that you can outperform a baseline model by finding more than 75 of the 143 anomalies. The data was created using the torchaudio MelSpectrogram function and is a subset of a newer dataset developed for machine condition detection. This data can be used for class projects and the melspectrogram starter code can be used for anyone working with acoustic data.

Deliverables: A working model, histogram plot, mean and std. dev. of normal data, and confusion matrix for a 2 standard deviation threshold as results. Match your test/validation/training names to final given code to calculate true positives.

Describe your design decisions regarding your model. Choice of layers, layer sizing, choice of loss function, choice of nonlinearity, shape of data input and why, batch sizing, hyper parameters, etc.

Attachment:- Classifiers and Autoencoding.rar