Reference no: EM132246394

Neural Network Lab - Supervised Learning

Perceptrons

We start by creating a perceptron with two input nodes and one output node by creating a 1x2 weight matrix, W (i.e. 1 output and 2 inputs) initialized to zero, a bias weight, b, initialized to zero, and a thresholding transfer function for the output unit:

>> W=zeros(1,2);
>> b=zeros(1,1);
>> net_inp=W*inp+b;
>> out=sign(sign(net_inp)+1);

If the net input to the output node is larger than 0, the output will be 1, otherwise 0. A bias weight can be thought of connecting an additional input node whose value is always 1 to the output node. The bias weight changes the point where this decision is made. For example, if the bias is -5, the net sum has to exceed 5 in order for the output to yield 1 (the bias of -5 is subtracted from the net sum. If it is less than 5, it will fall below 0).

Let's set the weights from the two input nodes to 1 and -1, respectively.

If the sum of the input times the weights meets or exceeds 0, the network should return 1, otherwise 0. Let's try it by feeding it some inputs:

inp1 = [1; 0.5] %Note: The first, positive weighted input larger than the second

inp2 = [0.5; 1] %Note: The second, negative weighted input is larger than the first

Your output should be 1 for inp1 and 0 for inp2. The network classified these inputs correctly. You can now feed the network a large number of random numbers and see if they are classified correctly, like this:
a = rand(2,10) %Create 20 random numbers, arranged as 2 rows, 10 columns

If the value in the first input row is larger than the value in the second input row (for a given column), the output value (for that column) should be 1, otherwise 0.

Exercise 1: Adjust the bias to some arbitrary value and see how the input/output mapping changes.

Exercise 2: Explore some points in the space and see what their output is. Is it about what you expect? Does the network make gross errors? What about the initial input vector?

Exercise 3: Use a linear neural network to classify the "good" and "bad" Greebles used in the previous neural network lab. Initially, set the weights to 1 and set the bias to zero.

Exercise 4: Solve the Greeble problem with a multilayer feedforward network using a sigmoid transfer function using the backpropagation learning rule. What are the results?

Exercise 6: What happens if you add two different frequencies of sine waves and play it?

Exercise 7: Play it again at a higher/lower sampling rate. What is happening?

Exercise 8: What is the person saying? Hint: The signal was sampled at 22050 Hz. You might want to take that into account when playing it. Look at the spectrogram, too.

Project

The project is to create a network that correctly classifies the gender of two target speech bites. In order to this, train the network with a total of 6 speech bites of both genders. Load these files (called Train_Kira1 to 3 and Train_Pascal1 to 3). Of course, the two speakers differ in all kinds of ways other than just gender (age, race, English as a first/second language, idiosyncratic speech characteristics, life style, etc.). If one were to face this problem in real life, one would have to train the network with a large number of speakers from both genders so that the network can abstract from all these irrelevant dimensions. But for our purposes, this will be fine. Specifically, you should do the following:

Create a network that reliably distinguishes the gender of the two target speech bites. Provide some evidence that this is the case and submit the source code.

Hints:
• You should implement a network using the backpropagation learning rule.
• Take the spectrogram of the sound files. Use those as the inputs to your neural network. Below (Figure 10): Amplitude and Spectrogram of the sentence: "The dog jumped over the fence". Left subplot is a female speaker, right subplot is a male speaker.
A. B.
Figure 10. The amplitudes (A) and spectrograms (B) of a female and male speaker uttering "The dog jumped over the fence"

• a = spectrogram(b) will return an array of complex numbers in a. They have an imaginary and a real part. For our purposes, the real part will do. For this, type c = real(a). c will now contain the real part of the imaginary numbers in a.

• This project is deliberately under-constrained. Basically, you can try whatever you want to solve the problem. As a matter of fact, we encourage this, since it will help you understand neural networks that much better. Don't be frustrated and don't panic if you can't figure it out right away.
• If you take a spectrogram of the speech patterns that you are supposed to train the network with, it will return 129 rows and several thousand columns. A neural network that takes the entire information from this matrix has to have 129 inputs.
• This is obviously rather excessive. If you closely observe the spectrogram (see Figure 10B), you might be able to get away with less. The rows tessellate the frequency spectrum (y axis below). Power in a particular frequency band is at a particular row. Power in a particular frequency band at a particular time is in a particular combination of row and column.
• The point is that the left spectrogram has much more power in the upper frequencies than the one on the right. If you properly combine frequency bands (or sample them), you might be able to get away with a neural network that has only 5 or 10 inputs.
• The same applies for time. Your spectrogram will have several thousand columns. This kind of resolution is not necessary to get the job done. Try combining (averaging) the values in 100 or so columns into one. Then feed that to the neural network for training.
• Try using one hidden layer
• Try having a large number of hidden units (definitely more than 1)
• You will be using a supervised learning rule. The target is defined by which file the data came from (Pascal or Kira). Create an artificial index, assigning 1 and 0 (natural) or 1 and 2 (politically correct) to each.
• Always remember what the rows and columns of the variables you are using represent. Be aware of transformations in dimensions.

Attachment:- Lab4_NeuralNet.rar