Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

Uneven load distribution in parallel computers, Q. Uneven Load Distribution...

Q. Uneven Load Distribution in parallel computers? In parallel computers the problem is split in sub-problems in addition is assigned for computation to several processors howe

Page directive include, What is the difference between, page directive incl...

What is the difference between, page directive include, action tag include? Ans) One difference is whereas using the include page directive, in translation time it is making t

Determine bhca rating for processor and call completion rate, A call proces...

A call processor in an exchange requires 120 ms to service a complete call. What is the BHCA rating for the processor? If the exchange is capable of carrying 700 Erlangs of traffic

Flow charts, n=(x*2)/(1=0) the value x=0 and is used to stop the algerithin...

n=(x*2)/(1=0) the value x=0 and is used to stop the algerithin.The calculation is repeated using values of x=0 is input. There is only a need to check for error positions. The va

Explain compiler, What is compiler? A system software program known as ...

What is compiler? A system software program known as a compiler translates the high-level language program into a suitable machine language program having instruction such as t

Explain the term internet, Explain the term Internet. Internet: ...

Explain the term Internet. Internet: The Internet, an umbrella term covering countless network and services that comprise a super-network, is a global network of compute

Neural network programming using matlab, I need the matlab basics for crea...

I need the matlab basics for creating and configuring a neural network with 2 hidden layers

What is mmu, What is MMU? MMU is the Memory Management Unit. It is a sp...

What is MMU? MMU is the Memory Management Unit. It is a special memory control circuit used for executing the mapping of the virtual address space onto the physical memory.

Describe briefly how firewalls prevent network, Describe briefly how firewa...

Describe briefly how firewalls prevent network. A firewall is only a program or hardware device which filters the information coming via the Internet connection within your pr

What do you mean by u-area or u-block, What do you mean by u-area (user are...

What do you mean by u-area (user area) or u-block? This having the private data that is manipulated only by the Kernel. This is local to the Process, i.e. every process is a

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd