Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

Explain the operation of JK flip flop when all inputs are 0, Write the trut...

Write the truth table for a clocked J-K flip-flop that is triggered by the positive-going edge of the clock signal. Explain the operation of this flip-flop for the following condit

What are set up time & hold time constraints, What do they signify Which on...

What do they signify Which one is critical for calculating maximum clock frequency of a circuit? Ans) Set up time constraint implies how late the input signal can arrive befor

Subtract, Write the Add/subtract rule for floating point numbers. Ans:...

Write the Add/subtract rule for floating point numbers. Ans: a. Select the number with the smaller exponent and shift its mantissa right a number of steps equal to the differe

Granularity-concept of parallel execution and concurrent , Granularity ...

Granularity Granularity refers to the quantity of computation done in parallel relative to the dimension of the entire program. In parallel computing, granularity is a qualitat

Characteristics of decision support system, Q. Characteristics of decision ...

Q. Characteristics of decision support system? Characteristics of DSS- The qualities of the DSS are as follows: - 1. DSS focus on towards providing assistance in analyzing s

Explain the uses of arithmetic processor, Explain the uses of arithmetic pr...

Explain the uses of arithmetic processor A processor has arithmetic processor (as a sub part of it) which executes arithmetic operations.  The data  type,  assumed  to  reside

What does formal verification mean, What does formal verification mean? ...

What does formal verification mean? Formal verification uses Mathematical techniques by proving the design by assertions or properties. Correctness of the design can be achiev

What is capacitor-based keyboards, Q. What is Capacitor-Based Keyboards? ...

Q. What is Capacitor-Based Keyboards? These keyboards are based on the idea of Capacitance. A simple capacitor comprises a pair of conductive plates having opposite charges and

Develop the multi agent system, The new season of AFL, and the excited comi...

The new season of AFL, and the excited coming FIFA World Cup in South Africa this June, many sports fans are willing to find the latest results of their beloved sports teams. In th

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd