Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

Background, Background, Examples and Hypothesis: Now we will switch of...

Background, Examples and Hypothesis: Now we will switch off with three logic programs. So very firstly, we will have the logic program representing a set of positive examples

Illustrate about the single inheritance, Illustrate about the single inheri...

Illustrate about the single inheritance During inheritance, superclass feature may by override by a subclass defining that feature with the same name. The overriding features (

Average waiting time, Customer arrivals at a gas station can be characteriz...

Customer arrivals at a gas station can be characterized by exponential distribution with mean of 10 minutes. The amount of time they take to pump gas can be characterized by an ex

Define bootstrap loader, Define bootstrap loader? The ROM portion of ma...

Define bootstrap loader? The ROM portion of main memory is required for storing an initial program known as bootstrap loader. It is a program whose function is to start the com

Design a bcd to seven segment decoder, Design a BCD to seven segment decode...

Design a BCD to seven segment decoder that accepts a decimal digit in BCS and generates the appropriate output for segments in display indicator. Ans: BCD-TO-seven-Segment

What do you mean by memory mapped i/o, What do you mean by memory mapped I/...

What do you mean by memory mapped I/O? In Memory mapped I/O, there is no particular input or output instructions. The CPU can manipulate I/O data residing in interface register

Disadvantages of pipeline - computer architecture, Disadvantages of pipelin...

Disadvantages of pipeline: Pipeline architecture has 2 major disadvantages.  First is its complexity and second is the inability to constantly run the pipeline at full speed,

What is a deadlock, What is a Deadlock? Deadlock is a situation, in th...

What is a Deadlock? Deadlock is a situation, in that processes never complete executing and system resources are tied-up, preventing another job form starting. If the resou

Simple instruction formats, Simple instruction formats: RISC uses simpl...

Simple instruction formats: RISC uses simple instruction formats. Usually only one or a few instruction formats are used. In these machines instruction length is fixed and alig

What is response ratio, What is response ratio? HRRN that is Highest Re...

What is response ratio? HRRN that is Highest Response Ratio Next scheduling is a non-preemptive discipline, as same to Shortest Job First (SJF) wherein the priority of all jobs

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd