Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

How do you recognize the performance bottlenecks, Performance Bottlenecks c...

Performance Bottlenecks can be identified by using monitors. These monitors might be application server monitors, database server monitors, web server monitors and network monitors

Determine the output waveforms for and and or gates, The voltage waveforms ...

The voltage waveforms shown in given fig. are applied at the inputs of 2-input AND and OR gates. Determine the output waveforms. Ans. The Output waveforms for AND and O

Computer organisation, How many 32K X 1 RAM chips are needed to provide a m...

How many 32K X 1 RAM chips are needed to provide a memory capacity 256 kilobytes?

Explain language processing, System Software 1. Explain language proces...

System Software 1. Explain language processing? Explain its activities in detail. 2. Discuss briefly about Assembler directives. Also explain about Macro definition and call

4bit counter using JK flip flop and unique sequence, Need to build a 4bit J...

Need to build a 4bit JK flip flop counter with a unique patter. 6 4 2 5 3 1 F C B A

How can one decide this decimal position, Q. How can one decide this decima...

Q. How can one decide this decimal position? Decimal position can be signified by a position between flip-flops (storage cells in computer). However how can one decide this dec

What is called checking CRC in cyclic redundancy, In cyclic redundancy chec...

In cyclic redundancy checking CRC is the? Checking CRC, in cyclic redundancy is the remainder. Normal 0 false false false EN-IN X-NONE X-NONE

Explain conditions for the critical section problem solution, What are the ...

What are the properties which a data item should possess to implement a critical section? The Critical Section Problem Solution should satisfy the following conditions: 1.

What are the uses of interrupts, What are the uses of interrupts?  ...

What are the uses of interrupts?  Recovery from errors Debugging Communication among programs Use of interrupts in operating system

Extra variables by which customers call for work to work out, What are the ...

What are the extra variables by which customers call for to work out? All customers who do not purchase Internet connectivity through a Service Provider do call for to work out

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd