Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

Salient features of direct addressing mode, Q. Salient features of direct a...

Q. Salient features of direct addressing mode? A number of salient points about this technique are: This scheme offers a limited address space since if address field has

What are the aspects of CAD, What are the aspects of CAD Increased use ...

What are the aspects of CAD Increased use of computers has led to many of the above affects. Other aspects to consider include: -  deskilling of the work force (for illustra

Write about TSR, Write about TSR TPA also holds TSR (terminate and stay...

Write about TSR TPA also holds TSR (terminate and stay resident) programs which remain in memory in an active state until activated by a hot-key sequence or another event like

What the first part of the address identifies in e-mailbox, The first part ...

The first part of the address in electronic mailbox identifies? The first part of address in E-Mail identifies the user's mail box.

Describe the object modeling notations, OBJECT MODELING NOTATIONS: BASIC ...

OBJECT MODELING NOTATIONS: BASIC Concepts A system is the collection of subsystems which organised to accomplish a purpose and described by a set of models from variou

Operating system, explain network operating system and design issues?

explain network operating system and design issues?

Determine the return loss and round trip delay for echo, In a national tran...

In a national transmission system, the characteristic impedances of the 4-wire circuit and the 2-wire circuit are 1200 ? and 1000 ? respectively. The average phase velocity of the

What is telex, Telex is a (A)  Telephone Service between various subsc...

Telex is a (A)  Telephone Service between various subscribers (B)  Tele printer Service between various subscribers (C)  Television Service between various subscri

Define interrupt processing, Q. Define Interrupt Processing? On complet...

Q. Define Interrupt Processing? On completion of execution of an instruction machine checks whether there is any pending interrupt request for interrupts which are enabled. If

Local minima - sigmoid units, Local minima - sigmoid units: Alternativ...

Local minima - sigmoid units: Alternatively in addition to getting over some local minima where the gradient is constant in one direction or adding momentum will increase the

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd