Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

What is domain analysis, What is domain analysis? Domain analysis is c...

What is domain analysis? Domain analysis is concerned with devising a precise, concise, understandable and correct model of the real world. Analysis starts with problem statem

Discuss program testing and debugging in detail, Discuss program testing an...

Discuss program testing and debugging in detail. In program debugging and testing significant steps are as follows: a. For the program, construction of test data b. Analy

Testmanager can be used for performance testing, Rational Testmanager is a ...

Rational Testmanager is a complicated tool that can be used for automating performance tests on client/server systems. A client/server system have client applications accessing a d

Explain the properties of hypercube, Q. Explain the properties of Hypercube...

Q. Explain the properties of Hypercube? Properties of Hypercube: Hypercube is both edge and node symmetric. The labels of any two neighbouring nodes vary in exactl

Traditional schema model, (a) Why did SAP introduce the extended star schem...

(a) Why did SAP introduce the extended star schema? Explain why it is reported to be better than the traditional schema model? (b) What is the difference between a dimension use

State the term in detail $strobe, State the term in detail $strobe $str...

State the term in detail $strobe $strobe. This task is very similar to $display task except for a slight difference.  If many other statements are executed in same time unit as

What is one-way association, What is one-way association? One-way assoc...

What is one-way association? One-way association means association will traverse in only in one direction and implement using the actual implementation. If the multiplicity is

Basic need of search engines, Q. Basic need of Search Engines? Search E...

Q. Basic need of Search Engines? Search Engines are programs which search the web. Web is a big graph with pages being the nodes and hyperlinks being the arcs. Search engines c

What do you call an event and when do you call an assertion, What do you ca...

What do you call an event and when do you call an assertion? Assertion based Verification Tools, checks whether a statement holds a explained  property or not, while, Event bas

Explain standard set of procedure to complete a call process, Explain a s...

Explain a standard set of procedure for completing a local telephone call or call processing. Completing a local telephone call among two subscribers connected to the similar

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd