Over fitting considerations - artificial intelligence, Computer Engineering

Assignment Help:

Over fitting Considerations - artificial intelligence

Left  unexamined ,  back  propagation  in  multi-layer  networks  may  be very susceptible  to over fitting itself to the training examples. The following graph plots the error on the training and test set as the number of weight updates increases. It is error prone of networks left to train unchecked.

810_Over fitting Considerations.png

Alarmingly, even though the error on the training set continues to slowly decrease, the error on the test set essentially begins to increase towards the end. It is clearly over fitting, and it relates to the network starting to find and fine-tune to idiosyncrasies in the data, rather than to general properties. Given this phenomena, it would not be wise to use some sort of threshold for the error as the termination condition for back propagation.

In the cases where the number of training examples is high, one antidote to over fitting is to crack the training examples into a set to use to train the weight and a set to hold back as an internal validation set. This is a mini-test set, which may be used to keep the network in check: if the error on the validation set reaches minima and then start to increase, then it could be over fitting in beginning to occur.

Note that (time permitting) it is good giving the training algorithm the advantage of the doubt as much as possible. That is, in the validation set, the error may also go through local minima, and it is unwise to stop training as soon as the validation set error begin to increase, as a better minima can be achieved later on. Of course, if the minima are never bettered, then the network which is in final presented by the learning algorithm should be re-wound to be the 1 which produced the minimum on the validation set.

Another way around over fitting is to decrease each weight by a little weight decay factor during each epoch. Learned networks with large (negative or positive) weights tend to have over fitted the data, because larger weights are needed to accommodate outliers in the data. Thus, keeping the weights low with a weight decay factor can help to steer the network from over fitting.


Related Discussions:- Over fitting considerations - artificial intelligence

Operating system., what is the minimum number of page faults for an optimal...

what is the minimum number of page faults for an optimal page replacement strategy?

Determine the uses of defparam, Using defparam Parameter values can be ...

Using defparam Parameter values can be changed in any module instance in the design with keyword defparam. Hierarchical name of the module instance can be used to override para

Define congestion and grade of service, Define congestion and grade of serv...

Define congestion and grade of service. Congestion : This is uneconomic to give sufficient equipment to carry all the traffic which could possibly be offered to a telec

Types of software - computer architecture, Types of software: A layer ...

Types of software: A layer structure is described where Operating System is located on generally used software systems on desktops   System software System sof

Explain about dual in line memory modules, Q. Explain about Dual In line Me...

Q. Explain about Dual In line Memory Modules? A DIMM is capable of delivering 64 data bits right away. Usual DIMM capacities are 64MB and up. Every DIMM has 84 gold patted conn

Functions - first-order logic, Functions - first-order logic: Function...

Functions - first-order logic: Functions can be thought of as exceptional predicates, wherever we think of all but one of the arguments as input and the output as final argume

Explain about the value chain for a manufactured product, Explain about the...

Explain about the value chain for a manufactured product. The products sold in shops and purchased for use within organizations are the result of a complicated web of relations

Explain about the network level in detail, Explain about the network level ...

Explain about the network level in detail. Network Level Firewall/Packet Filters: At the Network level firewalls operate upon the mechanism of filtering individual IP pa

How does output caching work in asp.net, How does output caching work in AS...

How does output caching work in ASP.NET?    Output caching is a powerful method that enhances request/response throughput by caching the content generated from dynamic pages. O

What is information technology, Information Technology is nothing but the s...

Information Technology is nothing but the study, design, development, execution, support and management of computer based information systems mainly the applications software & har

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd