Adding momentum - sigmoid units, Computer Engineering

Assignment Help:

Adding Momentum - Sigmoid units:

However imagine a ball rolling down a hill as it does so then it gains momentum in which its speed increases and it becomes more difficult to stop. Alternatively as it rolls down the hill towards the valley floor as the global minimum, then it might occasionally wander into local hollows. Moreover,, there it may be that the momentum it has obtained stays it rolling up and out of the hollow and back on track to the valley floor. 

Hence the crude analogy describes one heuristic technique for avoiding local minima that called adding momentum, funnily enough. Thus the method is simple as: now for each weight remember as the previous value of Δ that was added on to the weight in the last epoch. Rather then, where updating that weight for the current epoch, add on a little of the previous Δ. Now how small to make the additional extra is controlled through a parameter α that's called the momentum which is set to a value between 0 and 1. 

Alternatively to see why this might help bypass local minima so note there that if the weight change carries on in the direction it was going in the previous epoch and then the movement will be a little much more pronounced in the current epoch. Thus this effect will be compounded as the search continues in the same direction. Where the trend finally reverses so then the search may be at the global minimum case there it is hoped that the momentum won't be enough to take it anywhere other than where it is belongs. Conversely the search may be at a fairly narrow local minimum. So next there in this case, even though the back propagation algorithm which dictates Δ will change direction then it may be that the additional extra from the previous epoch as the momentum may be enough to  counteract this effect for a few steps. Then we can saythese few steps may be all that is utilised to bypass the local minimum.


Related Discussions:- Adding momentum - sigmoid units

Receive a message, Receive a message. tid is integer task identifier of tra...

Receive a message. tid is integer task identifier of transmitting process supplied by user and msgtag is message tag supplied by user( must be non negative integer). The process re

What is a multiple line field, What is a multiple line field? A multipl...

What is a multiple line field? A multiple line field is a special part of field which allows the user to enter multiple lines of data into it.

Data parallel model - parallel programming model, In the data parallel mode...

In the data parallel model, many of the parallel work focus on performing operations on a data set. The data set is usually organized into a common structure, such as an array or a

Convert the binary number to gray code, Convert the binary number 10110 to ...

Convert the binary number 10110 to Gray code ? Ans. For changing binary number 10110 in its equivalent Gray code the rules are as, the left most bit that is MSB in Gray code is 1

Find values of x using 7s complement, Q. Perform binary subtraction, using ...

Q. Perform binary subtraction, using 1s & 2s complement: 1) 1010-1011 2) 0.1111-0.101 3) 11.11-10.111 Q.  (192.25)10 - (C0.C)16 = (x)7 Find values of x, using 7's compl

Pruning - artificial intelligence, Pruning - Artificial intelligence Re...

Pruning - Artificial intelligence Remember that pruning a search space means deciding that particular branches should not be explored. If an agent  surly  knows that exploring

How many lines of address bus used for memory of 2048 bytes, How many lines...

How many lines of address bus must be used to access 2048 bytes of memory when available RAM chips 128 × 8.  How many lines of these will be common to each chip? Ans. AS chips

Describe jmx concepts and architecture, MX is conceptually easy, yet bears ...

MX is conceptually easy, yet bears the fruit of years of domain experience and research. In a nutshell, JMX describes a standard means for applications to expose management functio

Design a 8 to 1 multiplexer using the fourvariable function, Design a 8 to ...

Design a 8 to 1 multiplexer by using the fourvariable function given by F(A, B, C, D) = ∑ m(0,1,3,4,8,9,15). Ans. Design of 8 to 1 Multiplexer: It is a four-variable function a

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd