Adding momentum - sigmoid units, Computer Engineering

Assignment Help:

Adding Momentum - Sigmoid units:

However imagine a ball rolling down a hill as it does so then it gains momentum in which its speed increases and it becomes more difficult to stop. Alternatively as it rolls down the hill towards the valley floor as the global minimum, then it might occasionally wander into local hollows. Moreover,, there it may be that the momentum it has obtained stays it rolling up and out of the hollow and back on track to the valley floor. 

Hence the crude analogy describes one heuristic technique for avoiding local minima that called adding momentum, funnily enough. Thus the method is simple as: now for each weight remember as the previous value of Δ that was added on to the weight in the last epoch. Rather then, where updating that weight for the current epoch, add on a little of the previous Δ. Now how small to make the additional extra is controlled through a parameter α that's called the momentum which is set to a value between 0 and 1. 

Alternatively to see why this might help bypass local minima so note there that if the weight change carries on in the direction it was going in the previous epoch and then the movement will be a little much more pronounced in the current epoch. Thus this effect will be compounded as the search continues in the same direction. Where the trend finally reverses so then the search may be at the global minimum case there it is hoped that the momentum won't be enough to take it anywhere other than where it is belongs. Conversely the search may be at a fairly narrow local minimum. So next there in this case, even though the back propagation algorithm which dictates Δ will change direction then it may be that the additional extra from the previous epoch as the momentum may be enough to  counteract this effect for a few steps. Then we can saythese few steps may be all that is utilised to bypass the local minimum.


Related Discussions:- Adding momentum - sigmoid units

What is artificial intelligence fuzzy logic, Fuzzy logic is a form of vario...

Fuzzy logic is a form of various-valued logic; it deals with reasoning that is approximate rather than fixed & exact. In contrast with traditional logic theory, where binary sets h

How does the internet work, Every computer connected to Internet has a uniq...

Every computer connected to Internet has a unique address. Let's just say your IP address is 1.2.3.4 and you want to send a message to computer with IP address 5.6.7.8. Message you

Selecting integerated learning experiences, Q. Selecting integerated learni...

Q. Selecting integerated learning experiences? It is important that we select appropriate experiences for children, more so those that are from the child's world and emerge fro

What are the reasons for feedback in a control system, Question: a) Wha...

Question: a) What are the reasons for feedback in a control system? b) What are the roles of the configuration and fault managers in a real-time system? c) What are stimu

Explain asynchronous decade counter, Draw the circuit diagram of Asynchrono...

Draw the circuit diagram of Asynchronous decade counter and explain its working. Ans: To  design  a  circuit diagram of decade  asynchronous  counter  initially  we  dr

Explain basic time division and time switching method, What is time divisio...

What is time division switching? With the help of block diagram explain basic time division and time switching method. Time Division Switching: A switching component can be s

What are the stages of data mining, What are the stages of data mining? ...

What are the stages of data mining? The procedure of data mining comprises three stages, which are given below: a) The initial exploration b) Model building c) Deploym

Restore the system defaults, In order to restore the system defaults for al...

In order to restore the system defaults for all changes made with the format statement is Format Reset

Ida* search - artificial intelligence, IDA* Search - artificial intelligenc...

IDA* Search - artificial intelligence: A* search is a sophisticated and successful search strategy. In fact, a problem with A* search is that it must keep all states in its me

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd