Use intuitive arguments to justify these four heuristics

Assignment Help Basic Computer Science

Reference no: EM131108996

In Jacobs (1988), the following heuristics are proposed to accelerate the convergence of on-line back-propagation learning:

(i) Every adjustable network parameter of the cost function should have its own learn ingrate parameter.

(ii) Every learning-rate parameter should be allowed to vary from one iteration to the next.

(iii) When the derivative of the cost function with respect to a synaptic weight has the same algebraic sign for several consecutive iterations of the algorithm the learning-rate parameter for that particular weight should be increased.

(iv) When the algebraic sign of the cost function with respect to a particular synaptic weight alternates for several consecutive iterations of the algorithm, the learning-rate parameter for that weight should be decreased.
These four heuristics satisfy the locality constraint of the back-propagation algorithm.

(a) Use intuitive arguments to justify these four heuristics.

(b) The inclusion of a momentum in the weight update of the back-propagation algorithm may be viewed as a mechanism for satisfying heuristics (iii) and (iv). Demonstrate the validity of this statement.

Reference no: EM131108996

Questions Cloud

Calculate the resulting dividend payout ratio : Chancellor Industries has retained earnings available of $1.2 million. The firm plans to make two investments that require financing of $950,000 and $1.75 million, respectively. Chancellor uses a target capital structure with 60% debt and 40% equity...

The dutzel diesel case : Jack Haley, a senior supply manager for the Dynamite Truck Company, was confronted with an interesting predicament-and possibly a trip overseas. Rising gasoline costs and increased foreign competition had caused the management at Dynamite Truck to..

When will the stock begin selling ex dividend : Stephanie’s Cafes, Inc., has declared a dividend of $1.30 per share for shareholders of record on Tuesday, May 2. The firm has 200,000 shares outstanding and will pay the dividend on May 24. How much cash will be needed to pay the dividend? When will..

How would dexter account for the temporary differences : How should Dexter classify the deferred tax consequences of the temporary differences on its balance sheet?

Use intuitive arguments to justify these four heuristics : When the algebraic sign of the cost function with respect to a particular synaptic weight alternates for several consecutive iterations of the algorithm, the learning-rate parameter for that weight should be decreased.These four heuristics satisfy..

Compare a stock split with a stock dividend. : Compare a stock split with a stock dividend.

How does the unemployment rate of your state compare to us : How does the unemployment rate of your state compare to that of the U.S. as a whole? What factors do you feel most contribute to your state's unemployment numbers?

Derive the obd procedure as a special case of the optimal : Using this approximation, derive the OBD procedure as a special case of the optimal-brain-surgeon (OBS) algorithm, studied in Section 4.14.

The issuance of bonds on december 31 : The issuance of bonds on December 31, 2015 and the first through fourth interest payments on each June 30 and December 31.

User Account

All Pages