Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Elements of composition, The Elements of composition are the formal languag...

The Elements of composition are the formal language used to communicate in the visual arts. The elements of composition can be loosely broken into 5 main elements, Line, shape, val

Which of the memories stores the most number of bits, Which of the memories...

Which of the memories stores the most number of bits ? Ans. most number of bits stores in 32M x 8 As 2 5 x 2 20   = 2 25 Therefore 1M = 2 20 = 1K x 1K = 2 10 x 2 10

Name the various display devices, Name the various Display devices Diff...

Name the various Display devices Different types of display devices are discussed along with the principles on which these work. The main display devices used with CAD systems

Explaintask and parallel task, Task   A task is logically discrete ...

Task   A task is logically discrete section of computational work. A task is normally a program or else set of instructions which are executed by a processor. Parallel

Define a variant of top-down parsing without backtracking, A parser which i...

A parser which is a variant of top-down parsing without backtracking is? Ans. Recursive Descend is a variant of top-down parsing without backtracking.

Explain the working of a binary counter, In binary counter the flip flop of...

In binary counter the flip flop of lowest order position is complemented with each pulse. This means that JK input position must be maintained with logic one

What are the advantages of batch processing, What are the advantages of bat...

What are the advantages of batch processing This has the advantage which jobs can be processed when computer resources are less busy (for example during the evening or at night

Define usb, Define USB. The Universal Serial Bus(USB) is an industry st...

Define USB. The Universal Serial Bus(USB) is an industry standard developed to give two speed of operation known as low-speed and full-speed. They provide simple, low cost and

Optimality - heuristic search strategies, Optimality - Heuristic search str...

Optimality - Heuristic search strategies: The path cost of a solution is considered as the sum of the costs of the actions that led to which solution is given. This is only on

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd