Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Determine the term- security, Determine the term- Security When using ...

Determine the term- Security When using Internet, security can be enhanced using encryption. Debit and credit card transactions can also be protected by a specific type of pas

What is hard drive interface, Q. What is Hard Drive Interface? Secondar...

Q. What is Hard Drive Interface? Secondary storage devices need a controller to proceed as an intermediary between device and rest of the computer system. On some computers the

What do you mean by perfect shuffle permutation, Q. What do you mean by Per...

Q. What do you mean by Perfect Shuffle Permutation? Perfect Shuffle Permutation: It was proposed by Harold Stone (1971). Consider N objects each is represented by n bit number

What is skew, What is skew? Clock Skew: In circuit design, clock ske...

What is skew? Clock Skew: In circuit design, clock skew is a phenomenon within synchronous circuits wherein the clock signal (sent through the clock circuit) arrives at dive

What are the different methods used for handling, What are the different me...

What are the different methods used for handling the situation when multiple interrupts occurs? 1) Vectores interrupts 2) Interrupt nesting 3) Simultaneous Requests.

Determine a ring counter that consisting of five flip-flops, A ring counter...

A ring counter consisting of five Flip-Flops will have ? Ans. A ring counter have 5 states while consisting of Five Flip-Flops.

NETWORK ADMIN, Discuss the risks of having a single root user and how more ...

Discuss the risks of having a single root user and how more limited management abilities can be given to others users on Linux/UNIX systems.uestion #Minimum 100 words accepted#

Sigmoid units, Sigmoid units: Always remember that the function inside...

Sigmoid units: Always remember that the function inside units take as input the weighted sum, S and of the values coming from the units connected to it. However the function i

Digital electronics, what is Asynchronous Finite State Machines?

what is Asynchronous Finite State Machines?

What is configuration, What is configuration?  Turing  machine  compute...

What is configuration?  Turing  machine  computes,  changes  occur  in  the  current  state,  the  current  tap contents  and  the  current  head  location. A  setting  of  the

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd