Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Proof by contradiction, Proof by Contradiction: Now for forward chaini...

Proof by Contradiction: Now for forward chaining and backward chaining both have drawbacks. But another approach is to think about proving theorems by contradiction. So we can

What is read-only-memory, Q. What is Read-only-Memory? Read only memory...

Q. What is Read-only-Memory? Read only memory is an illustration of a Programmable Logic Device (PLD) it implies that binary information which is stored within a PLD is specifi

Complicated question, Hi I need a help in this question : A telephone sw...

Hi I need a help in this question : A telephone switchboard handles ? calls on average during a rush hour, and the switchboard can at most make k connections per minute. Write a

What is port, What is port? What are the types of port available? An I/...

What is port? What are the types of port available? An I/O interface having of circuitry needed to connect an I/O device to computer bus. One side consists of a data path with

Illustrate the table of types of serach engines, Illustrate the table of ty...

Illustrate the table of types of serach engines Type in: CIE and the search engine would return about 22 million hits Type in

Characteristics of the id process, The following is taken from the first ed...

The following is taken from the first edition of the Set Book. We have included this because we find these characteristics helpful. We will refer to them later. In addition to t

Discuss about second generation electronic computers, Second Generation (19...

Second Generation (1954-1962) The second generation saw various significant developments at every level of computer system design, from technology used to build the basic circu

Name at least six file extensions of microsoft access, Some of the file ext...

Some of the file extensions present in MS Access are ? Access blank project template .adn ? Access database .accdb ? Access Workgroup .mdw ? Access project .adp ? A

What is dhcp and how it works, Full form of is DHCP: Dynamic Host Control P...

Full form of is DHCP: Dynamic Host Control Protocol. It is mainly used to bound MAC Addresses of system. Especially for Wireless System.

What is one-way association, What is one-way association? One-way assoc...

What is one-way association? One-way association means association will traverse in only in one direction and implement using the actual implementation. If the multiplicity is

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd