Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Which of the memories stores the most number of bits, Which of the memories...

Which of the memories stores the most number of bits ? Ans. most number of bits stores in 32M x 8 As 2 5 x 2 20   = 2 25 Therefore 1M = 2 20 = 1K x 1K = 2 10 x 2 10

Implementation of a data entry application, Develop and submit an original ...

Develop and submit an original implementation of a data entry application. Identify and declare variables which will hold all data that needs to be entered to fill in the attached

Explain in detail about real time processing, Explain in detail about Real ...

Explain in detail about Real time (transaction) processing When booking seats on a flight, for illustration, real time (transaction) processing would be used. Response to a que

Explain macro definition and call, Explain Macro definition and call. ...

Explain Macro definition and call. Macro: The assembly language programming frequently finds this necessary to repeat certain piece of code several times during the course of

Break keys, Like a normal read, your input should break on the new line cha...

Like a normal read, your input should break on the new line character "\n"   and also on the up and down arrow keys. You will process the input made by the user when he/she enters

How do you pass basic types by reference, How do you pass basic types by re...

How do you pass basic types by reference?   In the case of basic types like bool, int, uint, etc. you can in fact choose among the two forms. If you prefix the method parameter

Analysis of sort bitonic, Analysis of Sort_Bitonic(X) The bitonic sorti...

Analysis of Sort_Bitonic(X) The bitonic sorting network needs log n number of phases for performing task of sorting the numbers. The first n-1 phases of circuit can sort two n/

Multi-operating systems, The assignment enhances the acquisition of new kno...

The assignment enhances the acquisition of new knowledge through reading, research and practical work in class and at home. It requires critical thinking applied to real life tasks

Backpropagation learning routine, Backpropagation Learning Routine: Co...

Backpropagation Learning Routine: Conversely as with perceptrons there the information in the network is stored in the weights than the learning problem comes down to the ques

Describe about the layered computer design, Describe about the Layered comp...

Describe about the Layered computer design Introduction Layered computer design focuses on computer design. It uses top-down, layered approach to design and also to improve

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd