Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Explain transformational analogy, Question 1 Why is it important that a...

Question 1 Why is it important that an expert system be able to explain why and how questions related to a problem solving session. 2 Give the architecture of an Expert syst

Odd-even merging circuit, Odd-Even Merging Circuit Let's firstly unders...

Odd-Even Merging Circuit Let's firstly understand the idea of merging two sorted sequences using an odd-even merging circuit. The functioning of a merging circuit is as given b

What are the main data structures used in rdbms, What are the main data str...

What are the main data structures used in the following areas: RDBMS, Network data model and Hierarchical data model? RDBMS = Array (i.e. Array of structures) Network data m

Which is not a valid page replacement policy, Which is not a valid page rep...

Which is not a valid page replacement policy? Ans. RU policy (Recurrently used) is not a valid page replacement policy.

Operations of the web, Operations of the Web mainly depend on hypertext as ...

Operations of the Web mainly depend on hypertext as its means of interacting with users. But what is hypertext? Hypertext as such is the same as regular text that is it can be read

Why generalization is very strong, Even while Generalization satisfies Stru...

Even while Generalization satisfies Structural, Interface, Behaviour properties. It is mathematically very strong, as it is Antisymmetric and Transitive. Antisymmetric: employe

Conversion of decimal number to binary number, Conversion of Decimal Number...

Conversion of Decimal Number to Binary Number: For converting a decimal number to binary integer part and fractional part are handled separately. Let's describe it with help of

Write-allocate and no-write-allocate policy in a cache, (a) How does a da...

(a) How does a data cache take advantage of spatial locality? Give an suitable example. (b) What are the basic differences between a write-allocate and no-write-allocate po

Baic electrical, Ask question #Minimumstate and explain thevenins theorem a...

Ask question #Minimumstate and explain thevenins theorem as applicable to electrical circuits 100 words accepted#

Explain bit slice ALUs, Bit Slice ALUs It was possible to manufacture ...

Bit Slice ALUs It was possible to manufacture smaller like 4 or 8 bits fixed point ALUs on a single IC chip. If these chips are constructed as expendable types then employing

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd