Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Multiple valued logics, Multiple valued logics: Multiple valued logics...

Multiple valued logics: Multiple valued logics, where altered types of truth value such as "unknown" are may be allowed. These have some of the particular advantages of fuzzy

Algorithm to have a good best case running time, How can we change almost a...

How can we change almost any algorithm to have a good best case running time? Check whether the input constitutes an input at the very starting Or else run the original algo

Analysis of amdahls law, Q. Analysis of Amdahls law? The conclusions of...

Q. Analysis of Amdahls law? The conclusions of analysis of Amdahl's law are: 1) To optimize performance of parallel computers modified compilers should be developed that sho

Two ethernet switches sw1 and sw2 , Six machines H1, H2, ..., H6 are linked...

Six machines H1, H2, ..., H6 are linked by two Ethernet switches SW1 and SW2 as given below. Both switches have four ports drawn. The port numbers are as indicated. Suppose the two

Explain the concept of process, The Concept of Process Unofficially a p...

The Concept of Process Unofficially a process is a program in execution subsequent to the program has been loaded in the main memory. Though a process is more than just a progr

Define minterm and the maxterm - canonical form, Define Minterm and the Max...

Define Minterm and the Maxterm - Canonical Form? Any Boolean expression perhaps expressed in terms of either minterms or maxterms. The literal is a single variable within a t

What is a binary tree, What is a binary tree? A binary tree 'T' is desc...

What is a binary tree? A binary tree 'T' is described as A finite set of elements, known as nodes, such that: either (i) or (ii) happens: (i) T is empty (known as the 'nu

Describe collective communications, Q. Collective Communications? A num...

Q. Collective Communications? A number of message-passing systems allow communication involving more than two processors. Such type of communication can be known as collective

How does bus arbitration typically work, How does bus arbitration typically...

How does bus arbitration typically work? i.  A bus master waiting to use the bus asserts by  the bus request. ii.  A bus master cannot be the bus until it's request is grant

Example multi-layer ann with sigmoid units, Example Multi-layer ANN with Si...

Example Multi-layer ANN with Sigmoid Units: However we will concern ourselves here that with ANNs containing only one hidden layer and as this makes describing the backpropaga

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd