Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Operating system, explain the various types of computer systems

explain the various types of computer systems

Rule in a single direction - equivalences rules, Rule in a single direction...

Rule in a single direction - equivalences rules: Hence there the power to replace  sub expressions always allows use to prove theorems with equivalences: as given in the above

Explain pre-emptive scheduling, Explain Pre-emptive scheduling? Pre-em...

Explain Pre-emptive scheduling? Pre-emptive scheduling: in its approach, center processing unit can be taken away from a process if there is a require while in a non-pre-empt

What are function modules, What are function modules? Function m...

What are function modules? Function modules are general-purpose library routines that are available system-wide.

How can data and instructions are put into the computers, Q. How can data a...

Q. How can data and instructions are put into the computers? The data and instructions to a computer are provided by external environment. It determines that input devices are

Design a 32:1 multiplexer, Design a 32:1 multiplexer using two 16:1 multipl...

Design a 32:1 multiplexer using two 16:1 multiplexers and a 2:1 multiplexer Ans. Design a 32 X 1 MUX by using two 16 X 1 MUX and one 2 X 1. Now here total 32 input lines

Describe the horizontal frequency or cycle, Q. Describe the Horizontal Freq...

Q. Describe the Horizontal Frequency or cycle? Horizontal Frequency: The time to scan one line connecting the right edge to left edge of the screen horizontally is known as t

Explain the major consideration of laptop computers, Explain the major cons...

Explain the major consideration of laptop One of the major considerations when buying a laptop is battery life. This can rely on a number of things though one major factor is p

For which class address 192.5.48.3 belogs, Address 192.5.48.3 belongs to? ...

Address 192.5.48.3 belongs to? Address 192.5.48.3 belongs to class C.

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd