Entropy - learning decision trees, Computer Engineering

Assignment Help:

Entropy - learning decision trees:

Through putting together a decision of tree is all a matter of choosing that attribute to test at each node in the tree. Further we shall define a measure that is information gain that will be used to decide which attribute to test at each node. By considering through information gain is itself calculated utilising a measure called entropy that we first define for the case of a binary decision problem so then define for the general case. 

As per given a binary categorisation like C, and a set of examples, S, that are utilising the proportion of examples categorised as positive by C is p+  or the proportion of examples categorised  like negative by C is p-, then the entropy of S is as: 

641_Entropy - learning decision trees.png

Now next here instantly we defined entropy first for a binary decision problem is easier to get an impression of what it is trying to calculate. As Tom Mitchell puts this quite well: like 

"In order to define an information gain precisely so we begin by defining a measure commonly utilising in information theory that is called entropy in which characterizes the (im)purity of as an arbitrary collection of examples."


Related Discussions:- Entropy - learning decision trees

Explain how presentation layer helps in establishing, Explain how presentat...

Explain how presentation layer helps in establishing and processing data in End to End layers. The idea of the presentation layer is to stand for information to the communicati

Predicate logic - artificial intelligence, Translate the following sentence...

Translate the following sentences into predicate logic, using Russell's theory of definite descriptions to represent the article the. a. The wizzard of Oz fell. b. Everyone a

Explain advantages and disadvantages of a active document, Explain advantag...

Explain advantages and disadvantages of a active document. The advantage of an active document over a dynamic document lies in its capability to update the information contin

I2c protocol bus , I²C TECHNOLOGIES The I2C protocol bus is two bi-dire...

I²C TECHNOLOGIES The I2C protocol bus is two bi-directional wires, serial data (SDA) and serial clock (SCL), that transmit information between the devices connected to the bus.

Processes of information system, Processes of Information System The b...

Processes of Information System The basic purpose of the Information System is to convert and manage data into something that is meaningful. Bocij et al (2003) lists the basic

Determine about the radio frequency, Determine about the Radio Frequency (R...

Determine about the Radio Frequency (RFID)  This method includes using small electronic devices containing an Identification microchip and antenna; they work in a similar way

Which scheduling is suitable for time-shared operatin system, Which schedul...

Which scheduling policy is most suitable for a time-shared operating system? Ans. Round-Robin scheduling policy is most appropriate for a time-shared operating system.

EDC, Conparision of masfet and jfet

Conparision of masfet and jfet

Describe how management information systems, Question: a) Why do we u...

Question: a) Why do we use the Internet as the new distribution channel for e-banking products and services? b) In the context of e-banking or e-commerce, outline some o

Java Applet, Simple codes for robot using applet

Simple codes for robot using applet

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd