Example calculation of entropy, Computer Engineering

Assignment Help:

Example Calculation:

If we see an example we are working with a set of examples like S = {s1,s2,s3,s4} categorised with a binary categorisation of positives and negatives like that s1  is positive and the rest are negative. Expect further there that we want to calculate the information gain of an attribute, A, and  A can take the values {v1,v2,v3} obviously. So lat in finally assume that as: 

1745_Example Calculation of Entropy.png

Whether to work out the information gain for A relative to S but we first use to calculate the entropy of S. Means that to use our formula for binary categorisations that we use to know the proportion of positives in S and the proportion of negatives. Thus these are given such as: p+ = 1/4 and p- = 3/4. So then we can calculate as: 

Entropy(S) = -(1/4)log2(1/4) -(3/4)log2(3/4) = -(1/4)(-2) -(3/4)(-0.415) = 0.5 + 0.311

= 0.811 

Now next here instantly note that there to do this calculation into your calculator that you may need to remember that as: log2(x) = ln(x)/ln(2), when ln(2) is the natural log of 2. Next, we need to calculate the weighted Entropy(Sv) for each value v = v1, v2, v3, v4, noting that the weighting involves multiplying by (|Svi|/|S|). Remember also that Sv  is the set of examples from S which have value v for attribute A. This means that:  Sv1 = {s4}, sv2={s1, s2}, sv3 = {s3}. 

We now have need to carry out these calculations: 

(|Sv1|/|S|) * Entropy(Sv1) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

(|Sv2|/|S|) * Entropy(Sv2) = (2/4) * (-(1/2)log2(1/2) - (1/2)log2(1/2))

                                      = (1/2) * (-(1/2)*(-1) - (1/2)*(-1)) = (1/2) * (1) = 1/2 

(|Sv3|/|S|) * Entropy(Sv3) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

Note that we have taken 0 log2(0) to be zero, which is standard. In our calculation,

we only required log2(1) = 0 and log2(1/2) =  -1. We now have to add these three values together and take the result from our calculation for Entropy(S) to give us the final result: 

Gain(S,A) = 0.811 - (0 + 1/2 + 0) = 0.311 

Now we look at how information gain can be utilising in practice in an algorithm to construct decision trees.


Related Discussions:- Example calculation of entropy

Illustrate the system architecture, Question: a) Explain why pervasive...

Question: a) Explain why pervasive computing can be termed as a "technology that disappears". b) List and describe four main components of a MOTE used in Wireless Sensor N

Give the syntax of if-else statement, Give the syntax of "if-else" and "sw...

Give the syntax of "if-else" and "switch" statements and explain. if else This is used to decide whether to do something at a special point, or to decide between two courses

What do you mean by data warehousing, What do you mean by data warehousing?...

What do you mean by data warehousing? Data warehouse implies as: a) It is subject oriented b) It is integrated and c) It is time variant d) It is non-volatile colle

Programming project, l need a help for my project pllz:) Write a program th...

l need a help for my project pllz:) Write a program that will simulate ATM machine. The Program should prompt the user to enter a valid PIN password number of 5 digits (PIN: 12312)

Development tools environment , Main Objectives: • MPLAB In-Circuit Debugg...

Main Objectives: • MPLAB In-Circuit Debugger (ICD 2) functionality • ICD 2 Connection design • MPLAB ICD 2 setup with the PC and Interface board designed • I²C Protocol buses tech

Determine the binary equivalent of hexadecimal FA, The binary equivalent of...

The binary equivalent of (FA) 16   is ? Ans. (FA) 16 = (11111010) 10

Describe about the layered computer design, Describe about the Layered comp...

Describe about the Layered computer design Introduction Layered computer design focuses on computer design. It uses top-down, layered approach to design and also to improve

What are the authentication methods in .net, What are the authentication me...

What are the authentication methods in .NET?   1. WINDOWS AUTHENTICATION 2. FORMS AUTHENTICATION 3. PASSPORT AUTHENTICATION The authentication option for the ASP.N

Interfacing assembly language routines, By now you can write procedures bot...

By now you can write procedures both internal andexternal and pass parameters particularly through stack.Let's us use these concepts to see how assembly language can be interfaced

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd