Example calculation of entropy, Computer Engineering

Assignment Help:

Example Calculation:

If we see an example we are working with a set of examples like S = {s1,s2,s3,s4} categorised with a binary categorisation of positives and negatives like that s1  is positive and the rest are negative. Expect further there that we want to calculate the information gain of an attribute, A, and  A can take the values {v1,v2,v3} obviously. So lat in finally assume that as: 

1745_Example Calculation of Entropy.png

Whether to work out the information gain for A relative to S but we first use to calculate the entropy of S. Means that to use our formula for binary categorisations that we use to know the proportion of positives in S and the proportion of negatives. Thus these are given such as: p+ = 1/4 and p- = 3/4. So then we can calculate as: 

Entropy(S) = -(1/4)log2(1/4) -(3/4)log2(3/4) = -(1/4)(-2) -(3/4)(-0.415) = 0.5 + 0.311

= 0.811 

Now next here instantly note that there to do this calculation into your calculator that you may need to remember that as: log2(x) = ln(x)/ln(2), when ln(2) is the natural log of 2. Next, we need to calculate the weighted Entropy(Sv) for each value v = v1, v2, v3, v4, noting that the weighting involves multiplying by (|Svi|/|S|). Remember also that Sv  is the set of examples from S which have value v for attribute A. This means that:  Sv1 = {s4}, sv2={s1, s2}, sv3 = {s3}. 

We now have need to carry out these calculations: 

(|Sv1|/|S|) * Entropy(Sv1) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

(|Sv2|/|S|) * Entropy(Sv2) = (2/4) * (-(1/2)log2(1/2) - (1/2)log2(1/2))

                                      = (1/2) * (-(1/2)*(-1) - (1/2)*(-1)) = (1/2) * (1) = 1/2 

(|Sv3|/|S|) * Entropy(Sv3) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

Note that we have taken 0 log2(0) to be zero, which is standard. In our calculation,

we only required log2(1) = 0 and log2(1/2) =  -1. We now have to add these three values together and take the result from our calculation for Entropy(S) to give us the final result: 

Gain(S,A) = 0.811 - (0 + 1/2 + 0) = 0.311 

Now we look at how information gain can be utilising in practice in an algorithm to construct decision trees.


Related Discussions:- Example calculation of entropy

Explain bus, Define bus. When a word of data is transferred among units...

Define bus. When a word of data is transferred among units, all the bits are transferred in parallel over a set of lines called bus. In addition to the lines that take the data

Explain simplifying the sop boolean expression using k-map, Explain Simplif...

Explain Simplifying the SOP f the Boolean expression using the K-Map? To simplify the SOP of the Boolean expression using the K map, first identify all the input combinations tha

Validate the xml document , As an XML expert you are needed to model a syst...

As an XML expert you are needed to model a system for an online furniture shop. After an interview with the shop manager you have the certain information: The detail of th

Why should i use cgi, CGI is significant whenever you require to retain sta...

CGI is significant whenever you require to retain state information about a user, or run an application which communicates with the server. Things like guestbook's, Chat clients, d

Define master construct in fortan, Q. Define Master Construct in FORTAN? ...

Q. Define Master Construct in FORTAN? The master directive has following general form:  #pragma omp master structured_block  It causes master thread to execute structu

Computer based accounting systems, For the purposes of this assignment, you...

For the purposes of this assignment, you are the accountant for an industrial property development company, Devprop Leasing Co., that typically develops warehouse and industrial co

Parallel computer architecture , Parallel Computer Architecture Intr...

Parallel Computer Architecture Introduction We have talked about the classification of parallel computers and their interconnection networks in that order in units 2 and

Finest way to get a reference to the viewport, What is finest way to get a ...

What is finest way to get a reference to the viewport from anywhere in the code? Ans) You can use refs config to set a reference on the Application/Controllers

What are the types of parallel programming, Q. What are the types of parall...

Q. What are the types of parallel programming? There are various parallel programming models in general use. A few of them are:  Data Parallel programming Message P

What is verilog case 1, What is verilog case (1) ? wire [3:0] x; al...

What is verilog case (1) ? wire [3:0] x; always @(...) begin case (1'b1) x[0]: SOMETHING1; x[1]: SOMETHING2; x[2]: SOMETHING3; x[3]: SOMETHING4; endcase

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd