Example calculation of entropy, Computer Engineering

Assignment Help:

Example Calculation:

If we see an example we are working with a set of examples like S = {s1,s2,s3,s4} categorised with a binary categorisation of positives and negatives like that s1  is positive and the rest are negative. Expect further there that we want to calculate the information gain of an attribute, A, and  A can take the values {v1,v2,v3} obviously. So lat in finally assume that as: 

1745_Example Calculation of Entropy.png

Whether to work out the information gain for A relative to S but we first use to calculate the entropy of S. Means that to use our formula for binary categorisations that we use to know the proportion of positives in S and the proportion of negatives. Thus these are given such as: p+ = 1/4 and p- = 3/4. So then we can calculate as: 

Entropy(S) = -(1/4)log2(1/4) -(3/4)log2(3/4) = -(1/4)(-2) -(3/4)(-0.415) = 0.5 + 0.311

= 0.811 

Now next here instantly note that there to do this calculation into your calculator that you may need to remember that as: log2(x) = ln(x)/ln(2), when ln(2) is the natural log of 2. Next, we need to calculate the weighted Entropy(Sv) for each value v = v1, v2, v3, v4, noting that the weighting involves multiplying by (|Svi|/|S|). Remember also that Sv  is the set of examples from S which have value v for attribute A. This means that:  Sv1 = {s4}, sv2={s1, s2}, sv3 = {s3}. 

We now have need to carry out these calculations: 

(|Sv1|/|S|) * Entropy(Sv1) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

(|Sv2|/|S|) * Entropy(Sv2) = (2/4) * (-(1/2)log2(1/2) - (1/2)log2(1/2))

                                      = (1/2) * (-(1/2)*(-1) - (1/2)*(-1)) = (1/2) * (1) = 1/2 

(|Sv3|/|S|) * Entropy(Sv3) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

Note that we have taken 0 log2(0) to be zero, which is standard. In our calculation,

we only required log2(1) = 0 and log2(1/2) =  -1. We now have to add these three values together and take the result from our calculation for Entropy(S) to give us the final result: 

Gain(S,A) = 0.811 - (0 + 1/2 + 0) = 0.311 

Now we look at how information gain can be utilising in practice in an algorithm to construct decision trees.


Related Discussions:- Example calculation of entropy

Explain the state space of search problem, Question: (a) Assume you are...

Question: (a) Assume you are a taxi driver. Your taxi can hold four passengers. Passengers pay a flat fee for a ride to the airport, so your goal is to pick up four passengers

What is electronic cash, What is electronic cash? E-cash is cash which ...

What is electronic cash? E-cash is cash which is shown by two models. One is the on-line form of e-cash which permits for the completion of all types of internet transactions.

Resetting the computer, Sometime you may have to reset your computer (i.e.,...

Sometime you may have to reset your computer (i.e., Reboot DOS) when it is still running because DOS does not work accurately. To reset your computer you have two choices: 1. P

What are the 3 things that are needed to specify a route, 1. URL Pattern - ...

1. URL Pattern - You can contain placeholders in a URL pattern so that variable data can be passed to the request handler without having a query string. 2. Handler - The handle

Cim and holonic manufacturing system, CIM and Holonic Manufacturing System ...

CIM and Holonic Manufacturing System The improvement of computer integrated manufacturing has demonstrated that the automation has the same potential such that of computers in

What are delay systems in telecommunication networks, What are delay system...

What are delay systems in telecommunication networks? Delay System: A class of telecommunication networks like data a network that places the call or message arrivals in a qu

Program accept decimal number and convert it into given base, Q Develop a p...

Q Develop a program that accepts decimal number and converts it into given base. Fractional numbers are allowed and negative numbers are not allowed. Also check that entered number

Explain about the data logging, Data logging  This technique involves ...

Data logging  This technique involves collecting data automatically using sensors; frequently used when doing scientific experiments or monitoring a system.

Explain the working of a weighted register d/a converter, With the help of ...

With the help of a neat diagram, explain the working of a weighted-resistor D/A converter. Ans Weighted Register D/A Converter:   Digital input that has 4 bits

What are the features of client/server computing, What are the features of ...

What are the features of Client/Server Computing? Although there are several different configurations, different hardware and software platforms and even dissimilar network pro

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd