Example calculation of entropy, Computer Engineering

Assignment Help:

Example Calculation:

If we see an example we are working with a set of examples like S = {s1,s2,s3,s4} categorised with a binary categorisation of positives and negatives like that s1  is positive and the rest are negative. Expect further there that we want to calculate the information gain of an attribute, A, and  A can take the values {v1,v2,v3} obviously. So lat in finally assume that as: 

1745_Example Calculation of Entropy.png

Whether to work out the information gain for A relative to S but we first use to calculate the entropy of S. Means that to use our formula for binary categorisations that we use to know the proportion of positives in S and the proportion of negatives. Thus these are given such as: p+ = 1/4 and p- = 3/4. So then we can calculate as: 

Entropy(S) = -(1/4)log2(1/4) -(3/4)log2(3/4) = -(1/4)(-2) -(3/4)(-0.415) = 0.5 + 0.311

= 0.811 

Now next here instantly note that there to do this calculation into your calculator that you may need to remember that as: log2(x) = ln(x)/ln(2), when ln(2) is the natural log of 2. Next, we need to calculate the weighted Entropy(Sv) for each value v = v1, v2, v3, v4, noting that the weighting involves multiplying by (|Svi|/|S|). Remember also that Sv  is the set of examples from S which have value v for attribute A. This means that:  Sv1 = {s4}, sv2={s1, s2}, sv3 = {s3}. 

We now have need to carry out these calculations: 

(|Sv1|/|S|) * Entropy(Sv1) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

(|Sv2|/|S|) * Entropy(Sv2) = (2/4) * (-(1/2)log2(1/2) - (1/2)log2(1/2))

                                      = (1/2) * (-(1/2)*(-1) - (1/2)*(-1)) = (1/2) * (1) = 1/2 

(|Sv3|/|S|) * Entropy(Sv3) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

Note that we have taken 0 log2(0) to be zero, which is standard. In our calculation,

we only required log2(1) = 0 and log2(1/2) =  -1. We now have to add these three values together and take the result from our calculation for Entropy(S) to give us the final result: 

Gain(S,A) = 0.811 - (0 + 1/2 + 0) = 0.311 

Now we look at how information gain can be utilising in practice in an algorithm to construct decision trees.


Related Discussions:- Example calculation of entropy

Determine if the product of two integers are even, If the product of two in...

If the product of two integers a and b is even after that prove that either a is even or b is even. Answer: It is illustrated that product of a and b is even so let a * b = 2n.

Determine the term- security, Determine the term- Security When using ...

Determine the term- Security When using Internet, security can be enhanced using encryption. Debit and credit card transactions can also be protected by a specific type of pas

Processors hypercube and utilisation displays, Processors Hypercube Thi...

Processors Hypercube This is specific to in the hypercube: Here, every processor is depicted by the set of nodes of the graph and the several arcs are represented with communic

Describe how the it infrastructure is designed, IT Management 1. Descri...

IT Management 1. Describe how the IT infrastructure is designed. 2. Explain briefly the audit planning phase in IT Audit 3. Explain localized and distributed load balanci

Symbol for the new euro currency in my spreadsheet, How can I go in the sym...

How can I go in the symbol for the new Euro currency in my spreadsheet? Ans) Microsoft suggestted a new Tahoma font with the Euro symbol.

Illustrate about fifth generation electronic computers, Fifth Generation (1...

Fifth Generation (1984-1990) The advancement of the next generation of computer systems is characterized majorly by the acceptance of parallel processing.  Until this time para

Advantages and disadvantages of public-key cryptography, Advantages and Dis...

Advantages and Disadvantages of Public-key cryptography Advantages (i) Enhanced security and convenience  (ii) Electronic records may be authenticated by affixing

Single instruction and single data stream (sisd), Single Instruction and Si...

Single Instruction and Single Data stream (SISD) In this organisation, sequential implementation of instructions is executed by one CPU having a single processing element (PE

Illustrate about first generation computers, Q. Illustrate about First Gene...

Q. Illustrate about First Generation Computers? It is certainly ironic that scientific inventions of great impact have frequently been linked with supporting a very sad as well

Name the concepts of object oriented, Name the concepts of Object oriented ...

Name the concepts of Object oriented There are various concepts of object oriented analysis Abstraction, Encapsulation, Identity, Modularity, Hierarchy, Typin

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd