Example calculation of entropy, Computer Engineering

Assignment Help:

Example Calculation:

If we see an example we are working with a set of examples like S = {s1,s2,s3,s4} categorised with a binary categorisation of positives and negatives like that s1  is positive and the rest are negative. Expect further there that we want to calculate the information gain of an attribute, A, and  A can take the values {v1,v2,v3} obviously. So lat in finally assume that as: 

1745_Example Calculation of Entropy.png

Whether to work out the information gain for A relative to S but we first use to calculate the entropy of S. Means that to use our formula for binary categorisations that we use to know the proportion of positives in S and the proportion of negatives. Thus these are given such as: p+ = 1/4 and p- = 3/4. So then we can calculate as: 

Entropy(S) = -(1/4)log2(1/4) -(3/4)log2(3/4) = -(1/4)(-2) -(3/4)(-0.415) = 0.5 + 0.311

= 0.811 

Now next here instantly note that there to do this calculation into your calculator that you may need to remember that as: log2(x) = ln(x)/ln(2), when ln(2) is the natural log of 2. Next, we need to calculate the weighted Entropy(Sv) for each value v = v1, v2, v3, v4, noting that the weighting involves multiplying by (|Svi|/|S|). Remember also that Sv  is the set of examples from S which have value v for attribute A. This means that:  Sv1 = {s4}, sv2={s1, s2}, sv3 = {s3}. 

We now have need to carry out these calculations: 

(|Sv1|/|S|) * Entropy(Sv1) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

(|Sv2|/|S|) * Entropy(Sv2) = (2/4) * (-(1/2)log2(1/2) - (1/2)log2(1/2))

                                      = (1/2) * (-(1/2)*(-1) - (1/2)*(-1)) = (1/2) * (1) = 1/2 

(|Sv3|/|S|) * Entropy(Sv3) = (1/4) * (-(0/1)log2(0/1) - (1/1)log2(1/1)) = (1/4)(-0 -

(1)log2(1)) = (1/4)(-0 -0) = 0 

Note that we have taken 0 log2(0) to be zero, which is standard. In our calculation,

we only required log2(1) = 0 and log2(1/2) =  -1. We now have to add these three values together and take the result from our calculation for Entropy(S) to give us the final result: 

Gain(S,A) = 0.811 - (0 + 1/2 + 0) = 0.311 

Now we look at how information gain can be utilising in practice in an algorithm to construct decision trees.


Related Discussions:- Example calculation of entropy

How to set cell properties in dreamweaver, Q. How to Set Cell Properties in...

Q. How to Set Cell Properties in dreamweaver? Cell height determines the height of space within a cell row. Notice the space around Compass logo. You want to remove the extra s

Illustrate the advantages of encapsulation, Advantages of Encapsulation ...

Advantages of Encapsulation You can also delay the resolution of the details until after the design.  You can keep your code modular.

What are the roll and page areas, What are the roll and page areas? Ro...

What are the roll and page areas? Roll and page areas are SAP R/3 buffers used to kept user contexts (process requests).  The SAP dispatcher assigns procedure requests to work

Logic-based expert systems, Logic-based Expert Systems : Expert system...

Logic-based Expert Systems : Expert systems are agents that are programmed to make decisions just about real world situations are place together by utilising knowledge illicit

Minimum total number of shelv, Write a program that finds the minimum total...

Write a program that finds the minimum total number of shelv, C/C++ Programming

EDC, Conparision of masfet and jfet

Conparision of masfet and jfet

Explain height of a tree, Height of a tree The length of the longest pa...

Height of a tree The length of the longest path from root to any node is called as the height of the tree.

Data warehouse, 1) Define a job scheduling strategy that will meet business...

1) Define a job scheduling strategy that will meet business requirement of reporting availability by 6am CST for the following cubes? Show the job scheduling dependencies in a pict

different versions of microsoft access to open the database, What format s...

What format should my database be saved in to allow different versions of Microsoft Access to open the database? Ans) MS Access is not backwards compatible. To share tables amon

Show the mips addressing modes, Q. Show the MIPS Addressing Modes? MIP...

Q. Show the MIPS Addressing Modes? MIPS Addressing Modes MIPS employs various addressing modes: 1. Uses Register as well asimmediate addressing modes for operations.

Write Your Message!

Captcha
Free Assignment Quote

Assured A++ Grade

Get guaranteed satisfaction & time on delivery in every assignment order you paid with us! We ensure premium quality solution document along with free turntin report!

All rights reserved! Copyrights ©2019-2020 ExpertsMind IT Educational Pvt Ltd