What is the key to avoiding selection bias

Reference no: EM131926806

Problem

Suppose you try one thousand configurations of the same investment strategy, and perform a CV on each of them. Some results are guaranteed to look good, just by sheer luck. If you only publish those positive results, and hide the rest, your audience will not be able to deduce that these results are false positives, a statistical fluke. This phenomenon is called "selection bias."

(a) Can you imagine one procedure to prevent this?

(b) What if we split the dataset in three sets: training, validation, and testing? The validation set is used to evaluate the trained parameters, and the testing is run only on the one configuration chosen in the validation phase. In what case does this procedure still fail?

Reference no: EM131926806

Questions Cloud

Compute the rolling standard deviation of the sampled bars : Sample bars using the CUSUM filter, where {yt} are absolute returns and h = 0.05. Compute the rolling standard deviation of the sampled bars.

Compute the rolling standard deviation of two-sampled series : Compute the rolling standard deviation of the two-sampled series. Which one is least heteroscedastic? What is the reason for these results?

What method achieves the lowest test statistic : Apply the Jarque-Bera normality test on returns from the three bar types. What method achieves the lowest test statistic?

What would be the mathematical grounds for disregarding : Suppose that you develop a momentum strategy on a futures contract. What would be the mathematical grounds for disregarding the second result, if any?

What is the key to avoiding selection bias : What if we split the dataset in three sets: training, validation, and testing? What is the key to avoiding selection bias?

Why does shuffling defeat the purpose of k-fold cv : Why is shuffling a dataset before conducting k-fold CV generally a bad idea in finance? Why does shuffling defeat the purpose of k-fold CV in financial dataset?

What can you conclude about productivity improvement : Last year, Chop-M-Up, Inc. implemented a new labor process and redesigned its product, hoping to increase input usage efficiency.

Should the proposed discount be offered : The firm's current average collection period is 60 days, sales are 40,000 units, selling price is $45 per unit, and variable cost per unit is $36.

What break date does the method select : Compute the SDFC (Chow-type) explosiveness test. What break date does this method select? Is this what you expected?

User Account

All Pages