New posts

Reply to thread

Message

Nick,

Thanks for your explanation.

Not sure if you're referring to my recent post here, but I certainly didn't say or conclude that.

My conclusion was essentially that one needs to be wary of backtesting over conditions that rarely occur in the universe being tested.

That is perhaps the significant point. With a robust system it shouldn't matter, but on the road to developing a robust system you need to be sure that your backtesting is providing reasonably reaslistic results, otherwise you might end up thinking you have a profitable and robust system when in fact you haven't.

As an example, during testing of a system under development I was using the ASX300 as the universe and getting some pretty good results, even with Monte Carlo testing. However, on closer inspection of the results I noticed that the main winning trades were ones purchased at very low share prices, which would have mostly been before those stocks were in the ASX300. As an attempt to closer simulate the current ASX300 I then set a minimum share price of $2. Running the same backtests again showed a drop in return to almost nothing. That's the sort of thing I'm suggesting one needs to be wary of.

Cheers,
GP

<blockquote data-quote="GreatPig" data-source="post: 177639" data-attributes="member: 56">Nick,Thanks for your explanation.Not sure if you&#039;re referring to my recent post here, but I certainly didn&#039;t say or conclude that.My conclusion was essentially that one needs to be wary of backtesting over conditions that rarely occur in the universe being tested.That is perhaps the significant point. With a robust system it shouldn&#039;t matter, but on the road to developing a robust system you need to be sure that your backtesting is providing reasonably reaslistic results, otherwise you might end up thinking you have a profitable and robust system when in fact you haven&#039;t.As an example, during testing of a system under development I was using the ASX300 as the universe and getting some pretty good results, even with Monte Carlo testing. However, on closer inspection of the results I noticed that the main winning trades were ones purchased at very low share prices, which would have mostly been before those stocks were in the ASX300. As an attempt to closer simulate the current ASX300 I then set a minimum share price of $2. Running the same backtests again showed a drop in return to almost nothing. That&#039;s the sort of thing I&#039;m suggesting one needs to be wary of.Cheers,GP</blockquote>

[QUOTE="GreatPig, post: 177639, member: 56"] Nick, Thanks for your explanation. Not sure if you're referring to my recent post here, but I certainly didn't say or conclude that. My conclusion was essentially that one needs to be wary of backtesting over conditions that rarely occur in the universe being tested. That is perhaps the significant point. With a [i]robust[/i] system it shouldn't matter, but on the road to developing a robust system you need to be sure that your backtesting is providing reasonably reaslistic results, otherwise you might end up thinking you have a profitable and robust system when in fact you haven't. As an example, during testing of a system under development I was using the ASX300 as the universe and getting some pretty good results, even with Monte Carlo testing. However, on closer inspection of the results I noticed that the main winning trades were ones purchased at very low share prices, which would have mostly been before those stocks were in the ASX300. As an attempt to closer simulate the current ASX300 I then set a minimum share price of $2. Running the same backtests again showed a drop in return to almost nothing. That's the sort of thing I'm suggesting one needs to be wary of. Cheers, GP [/QUOTE]

Verification