Guest Post by DV 34

Here is the second in Guest Blogger DV34's series of articles on his experience with back testing.

In this article DV looks at the pluses and minuses of back testing and at what a strategy's percentage of winning trades is likely to mean for your worst case horror stretch for number of  losing trades in a row

  

Continuing on from the introduction I thought it would be useful to outline the basic principals, key concepts and how each of the components effectively fits into the grander scheme of things - at least from my perspective. 

5)                  TESTING & VERIFICATION

A)    THE BASICS 

i)                    Why Backtest? Pro's vs. Con's 

Back testing involves studying your strategy’s performance over historical data, under the premise that how it performed in the past - will likely reflect how it ‘may’ perform in the future.

The Pro’s

  • You learn if the system produces a net profit and possibly has a positive mathematical edge
  • You obtain statistics on how it performs over time, ideal market conditions etc
  • You can use the statistics to determine a realistic expectation: -  Number of trades per year (month/ week etc), possible draw downs, average return etc
  • If performing manual back testing you rehearse/ practice executing the strategy 100’s if not 1,000’s of times and learn nuances through observation,

(This is sometimes not the case with automated strategies which can data mine independently)

  • Can verify third party claims for a bought system
  • You get to test ALL setups, and find out if you can trade this effectively with available time at screen 

The Con’s

  • The data may be poor quality and/ or wrong time zone, include/ exclude daylight savings time weekend adjustments. Or does not reflect your brokers pricing which is effectively what you will be trading (As I found out the hard way, 3 months testing and an automated code later…)
  • May not include spread, brokerage, slippage and carrying costs for leveraged interest bearing instruments (cfd’s)
  • Does not account for major news events that may prevent you from trading according to your rules i.e. NFP release..?
  • Psychological Biases -  which can overstate results including  hindsight bias, data mining, curve fitting etc
  • Does not reflect your ability to execute the strategy well, or how you perform as a trader
  • Is time intensive and difficult to do well 

ii)                  Embracing uncertainty - it's all about probabilities - why they matter...

 Trading strategies work on the premise that the outcomes of individual trades are unknown.

Most traders acknowledge this fact and accept there are no strategies that win 100% of the time.  Therefore all strategies have a degree of uncertainty 

Therefore strategies must rely on statistical edges that are based on a large sample of trades in order to deduce if they will make money over the long haul using probability and statistics. 

The only things we can control in trading are our timing of entries/ exits and the level of risk we expose ourselves to; both at the trade level and at the overall portfolio level. 

iii)                What are my objectives? Goals? Desired outcomes?  

Goals and objectives vary by individual but they should be very specific and written down as they guide your overall direction and focus. 

To give an indication, my own shorter term trading (intraday/ swing) objectives are:

To make as much money as my system will allow while limiting my maximum drawdown to -20% with a full trade review if this reaches -15%

(A drawdown is the peak to trough percentage loss of your account from the most recent all time high)

My ideal system would have an annual ‘gain to pain’ ratio (annual return/ maximum drawdown) of 2:1 or higher if possible 

iv)                What is my risk tolerance! How can I limit or mitigate it? 

As stated above, we can control the level of risk we take, but only if we understand how our strategy performs, we can see examples of this at the end of this article 

For myself, I get uncomfortable with draw downs >15% and have had a drawdown in the past of 27% with a series of 12 losses in a row… so these things can and do happen… 

Defining your risk tolerance is important in my view because most people believe they can handle greater losses than they actually can before emotions start to creep in. 

A good rule of thumb is if you think you will be ok with a 20-30% drawdown (or more) then halve it and position size accordingly!

ii)                  Dr Van Tharp's Approach - Using statistics to quantify and measure a Trading Strategy  

Dr Van Tharp covers the principals of trading in many of his great books, including ‘Trade your way to Financial Freedom’, ‘Super Traders’ and ‘The definitive guide to position sizing strategies’ but the basic premise is that most strategies results can be boiled down to some key basic statistics. 

Statistics remove the focus away from dollars/ pips/ points etc and focus purely on key ratios and percentages, these tell you if you have a positive edge in the market.. 

The key basic statistics are the average reward to risk ratio and the winning % - which make up Expectancy 

Van Tharps own website is probably the best resource to read more on these principals and concepts if you are unfamiliar with them, you can find them here: 

http://www.vantharp.com/tharp-concepts.asp

This will explain all of the following and more:

a)      Risk and R-Multiples

b)      Money Mangement/ Position Sizing

c)      Expectancy

d)     System Quality Number (SQN) 

The only important thing to note is that in back testing you need at least 30 trades to be statistically relevant although Van Tharp recommends between 100-200 trades 

ii)                  Winning % 

Win% is simply the number of (winning trades/ total trades taken) * 100 

Win% on its own it doesn’t tell us anything about the systems profitability, the only information that is actually useful from the win% alone is:

a)      A high win% gives a smoother equity curve (although not necessarily positive)

b)      A low win% gives a more volatile equity curve/ much larger draw downs

c)      The win% can give an indication of the probability of strings of losses you can expect using Monte Carlo simulations 

iii)                Time of Day - Can I trade it?  

For discretionary traders this is important, as our setups could form at times when we are unavailable and unable to execute a trade, this can also affect the overall results 

iv)                Quantifying and defining a comfortable level of risk/ setting reasonable boundaries/ limits around your strategy  

If like me, you get uncomfortable with >15% draw downs then your objective would be to ensure your position sizing strategy limits the overall maximum drawdown to <15% 

Seeing that any back testing only shows one series of wins vs. losses, it may not truly reflect what could happen in the future as the sequence of wins vs. losses will be different.

One way to counteract this is by using your win% in a monte carlo simulator that can give you some statistical evidence of the likelihood of having a series of losses. 

Monte Carlo simulations simply scramble the order of events repeatedly and plot the results in an effort to determine the likely zone that they will occur in and how sensitive they are to the inputs. 

An example of a monte carlo simulation is shown here except it is for an equity curve vs. a series of wins vs. losses (the concept is the same either way)

Source: Google Images 05/06/2013 CLICK ON CHART TOO ENLARGE Source: Google Images 05/06/2013
CLICK ON CHART TOO ENLARGE

  

There is a very useful free excel spreadsheet I found online that allows you to put your win% in and test for different streaks of losses in a row: Source: http://www.daytradinglife.com/ 

For a 30% Winning system, there is a ~1% probability of a 25 trade losing streak! (over 250 trades) and a 63% chance of having 2x streaks of 10 losses in a row

CLICK ON CHART TOO ENLARGE CLICK ON CHART TOO ENLARGE

For a 70% winning system, there is a ~1% probability of a 8 trade losing streak (over 250 trades)

And a 40% chance of having 2x streaks of 4 losses in a row

Click on chart to enlarge Click on chart to enlarge

So based on this information you can see that over 250 trades you could have:

-          A ~1% chance of 25 losses in a row in a 30% winning system. Or,

-          A ~1% chance of 8 losses in a row in a 70% winning system

….which is nice to know in advance…!! 

Although this is more of a guide and there are other (and arguably better) tools than Monte Carlo simulations, it can provide some insight as to how this will impact your account with various position sizes etc 

ii)                  Applying position sizing strategies to meet objectives  

Position sizing is really the key to limiting downside risk and/ or upside potential - but only when you understand how your strategy performs

 In the next article we will start getting into my actual test to show how I have done it, which is much easier to do in practice than to explain on paper..

 Hope it helps, 

DV34