Guest Post by DV 34

No matter what software or manual system you use to backtest trading strategies, it's important to focus on the statistics that provide the best insight into the relationship between risk and reward as well as trading opportunity.

In the next of his series of articles on the practical aspects of backtesting DV34 runs through the statistics he sees as important and why.

Backtesting Statistics

As per my original backtesting article the next phase of manual backtesting involves pulling out the statistics. Heading back to this article, extracting statistics involves some or all of the following:

Here is a clearer look at the statistics,

D) EXTRACTING KEY STATISTICS YOU WANT FROM THE TEST

  • The basic statistics

  1. The importance of exits… not entries
  2. No of Trades (Preferably 50-100+) more = better
  3. Win% - All Trades
  4. "R" Multiples (Reward to Risk Ratios - positive or negative)
  5. Avg. Reward to Risk over all Trades
  6. Expectancy over all trades
  7. Std Deviation for all "R" Multiples
  8. SQN number or Ratio of Expectancy/ Std Deviation
  9. Maximum Drawdown during test in terms of "R" Multiples
  10. % Long Trades, % Short Trades
  11. Avg. no of days in trade (carry costs?)
  12. No. opportunities over a set period (i.e. year, month, week etc)
  13. Expectunity
  14. Any others not listed you feel are important 

To recap where we left off in the last article – we summarised the trades together into a single R:R ratios from 03/01/2001. Below is the table of results for this AUDJPY test showing the last trade date 17/01/2013, So it has weathered 12-13 years of various conditions

Backtesting Statistics Summary Click to Enlarge Backtesting Statistics Summary
Click to Enlarge

Here is a clearer look at the statistics,

Key backtesting statistics Key backtesting statistics

I have shown 29 statistics above that I pulled out of the simple test above, I think in reality though you really only need about 6-8, or even 3 if you like to keep it simple!

Key statistics for backtesting trading strategies

I believe the key statistics you need are:

1)                  No Trades (must be greater than 30 – pref. much more)

2)                  No. Trades/ Year/ Month (whatever time period is relevant)

3)                  % Long Trades/ % Short Trades (is there a bias?)

4)                  Win%/ Success Rate

5)                  Average R:R

6)                  Expectancy (Average profit per dollar risked) – Must be a positive number!

7)                  Avg. no days in trade (Carry costs/ duration exposed to risk)

8)                  Maximum R multiple drawdown measured

9)                  Min/ Max/ Avg. Pips Risked

Backtesting Strategy Results

The results from this test were:

1)                   215                 = Total Trades

2)                   18.1                = Avg. trades/ year

3)                   56.7%             = Long Trades/ 43.3% Short    (Performs much better long)

4)                   61.9%             = Winning % (Higher for longs)

5)                   2.03                = Average R:R Ratio

6)                   0.8788            = Expectancy

7)                   1.9                  = Avg. days in trade

8)                   -4.9                 = Maximum measured ‘R’ multiple drawdown

9)                  10pips              = Min. Pips Risked

193pips            = Max. Pips Risked

40.7                 = Avg. Pips Risked

What you don’t see in the results is that 22-30% of the setups spotted were either missed or invalidated and therefore cancelled orders - which is also valuable information

This was consistent across timeframes and pairs and a natural side effect of the strategy itself as it was trading corrections only.

The key downside to this strategy was simply the number of opportunities

Across 1hr, 4hr, daily charts there was an average of 30 trades/ year on the Aud/jpy, and on average the winning % remained very similar but interestingly the expectancy increased significantly on the higher timeframes from 0.87 – 2.15 but the number of trades also decreased significantly as well…

I looked at the 15min on the Audusd only, mainly because after 580x trades (1,100 orders) I thought I would save my sanity and do the higher timeframes first, the 15min chart added another ~30x trades/ year

I had a quick look at time of day, day of week or even the month to see if there were any distinguishing periods that were more or less favorable - I must note there are really not enough trades for this to be taken too seriously

Trading Strategy Expectancy - time of day Trading Strategy Expectancy - time of day

Trading Strategy Expectancy - Day of Week Trading Strategy Expectancy - Day of Week

Trading strategy expectancy - month Trading strategy expectancy - month

It appears that trades triggered during the Tokyo – London sessions work slightly better than the US session as well as trades initiated on Monday - Wednesday.

The monthly performance shows March, July and December as poor performing months – however, the number of trades in each month was barely enough to be viable so should be taken with a grain of salt.

To give you an idea of where the back test puts this strategy (assuming it works similarly going forward) I have marked the overall expectancy it on an expectancy chart below which compares Winning % (vertical axis) vs Reward to Risk (horizontal axis)

This shows the potential margin of safety, to allow for carry costs/ slippage and/or poor entries/ exits.

The pink zone is where you can earn 4-5% per year in a savings account with very low risk, the other colour zones are in 0.5 increments

Trading success ratio v reward to risk Trading success ratio v reward to risk

The green oval/ box is the equivalent expectancy of a 70% winning system with 1:1 R:R targets @ ~0.40

The red oval boxes is the approximate expectancy of this tested system @ ~0.87

The blue box is an estimate of Larry Williams system expectancy which is nearly double

I want to emphasize that back testing is far from infallible and will not guarantee profits, this looks ok on paper but will be different when traded live

Testing is only a guide to give some confidence as to whether to pursue a strategy any further or not… and perhaps even save you some time / money and headaches in the future

As a side note, I have noted expectancy a lot above, but expectancy on its own does not show the complete picture because it negates the number of opportunities

I believe that combining expectancy with expectunity (avg. trade expectancy x no. opportunities) over a set period (month/year?) and also the average duration of trades (days/ weeks etc) would probably be a fairer measure to compare two strategies side by side apples with apples

We will cover position sizing in the next article and what that does to your strategy and also look at the ‘risk of ruin’ and see if we can define an acceptable position sizing strategy for my objectives based on how it performs.

Hope this helps,

DV34