Time Machine Test (Part 1)

This series of post is based on CSS Analytics’ Time-Machine post series. I recommend you read it, since I this is not my original idea. However, I do think that my implementation differs slightly from CSS Analytics’.

The following results represent backtests using my version of the algorithm on different equity indices on all free historical data available for each ticker. I used index data since the available data points go way further in history. I did test the algorithm on available ETFs with similar results. All results presented below are frictionless.

S&P 500

RUSSELL 2000

NASDAQ 100

S&P/TSX Comp. (Can)

FTSE 100 (U.K.)

NIKKEI 225 (JAP)

HANG SENG (CHN)

Summary

As the results above show, the algorithm is quite robust and does adapt fairly well to different market regimes as well as to the differences in market behavior for all these indices. Regardless, I think this is a very nice and simple concept that can still be improved.

I will try several modifications to the algorithm in the next few posts or so. For now you can expect tests on different asset classes: commodities, futures, currencies, etc. I also want to try to replace the t-test with a non-parametric Wilcoxon signed-rank test and see how the strategy performs when we get rid of the normality assumption when testing for significance. I also have other ideas in mind to improve the algorithm, stay tuned!

QF

Advertisements

22 thoughts on “Time Machine Test (Part 1)”

  1. Hello Quantum Financier, congratulations on starting a very interesting blog! Excellent research! Your results seem to be much more robust… in what principal ways is your algorithm / implementation different?
    Thank you, Anton

    1. Thank you Anton for the kind word. To answer your question it is hard to pinpoint the specific differences since the actual spreadsheet or code was never made public by CSS Analytics. I took the basic idea of creating a matrix of run length and went from there, I followed the workflow explained in the posts linked above. The main areas of differences would probably be where CSS less defined it’s method. The goal for this first post was to reproduce the results and to build on from what I have. I will go more into the specifics when I make improvements to the strategy. The way I allocate capital between significant signals is probably the only thing I can say for sure differs from Mr. Varadi’s implementation. I hope this helps.
      QF

      1. Thank you for sharing with us nice results. What would have to happen in osrder to share with us some more details, because I would really like to test the approach myself but can not make enough sense out of CSS Analytics posts. Thanks in advance for sharinig some more details about your algorithm/approach.

        1. Hello Samo,
          To answer you question, CSS Analytics would have to release the code for the strategy himself. This strategy is not my idea and I don’t want to post code for a strategy the author wanted unpublished. In future posts, I will probably build on the strategy and will publish the parts of code or the ideas that are mine.
          QF

  2. Hey Quantum Financier, just found you blog from a reference at Marketsci and it look very promising!
    I did my own test of the Time machine and my results are no way as good as yours. It would be very interesting to see how you performed the test in excel, is it possible that you could share the spreadsheet or maybe some code?
    Any way, keep up the good work and best wishes from Sweden!

    1. Thank you for the kind word Emil, I appreciate the feedback. I don’t think I will share my code on the blog, since I want to respect CSS Analytics choice not make the strategy code public, after all this is not my original idea. What I will do, is post code when I do modifications to the strategy to try and make it better since this part will entirely be my idea. If you want to send me what you did, I can compare my code to yours and tell you what difference I have, perhaps this is a good compromise.
      Best, QF

      1. “Hey Ma, look at this great thing I found” but no you can’t create it and I won’t talk about anything but my results… if these are to be true discussions rather than a telling, what gives, besides self promotion?

        1. Ian, read the post, never do I say that I found that, all I say is that I reproduced CSS Analytics’ results. He did not publish the results on his blog, so why would I publish it for him. He owns the idea for the strategy and I respect his will to not publish code or spreadsheet. Don’t forget it is a zero sum game we play. Look at the big blogs in quant finance, when they get results that are somewhat good enough to make them trade the strategy after a few modification you will never find the strategy thoroughly explained. If I was able to reproduce the results so should you.
          Best,
          QF

    2. Emil, det vore kul om vi kunde diskutera detta!
      Skicka mig ett email på christian_rolfsson at yahoo punkt se

  3. hi QF nice blog and quite interesting research so far. I was wondering how you were decaying past results so that the more recent period is more prominent in the learning. how far back do you look back at each point? Thanks and keep up the good work

    1. Hi red, thanks for the comment. At this point I do not decay the weights per se, as I just wanted to reproduce CSS Analytics results to introduce the strategy. In the near future I plan to try several modifications, adding a decay rate is on my radar. Right now, the algorithm use an average between 1 and 3 years significance on a rolling basis, all observations are weighted the same.
      QF

  4. QF-Quick question. I’m a little unsure about how to replicate the t-test on the 50 strategies. Are you 1) looking for instances in the data set where the 50 strategies occur on a day-by-day basis (dynamic) or 2) analyzing each 5 day period from Monday-Friday and then performing a t-test (static)?

    Freeman

    1. Hello Freeman,

      For this simple test, the t-test is dynamic, we also only count trades for the strategy (i.e. not when the strategy is in cash)

      Thank you for your question
      QF

      1. Hi QF,

        How do you handle strategies that trade at different frequencies when evaluating their past 5-trade performance?

        ciao,
        wilson

  5. Hi QF,
    When you trade N strategies simultaneously, how is the capital allocated between these “active” strategies? Is 1/N of the capital given to each strategy, or is the capital divided between the number of signals given for each day?

    (BTW, how much time does your algorithm take to compute the results for one index in R? It took about 84 seconds for mine to just select the active strategies over 1500 days of data in Matlab.)
    Wilson

    1. Hello Wilson,

      For these results, the capital is allocated between all active strategies 1/n. However this is flexible and ca be improved. For your speed question the code is pretty fast takes about 45 second to compute and plot the backtest. Thanks for your questions.

      QF

  6. Hi QF, One more related question. Out of the 50 strategies, what happens if a strategy gives an entry signal while it isn’t in cash (i.e. entry->entry->exit)? Do you start a new position, or the second entry is simply ignored?
    Wilson

      1. You mentioned that the look back period for testing significance varies between 1 and 3 years. I’m guessing there is a specific number of past trades you used for the t-test?
        (If the t-test isn’t based on the number of past trades, how does the algorithm adjust the look back period?)
        Thank you for the responses.
        Wilson

Comments are closed.