Comments on Max Dama on Automated Trading: Decision Tree Bagging System (R code)

Andrew, Please see the update I just posted at t...

2010-01-28T12:15:14.660-08:00

Andrew,

Please see the update I just posted at the top of the page.

Regards,
Max

Max, Thanks for posting this interesting example....

2010-01-28T10:09:50.420-08:00

Max,

Thanks for posting this interesting example. I would like to play with it using R (which I'm still learning) but I noticed the links to the R code, data and harness file are broken. Would you have the sample files available via other links?

warm regards,

Andrew

Thanks Chris.

2009-12-06T18:45:21.979-08:00

Thanks Chris.

Hi Max, thanks for sharing your insights on your ...

2009-12-03T05:03:27.289-08:00

Hi Max,

thanks for sharing your insights on your blog. I've been working with randomForest and other classification algos since many years (having a background in biostatistics, where these methods have been used for quite some time before they became famous in finance).

I compared my code with yours (my financial time-series are ordered past-to-present), using your gold data, and I think your R code doesn't do what you want it to do (I use R 2.9.1):

in the following lines in your code:

idx = rev(data.frame(targets = seq(1, btsamples*skip, skip)))
idx = data.frame(idx, data = idx$targets+horizon)

because the rev() command doesn't work as you want it (it does not reverse the rows in the data.frame), later on in your code the time-axis of 'preds' is wrong (still ordered from present to past, instead from past to present), yielding problems during the creation of the equities time-series (using cumprod()), where you do reverse the order of 'returns'.

so I suggest to replace those two lines above with something like (the idx$data you never use in your code, so I drop it
):

idx = data.frame(targets = rev(seq(1, btsamples*skip, skip)))

Tell me, if I'm wrong, but I think, this is the way, it's supposed to be.

Cheers, Chris

Henning, Definitely. Hopefully we've done enough ...

2009-11-16T13:52:02.692-08:00

Henning,

Definitely. Hopefully we've done enough regularization that we haven't allowed ourselves to overfit too much, but only out of sample testing can really show how it works.

Thanks for the compliment, I'm glad it's useful.

Regards,
Max

Hi Max, love your blog. Really great content... J...

2009-11-16T13:48:23.181-08:00

Hi Max,
love your blog. Really great content...

Just a tiny question.
Isn't this decision tree bagging system data snooping par excellence? It fit's parameters in the best possible way to explain the sample but what about out-of-sample performance? May be I'll just have to paper trade it :)

Cheers,
Henning

Noob, There are heuristics like that but you figu...

2009-11-16T04:01:01.591-08:00

Noob,

There are heuristics like that but you figure them out by playing around with it, there haven't been results proved about it, and I think it's impossible due to the random nature of the forest.

Feel free to modify the code, it's only a few lines actually.

Regards,
Max

Hi again Max, If I want to papertrade gold with t...

2009-11-15T07:35:41.777-08:00

Hi again Max,

If I want to papertrade gold with the DTB as a predictor how can I see what the prediction is for the next week/day?

Regards,
Noob

Hi Max, Is it possible somehow to add some tradin...

2009-11-15T06:12:30.861-08:00

Hi Max,

Is it possible somehow to add some trading rules into the evaluation? For example a stop-loss where the long position is kept until it turns down.

Noob

Hi Max, Thanks for sharing!!! Are there certain re...

2009-11-14T08:38:33.872-08:00

Hi Max,
Thanks for sharing!!!
Are there certain relations (or ranges) between: data (number of posts), btsamples, trainperiods and treesInBag (and other inputs)that is preferable?
For example: 2 * trainperiods = treesInBag.

Thank you
Noob

Anon, You need a csv file with 1 minute data to d...

2009-11-12T11:00:11.831-08:00

Anon,

You need a csv file with 1 minute data to do that. Then just replace the filename in the code with your new filename.

Regards,
Max

Hi Max Very interesting!! Thank you for sharing!!!...

2009-11-11T16:42:34.032-08:00

Hi Max
Very interesting!! Thank you for sharing!!!
How do I configure it to run on 1 minute data?

Max, I agree. It looks like it needs a lot of p...

2009-07-30T17:50:40.254-07:00

Max,

I agree. It looks like it needs a lot of points.

The Logistic Regression surprised me. I was fooling around with a couple of logistic models before I saw this. That's now on the back burner.

The Nearest-Neighbor scheme seems to pick up a lot, even with just a few points. I've only read about these models, I haven't had time to test them out. I'll probably look a little closer now.

The sad part about most financial problems is they don't have nice little "islands" to discover. It's just loud or louder noise.

Bill S

Thanks Bill. Surprising that Bayes Nets did so bad...

2009-07-30T08:09:26.437-07:00

Thanks Bill. Surprising that Bayes Nets did so badly.

Max

Max, I was testing the randomforest and svm metho...

2009-07-30T07:39:21.292-07:00

Max,

I was testing the randomforest and svm methods to see what the trade-off/advantages are. While I was pushing "gamma", waaaaay overfitting the svm test, I went looking for results from the tests of others. I bumped into the following and thought you might be interested.

http://home.comcast.net/~tom.fawcett/public_html/ML-gallery/pages/index.html

Notice the similarities of svm and randomforest.

Bill S

You're welcome Bill

2009-07-29T06:50:43.338-07:00

You're welcome Bill

Max, Thanks for the answers and the links. That...

2009-07-28T13:43:06.514-07:00

Max,

Thanks for the answers and the links. That course information will keep me busy for a while. I also subscribed to the Finance mailing list.

Thanks again,
Bill S

Bill, Yes, that's where the offsetting occurs- an...

2009-07-28T12:03:30.290-07:00

Bill,

Yes, that's where the offsetting occurs- anywhere you see the variable "horizon".

RandomForest selects random bootstrap samples to train each tree. That's why each run gives different results.

I haven't read a book on R. I took this class at UC Berkeley: http://www.stat.berkeley.edu/classes/s133/ The professor has all the notes online. The "schedule" link at the top left has the notes and the "resources" link has some interesting sites. It was a good class and the notes are clear.

R has a great community, just subscribe to the email lists: http://www.r-project.org/mail.html I'm on the SIG-Finance mailer.

Regards,
Max

Max, You answered a couple of questions. First, ...

2009-07-28T11:16:57.787-07:00

Max,

You answered a couple of questions.

First, it appears that I am interpreting rungoldtreesys.R and the data file correctly.

Second, yes, I was looking for a lag between "targets" and "returns".

This brings up some new questions:

1. In file rtreesystem.R, is the following line where the "targets" "data" offset occurs?

savedEnsemble = trainEnsemble(data[(t + horizon+1):(t + trainperiods+horizon+1), ], targets[(t + 1):(t + trainperiods+1)], verbose = TRUE, treesInBag, keepNFeatures)

If not, where does it occur?

2. Running the same data 10 times, I get 10 different answers (typically with a different order of variable importance). Does this mean that no single variable dominates the data, or something else?

3. Can you recommend a good book on R, hopefully with a lot of examples? I'm an engineer (a mediocre programmer in various languages).

Thanks,
Bill S

Bill, The dumber the questions you have to ask, t...

2009-07-27T21:17:55.290-07:00

Bill,

The dumber the questions you have to ask, the dumber the teacher.

The 9th variable/returns are not ignored, they're included in the inputs.

Are you thinking that this is essentially giving the model exactly the same data it's trying to predict and therefore cheating? Actually it's not because the input data is lagged by one period compared to the targets. So the returns from yesterday are used to predict the returns today.

If you're asking something else, please clarify.

Regards,
Max

Max, Please forgive me for the dumb questions, bu...

2009-07-27T14:58:44.845-07:00

Max,

Please forgive me for the dumb questions, but I'm new to R and randomForest.

From files gold01-09.csv and rungoldtreesys.R:

#setwd("C:\\Documents and Settings\\Max\\My Documents\\Quant\\R")
data = read.csv('gold00-09.csv', header=TRUE, stringsAsFactors=FALSE)
returns = data[[9]]
dates = data[[1]]
data = data[-c(1)]
targets = returns
targets[targets>=0]=1
targets[targets<0]=-1

source('rtreesystem.r')
# factormodel.tree(data, targets, returns, btsamples=250, horizon=1, trainperiods=150, leverage = 'kelly', keepNFeatures = 10, treesInBag = 200)

In the above code, the +1 -1 "targets" values are the ones you're trying to classify/predict. They are "digitized" from the 9th variable in "data" (which is GLDr1 in the data file, and "returns" in the above code). When you pass "data" and "returns" into "factormodel.tree" how do you tell it to ignore "returns" and/or the 9th variable for modeling/prediction?

Thanks,
Bill S

Bill, You can find the code here: http://cran.r-p...

2009-07-27T11:55:47.171-07:00

Bill,

You can find the code here: http://cran.r-project.org/web/packages/randomForest/index.html

Regards,
Max

Max, Interesting article. I'm trying to reproduc...

2009-07-27T11:33:48.802-07:00

Max,

Interesting article.

I'm trying to reproduce your results on my machine. I've loaded R, rtreesystem.r, rungoldtreesys.r, and gold01-09.csv.

When I run it, R comes back with:

Loading required package: randomForest

Error in trainEnsemble(data[(t + horizon + 1):(t + trainperiods + horizon + :
could not find function "randomForest"

In addition: Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, :
there is no package called 'randomForest'

Apparently, it can't find "randomforest". Can you post the code for "randomforest"?

Thanks,
Bill S

Max, Interesting article. I'm trying to reproduc...

2009-07-27T11:30:00.896-07:00

That'll do it.

2009-06-27T09:46:42.077-07:00

That'll do it.