I don’t think this is a rigorous way of selecting and optimising. Firstly, all your strategies are highly correlated so you should really look into the relative performances, regressed. Second, top(1) justification is based on prior performance which is basically just overfitting and I don’t think I need to justify why in this comment. Lastly, the concept of letting the “markets decide which ones are winners” by any sort of “burn rate” is just flawed. Market regimes change, so in theory this shouldn’t even work. The idea of stability of strategies really hamper this too. Picking the best out of the strategies that work in a brute force way and ignoring the ones that don’t work based on no economical or statistically significant way is not the way forward. You are basically ignoring losers and focusing on winners based on past performance and hoping that keeps on — which is the defn of overfitting.
Papers can be wrong, in this case think about it yourself. Does the methodology truly feel proper fit? Because the way you describe of picking the best based on some runs is overfitting. I’m not discounting the authors reputation of how well the paper is peer reviewed but think about it yourself instead of just relying on “it’s an academic paper”
I think the fact that it's been cited over 200 times (https://link.springer.com/article/10.1023/A:1007424614876), and thus essentially peer reviewed at least 200 times and looked over by tens of thousands of professionals tells me that the paper is probably not wrong. Also, get this, you can read it yourself and check the math.
so the game changer is you bet on the top(1) instead of top(n).
Any justification why top(1) is better than top(n)? top(n) seems to be slightly worse than averaging across all the strategies in your first try.. that makes one hesitate before jumping to any conclusion. How about top(2), top(3)..
Does this approach not bear the heavy risk of overfitting? Instead of directly optimising parameters you pick the strategies with the best fitting parameters?
These types of system are completely online and generally parameter-less, so you don't risk as much overfitting as say... a neural networks or something with a lot of parameters you need to tune.
The goal of adaptive systems is to adapt to the data in a universal way that still gives you some guarantees so you don't need to tune or train it at all. It performs out of the box.
did you try lagging the execution by 1 day?
I don’t think this is a rigorous way of selecting and optimising. Firstly, all your strategies are highly correlated so you should really look into the relative performances, regressed. Second, top(1) justification is based on prior performance which is basically just overfitting and I don’t think I need to justify why in this comment. Lastly, the concept of letting the “markets decide which ones are winners” by any sort of “burn rate” is just flawed. Market regimes change, so in theory this shouldn’t even work. The idea of stability of strategies really hamper this too. Picking the best out of the strategies that work in a brute force way and ignoring the ones that don’t work based on no economical or statistically significant way is not the way forward. You are basically ignoring losers and focusing on winners based on past performance and hoping that keeps on — which is the defn of overfitting.
I understand your critique. However, this concept comes from a highly influential and cited paper: https://www.sciencedirect.com/science/article/abs/pii/B9781558603776500438
You can find the PDF for free with a simple Google search.
Papers can be wrong, in this case think about it yourself. Does the methodology truly feel proper fit? Because the way you describe of picking the best based on some runs is overfitting. I’m not discounting the authors reputation of how well the paper is peer reviewed but think about it yourself instead of just relying on “it’s an academic paper”
I think the fact that it's been cited over 200 times (https://link.springer.com/article/10.1023/A:1007424614876), and thus essentially peer reviewed at least 200 times and looked over by tens of thousands of professionals tells me that the paper is probably not wrong. Also, get this, you can read it yourself and check the math.
so the game changer is you bet on the top(1) instead of top(n).
Any justification why top(1) is better than top(n)? top(n) seems to be slightly worse than averaging across all the strategies in your first try.. that makes one hesitate before jumping to any conclusion. How about top(2), top(3)..
Does this approach not bear the heavy risk of overfitting? Instead of directly optimising parameters you pick the strategies with the best fitting parameters?
These types of system are completely online and generally parameter-less, so you don't risk as much overfitting as say... a neural networks or something with a lot of parameters you need to tune.
The goal of adaptive systems is to adapt to the data in a universal way that still gives you some guarantees so you don't need to tune or train it at all. It performs out of the box.
Here's some resources about it:
Where I got the 'freezing the top-k' concept from: https://link.springer.com/content/pdf/10.1023/A:1007424614876.pdf
More about pruning and tracking the best experts: https://proceedings.mlr.press/v76/mourtada17a/mourtada17a.pdf
Exponential weights / gradients, which are useful in more complex online systems: https://www.face-rec.org/algorithms/Boosting-Ensemble/decision-theoretic_generalization.pdf
Other interesting readings:
https://parameterfree.com/2019/10/08/follow-the-regularized-leader-i-regret-equality/
https://parameterfree.com/2020/05/03/adahedge/
Let me know what you think.