Math and Science Solutions for Businesses/Mentor
QUESTION: Hello Randy,
Wow, What a resume. I have been using AllExperts for several years now and have greatly benefited from people such as you.
I have created a robotic stock market trading system that I have been running live for several years now. One of the parts I would like to improve upon is a "hypothesis generator" (HG). My system uses four different types of systems. One or more systems are selected as active by the HG according to market conditions.
Frankly I am stumped on how to improve this system component. Could you recommend an approach to understanding generally how current motion predicts changes in motion.
I apologize for not articulating my needs better.
Los Angeles area.
ANSWER: dp, thanks for the inquiry.
As I understand it, you need a systematic way to optimize the selection of one or more trading systems based on market conditions (or attributes) and perhaps on real-time performance. Ideally, we should come up with a way to predict future market attributes (trends or "motions") based on current attributes. I know you're well versed in so-called technical analyses (trendlines, Iron Condor, ...) so I will try to concentrate on a complimentary methodology. Since I don't have details of your system, I'll just describe some potential approaches.
In your case, the question is, given market attributes, how should 4 trading systems be combined (allocated) to maximize performance? This allocation is the job of your "hypothesis generator" HG. A summary of how I envison your system is shown in the attached image. This looks like a classic control system. Let me define some terms for the sake of discussion (I'm guessing a bit here so feel free to modify):
- Market attributes: trends in various sectors over various lengths of time, P/E, trading volume, interest rates, commoditiy prices,...
- Performance metrics: statistics for trade successes & puts/calls, fluctuations (volatility), revenue (overall, by sector), ...
Ignoring the feedback loop for the moment, this system is an open-loop system which must be "calibrated" by establishing a process (rules) in the HG that maximizes the output (performance) based on the input (market conditions). You are fortunate in having historical input and output from your model which can be used to optimize the HG. Some possiblilities:
• Multi-dimensional correlation between N market attributes and M performance metrics for given allocations of systems
- this gives rise to conditional probabilites for various causes and events that could be combined using Bayesian methods; i.e., given that certain favorable outcomes are desired, based on the performance metrics, which system combination is most likely to give rise to them.
• Linear programming to select allocation by optimizing cost function(s)
- determine highest performance(s) for given values of attributes; update in time
• Models of trading systems
- exploit time dependence through simple differential equations
- solutions can (should?) be non-linear, eg., birth/death process, multiplicative combinations.
• Feedback of performance metrics to HG process (closed loop)
- need to determine transformation of output units -> input units (box F in flowchart)
- could conceivably apply so-called PID methods to adjust allocation
- if appropriate, using feedback taps into a rich methodolgy (Laplace transforms)
The first thing to do is to tabulate (correlate) conditions and outcomes from previous runs. I can help you with this I hope. Also, for given market conditions, you probably have a reasonable idea of which system or systems to use. This can be used to establish a baseline for development. An important issue not specifically addressed above is how to "weight" the systems, i.e., how many resources should be given to each.
Interesting problem! Hope this helps.
---------- FOLLOW-UP ----------
QUESTION: Hi Randy,
Thank you for your generous offer to help. As it turns out I have decided to pursue another strategy. I have been trading for 16 years and have seen many forms of hypothesis formulation. Especially those that attempt to backtest and optimize parameters or those that attempt to use fractals applied to chart patterns or indicators. I am not a fan of any of those strategies.
The past is of little use in my opinion to the level of consistency needed to create winning fully "robotic" systems. You are correct in assuming I generally know under what conditions my systems perform best. So instead I have decided to abandon the HG concept entirely and take a subtle turn towards scanning for markets (symbols) that are currently acting in ways that my systems thrive in vs applying my systems to markets and waiting for the "right" conditions to arise..
The latter is attempting to be predictive while the former is attempting to capture present motion. Fortunately, my systems do very well at capturing anyway.
I believe I was just about to come to this realization when I contacted you. I apologize if I have wasted your time. I own some very powerful scanning software so my walkforward should be easy to test.
Thank you for your response and if you have any ideas that you think would benefit this strategy I would love to discuss them further.
Thanks for the follow-up. Sounds like you have plenty of experience to chart a reasonal course. I appreciate that you don't want to use retreads.
It sounds like a key feature to your new approach is to refine the detection of behaviors amongst a lot of jumble (noise). A decade ago, I led the developement of an anomaly detector for ship tracks (trying to detect covert, potentially threatening behavior i.e., mine laying) that used a self-organizing map. It worked pretty well and got some traction at ONR and USCG.
I'll leave that as food for thought and will think about its application to trading. At any rate, good luck to you. Let me know if you come across any other interesting problems.