Morningstar jumpstarts its star rating system with favorable self-review yet audit finds it misses the mark in bear markets, supporting claim of dubious predictive value cited by Wall Street Journal

Agree with the main points of the article. Any rating system based purely on historical data is going to falter when the market shifts direction...moving into either a bear, or bull, market. There is a rather simple way to handle this however that I haven't seen tried - and that has to do with the diversification within a given fund and the over/underweights to various sectors. One could forecast a fund's risk in a market disruption by using a "downside" covariance matrix approach by assuming something simple like a) all sector correlations move towards 1, and b) beta of individual assets within the fund increase by say 50%. Run a the fund's exposures through this "downside" covariance matrix and you've got a bear market estimate. Then, based on how over, or under, valued the market is you can forecast a probability of market downside risk occurring at any time. Use this as a Bayesian adjustment to Morningstar's standard approach. Maybe the folks at M'star have already played around with this and I'm just unaware, but seems a more robust approach than what's "state of the art" today.