Business valuers can’t afford to use bad data—an ESG case study

BVWire–UKIssue #28-2
July 20, 2021

Large data providers Refinitiv/LSEG and SPGI have backed European and UK efforts to add environmental, social, and governance (ESG) metrics to financial reporting disclosures for listed companies. Now Moody’s is in the game as well, and their new tool to “generate real-time predicted environmental, social, and governance scores” adds a new twist. The Moody’s ESG Score Predictor provides analysts with “quantitative data” (is that redundant?) for private equity portfolio companies and those who invest in them. This brings ESG ratings into the private small- and medium-sized enterprise market.

While these expensive rating tools may increase disclosures that help investors assess risk, they are not auditable for purposes of business valuation. Every valuer BVWire—UK has discussed ESG ratings with says simply that they can only capture the benefits or disadvantages of environmental, social, or governance characteristics through cash flows or perhaps with adjustments supported via industry risk premia—or by adjusting their transaction or listed company comparables.

The ESG rates available now are also not replicable—a problem documented in an updated academic research study that documents the changes in Refinitiv’s methodology for creating ESG ratings. The authors, Florian Berg (Massachusetts Institute of Technology), Kornelia Fabisik (Frankfurt School of Finance and Management), and Zacharias Sautner (Frankfurt School of Finance and Management and ECGI), take some pleasure exposing the failures of this methodology while pointing out that every investor from BlackRock to the World Economic Forum on down now emphasise how critical these ratings are to current investment strategy.

Rewriting History II: The (Un)Predictable Past of ESG Ratings, which has been updated several times since its original release last year, is worth reading as a warning to any financial analyst who relies on models and third-party data to value private assets—in other words, every business valuer.

The authors began by comparing the ESG ratings of listed European companies using the 2018 and 2020 “rewritings” of the Refinitiv product (they imply that similar revisions influence the other ESG rating services, too, but single out Refinitiv because it appears the most frequently in academic literature).

First, they discover that the data rewriting is ongoing—so users of the outputs would get different results depending on whether they’re using a historical update to match their valuation date.

Furthermore, the researchers find that “retroactive ratings changes” have altered the rankings and classification of firms and quantiles—generally to make the rankings more closely mimic returns data. More than four-fifths of the ratings change downward between the two rewritings, with little relationship. One example from the study of the trustworthiness of these ratings for business valuers is:

After inspecting the two downloads, we observed that the ESG scores for identical firm-years differed between the two data versions—in some cases dramatically. In fact, not a single ESG score was the same across the two versions. Thirteen percent (13%) of the sample observations were subject to a score “upgrade,” that is, the rewritten ESG score was higher than the initial ESG score. Even more remarkably, 87% of the observations were subject to a score downgrade. The effect of data rewriting is also economically large. While the overall ESG score in the rewritten version is on average 20.6% lower than in the initial version, the percentage deviations from the initial to the rewritten version for the E, S, and G subscores are -47.4%, 8.6%, and 116.2%, respectively.

Please let us know if you have any comments about this article or enhancements you would like to see.