The official blog of BNP Paribas Asset Management

Sustainable investing – The divergence of ESG ratings (research)

Here is the first in a new series of monthly articles on current academic research into a range of responsible investment topics. The papers discussed were presented at the latest annual GRASFI[1] conference.

Compelling academic papers

Jane Ambachtsheer, global head of sustainability at BNPP AM, introduces the series:

In 2017, the Global Research Alliance for Sustainable Finance and Investment (GRASFI) was formed as a global collaboration of universities committed to producing high-quality interdisciplinary research and curricula on sustainable finance and investment. This series will highlight 10 of the most compelling academic papers presented following their third academic conference, virtually hosted by Columbia University, with some ‘practitioner takeaways’ provided by BNP Paribas Asset Management investment professionals. We sponsor GRASFI to bring academic rigour to the pressing challenges of sustainable finance and investment. Our goal is to share these reflections with clients and the industry. We invite you to visit the GRASFI 2020 Conference website at

The conference’s winning paper – ‘Aggregate Confusion: The Divergence of ESG Ratings’

As interest in responsible investing rises globally, multiple methods of assessing companies on environmental, social and governance (ESG) criteria have emerged.

These scoring systems vary by scope, measurement and weighting of ESG factors, making it difficult to compare offerings or understand why some agencies might rate the same company differently.

Researchers from the Massachusetts Institute of Technology’s Sloan School of Management analysed and compared the methodologies of six rating agencies to understand the differences between each firm’s processes better.

Written by Florian Berg, Julian Kölbel (who is also affiliated with the University of Zurich) and Roberto Rigobon, ‘Aggregate Confusion: The Divergence of ESG Ratings’ focused on the ratings processes of:

  • KLD (part of MSCI)
  • Sustainalytics
  • Vigeo Eiris (majority owned by Moody’s)
  • RobecoSAM (owned by S&P Global)
  • Asset4 (owned by Refinitiv)
  • MSCI

Building on previous work in this area, the paper sought to explore and explain why ESG ratings differed between agencies and quantify from where these differences originate.

The researchers broke down their analysis into three areas:

  • What factors are and are not included in an assessment
  • How are these factors measured
  • How are they weighted when compiling an overall ESG score.

The first two areas were found to have more of an impact than the third, while a further ‘rater effect’ was also uncovered.

Measurement and scope matter

How each ESG factor was measured was the main driver of differentiation between each agency’s rating process, Berg, Kölbel and Rigobon found.

Effectively, the differences in how factors are assessed can mean that, even if two rating agencies agree on what should be measured, their ratings may still differ because of how they have measured.

The researchers analysed the measurements different rating agencies used and how these correlated with each other, to illustrate where they agreed and where they diverged. In areas such as human rights and product safety, ‘measurement divergence’ was particularly pronounced.

There was also significant divergence between agencies on the ESG factors they used, the researchers found. This often played a key role in how overall scores diverged, reflecting the wide array of views on what issues are important to responsible investing.

The authors analysed the indicators used by all six rating agencies to find those that were common across providers. They identified 64 indicators as being used by at least two of the rating agencies, and nine were used by all five (excluding KLD).

The way each factor is weighted in the overall ESG score was found not to have a particularly significant effect on the divergence between rating agencies.

The other factor – The ‘rater effect’

Berg, Kölbel and Rigobon explained a ‘rater effect’ could influence a company’s overall score from a rating agency.

The effect refers to a bias of sorts – essentially, the performance of a company in one category can influence its performance in other categories. This suggested structural reasons for the divergence between rating agencies, the authors said.

For ESG indicators such as human rights or labour practices, analysts at rating agencies must use an element of judgement rather than a purely financial or numerical assessment. Such scores are often based on values and ‘soft’ data, rather than financial data, making them harder to quantify, measure and compare. This means there is a chance that the scores in these areas depend on the individual assessing them.

Dealing with divergence

While it can be difficult to see why two rating agencies have scored the same company differently, Berg, Kölbel and Rigobon argued that it was possible to “determine with precision” how their three factors affect the construction of an overall ESG score.

The study clearly shows that rating agencies can disagree on judgement-based factors, but also on the underlying data informing the scope and structure of their ESG scores. This can have a significant impact on responsible investing strategies, but also upon research – the same analysis could produce different results if it is based on one rating agency’s scores versus another’s, the authors highlighted.

For an ESG approach to be truly effective, investors must understand the methods by which fund managers and allocators assess the assets they are trading, the authors concluded. Understanding where rating agencies differ in their approaches is a vital tool to help institutions develop their ESG strategies.

Commenting on the paper, Raul Leote de Carvalho, Deputy Head of the Quantitative Research Group at BNP Paribas Asset Management, said:

“This is the most exhaustive paper about why ESG ratings may diverge that I have come across so far and one that raises the right questions about these differences. In my view, this paper is a must read for all investment practitioners.”

Also read:

[1] The Global Research Alliance for Sustainable Finance and Investment is a worldwide network of 26 leading universities that was established in 2017 to promote rigorous academic research into finance and responsible investment. BNP Paribas Asset Management has been the asset management sponsor of GRASFI since 2018. Through its sponsorship, BNPP AM is able to access leading academic research into sustainable finance and investment, helping to inform the broader debate.

Any views expressed here are those of the author as of the date of publication, are based on available information, and are subject to change without notice. Individual portfolio management teams may hold different views and may take different investment decisions for different clients. This document does not constitute investment advice.

The value of investments and the income they generate may go down as well as up and it is possible that investors will not recover their initial outlay. Past performance is no guarantee for future returns.

Investing in emerging markets, or specialised or restricted sectors is likely to be subject to a higher-than-average volatility due to a high degree of concentration, greater uncertainty because less information is available, there is less liquidity or due to greater sensitivity to changes in market conditions (social, political and economic conditions).

Some emerging markets offer less security than the majority of international developed markets. For this reason, services for portfolio transactions, liquidation and conservation on behalf of funds invested in emerging markets may carry greater risk.

Related articles

Weekly insights, straight to your inbox

A round-up of this week's key economic and market trends, and insights on what to expect going forward.

Please enter a valid email
Please check the boxes below to subscribe