Summary
The wine industry is a particularly interesting case for studying the relevance of organic labels for three reasons:

1. There is a notorious lack of clarity in the industry about the relevance of organic labels among researchers, winemakers, and consumers.
2. The wine sector is particularly well suited to an investigation of the relationship between organic certification and quality, as wine, unlike many other agricultural products, is a highly differentiated product for which experts frequently publish quality ratings. It is therefore possible to examine a wide range of product characteristics such as vintage, grape variety, region of origin and price to measure the effect of organic certification on quality.
3. The wine industry has introduced different types of eco-labels, including labels that do not rely on third-party certification processes. This makes it possible to distinguish the effects of third-party certification from other labels.

This study aims to evaluate the influence of eco-labels on the assessment of product quality by experts, based on the examination of wines from France, one of the most important wine-producing countries. The data comes from 128,182 wines tasted by French wine experts and allows a comparison of eco-labelled wines (with or without certification) with conventional wines. Previous research suggests that third-party organic certification improves quality. However, it is not known whether this applies to all types of certifications (biodynamic or organic) and whether non-certified sustainability practices are also associated with quality improvements.

The researchers relied on expert opinions and wine descriptions published in three of the most important wine guides in France: the Gault & Millaud (GM), the Gilbert & Gaillard (GG) and the Bettane+Desseauve (BD), for the period from 2008 to 2015. The rating scales of the different wine guides were harmonised to create a common scale from 0 to 100. The information on certifications was also taken from the wine guides mentioned and supplemented where necessary with official data from organic and biodynamic (Demeter) certification bodies. In addition, the wine guides each provide information on the vineyard, vintage, region, appellation, grape variety(ies) and colour of the wine. It should be noted that the GG does not distinguish between organic and Demeter certifications. All wines in this guide are therefore classified as organic. Subsequently, the data were aggregated into six models (1) – (6) using specific statistical methods to assess the causal effect of organic labels on observable wine characteristics.

Figure 1: Gault Millau Sample Distribution.  Different quality scores for wines from biodynamic (dark green), organic (green), reasoned/ integrated (light green) and conventional (light grey) agriculture. The X-axis shows the score; higher scores stand for higher wine quality. For biodynamic wine, in particular, a clear distribution pattern emerges: the proportion of biodynamic wines increases markedly as the score rises.

Results
The results of the study show a significant difference between the quality of wines produced under the conditions of an organic label and conventionally produced wines. The experts rate the quality of organic and biodynamic wines positively. This is in stark contrast to earlier observations about consumer preference for these wines. Interestingly, biodynamic wines are rated better on average than organic wines. Biodynamic wines have gained a reputation for special quality despite a certification that many consumers find difficult to understand.

In model (1), which is the least restrictive and is based on 108,977 ratings from GM, BD and GG, the researchers observed a combined effect of organic and biodynamic practices of about +7.1 percentage points (pp) compared to the reference group composed of wines from conventional and integrated farming. In model (2), based on 108,977 evaluations from GG and BD, the influence of organic practices on wine quality is positive with about +3.3 pp, while the experts seem to be more convinced by biodynamic practices: an additional 7.3 pp (10.6 minus 3.3). In this context, it should be remembered that all biodynamic wines are organic by nature, as the biodynamic specifications are stricter than those of organic farming. These results are consistent with those obtained with models (3) and (4) using a subset of wines that are either organic/biodynamic or conventional, depending on the vintage in the data. Model (3) gives a combined effect of organic and biodynamic with an increase of 3.7 pp. In model (4) the effect of organic practices is not significant, while biodynamic practices result in a quality increase of 5.4 pp (5.4 minus 0).

No convincing empirical evidence could be found that non-certified sustainable practices have a positive impact on wine quality.

Conclusions
The results show that typical conventional producers could gain in quality by switching to organic practices (marginal gain = +6.2 pp). Switching from organic to biodynamic practices is also associated with a quality gain (marginal gain = +5.6 pp). The direct transition from conventional to biodynamic practices would lead to a quality gain of 11.8 pp. However, switching from conventional to reasoned/integrated agriculture without certification would not significantly change quality (marginal gain = 0 pp).

These results, based on a large sample of French wines, are in line with the results obtained by M. A. Delmas et al. in 2016 from a sample of 74,000 Californian wines. In the 2016 study, however, it was not possible to distinguish between organic and biodynamic wines because the number of wines in both categories was too small. Nor was it possible to evaluate non-certified practices. The large amount of organic and biodynamic wines assessed by GM, GG and BD allows for this important distinction while relying on more robust statistical models. Another relevant finding is the comparatively weak performance of self-reported sustainable practices compared to conventional practices.

Details of the original study
Authors: Magali A. Delmas, Olivier Gergaud
Title: Sustainable practices and product quality: Is there value in eco-label certification? The case of wine. 
Journal: Ecological Economics, Volume 183, 2021, 106953
Link: https://doi.org/10.1016/j.ecolecon.2021.106953

This study summary is based on the article «Étude: les vins certifiés bio et biodynamiques sont-ils meilleurs?» and was produced with the kind permission of Martin Quantin, Biodynamie Recherche.