Ten years of the Damages Directive: Three ways to better use economic analysis in court

When economic analysis is dismissed in favour of informal estimates in competition damages cases, it runs counter to the basic purpose of the Damages Directive: to compensate victims for the actual harm suffered. In this article, Jasper Haller and Soledad Pereiras [1] discuss the role of economics in damages cases and propose three practical ways to improve the efficient and effective use of economic evidence in court.

View the PDF version of this article.

The views expressed in this article are the views of the authors only and do not necessarily represent the views of Compass Lexecon, its management, its subsidiaries, its affiliates, its employees or its clients.

A milestone in the history of damage litigation

On 26 November 2014, Directive 2014/104/EU, known as the “Damages Directive”, came into force.[2] Article 3 of the Damages Directive describes its purpose succinctly: “Member States shall ensure that any natural or legal person who has suffered harm caused by an infringement of competition law is able to claim and to obtain full compensation for that harm.” The Damages Directive then sets out a series of instructions for EU member states to ensure that victims of violations of Articles 101 and 102 TFEU can seek damages via litigation effectively.

As member states enacted these rules, the number of court cases involving competition law damages in the EU increased significantly. Research by Laborde (2021) identified just over 50 judgments related to cartel damages in the entire EU between 1998 and 2014; in the six years between 2015 and 2020, that number jumped to nearly 300.[3] Cases have come in all shapes and sizes: from various forms of price coordination (car parts [4]) or market sharing (elevators [5]), to buyer cartels (ethylene [6]); from great depths (deep-sea power cables [7]) to dizzy heights (air cargo services [8]); and from healthy (milk [9]) to, well, less so (sugar [10]).

With the proliferation of competition damage claims, the question of quantification of the damage has taken centre stage. Most of the time, this has involved assessing the difference between actual market outcomes and the outcomes that would have occurred in a counterfactual situation where the competition law infringement never took place. Sometimes, that comparison relies on – and requires – statistical analysis. That analysis, of course, is the domain of economists, which is how economic analysis and econometrics entered the courtrooms.

The role of economists in litigation

In damages cases, both claimants and defendants retain economist experts. Their role is to provide an estimate of damages, grounded in economic analysis.

However, opposing experts often express different – sometimes significantly different – opinions. Damages estimation is not an exact science: it requires judgement calls, and different economists will come to different estimates of the damage incurred, even when acting in good faith and expressing independent opinions.

Some commentators have expressed dismay, and at least implicitly question whether economists fulfil their role in good faith, where the economic reports commissioned by the parties routinely produce results that are far apart from each other and favourable to the respective party.

In reaction to wide disagreement and (seemingly) favourable estimates, a number of rulings in recent years dismissed the estimations put forward by the parties’ economic experts and instead relied on informal damages estimates performed by the judges themselves.[11] This situation is concerning. The use of informal estimates runs counter to the basic purpose of the Damages Directive, which is to compensate victims of a competition law infringement for the actual harm suffered. There is little reason, if any, to believe that informal estimates or decisions that “split the baby” represent an adequate approximation of the harm they seek to address.

To understand how we got here, it is instructive to revisit the Practical Guide on Quantifying Harm in Actions for Damages (the “Practical Guide”), which the European Commission published together with the Damages Directive.[12]  The purpose of the Practical Guide was to “place at the disposal of courts and parties to damages actions economic and practical insights that may be of use when national rules and practices are applied” and ”help the claimant make factual submissions to the court concerning the amount of damages claimed and may assist the defendant in pleading his position vis-à-vis these submissions by the claimant".[13]It also has the (perhaps unrealistic) aim of “help[ing] parties in finding a consensual resolution of their disputes”.[14]

The Practical Guide is a carefully drafted document that contains sound insights. In plain language and quite some detail, it:

• discusses a range of different methods that can be used in the estimation of damages based upon the nature of the infringement and the data available;
• explains some fundamental principles of econometric estimations; and
• specifies recommended good practice, addressing a range of practical problems that can arise when quantifying damages.

Why, then, do some courts still dismiss the economic evidence presented to them? In our experience, it is a combination of factors that creates and maintains a gap between economic experts and judges. Courts do not always have the tools to correctly assess the economic evidence before them, particularly when that evidence is necessarily complex and sensitive to choices about assumptions or data. In some cases, economic experts exacerbate the situation by downplaying the limitations of their reports or hiding them under complex jargon and narrow instructions. 

In addition, the Practical Guide still doesn’t do enough to help close the gap between economic experts and judges. This is for two reasons.

• The Practical Guide focuses strictly on the substance of economic evidence: the techniques available to quantify damages and the proper interpretation of their results. As a document written abstractly to be applied in all competition damages litigation, regardless of the circumstances of the case and the jurisdiction, and ahead of the development of the litigation triggered by the Damages Directive, the Practical Guide can only go so far. Too often, the Practical Guide must resort to language that is too unspecific to be useful. Sentences such as “in a given case, the choice of technique will usually depend on a range of aspects, in particular the legal requirements and the factual circumstances of the case” [15] are obviously true, but so much so that they offer little practical assistance to a court faced with two conflicting economic reports.
• The Practical Guide does not offer advice on the process of engaging with economic evidence: how it should be presented or how courts should interrogate the evidence before them. Because the Practical Guide sidesteps the complexities of the judicial procedure [16], there often remains a wide gulf between the economic evidence itself and its translation into a court ruling.

In this context, additional guidance on how to evaluate and use the evidence provided by economic experts could be useful.

Three proposals for more effective use of economic evidence

It is not unusual for a proceeding to feature two reports prepared by expert economists that come to entirely opposite conclusions. With countless ingredients that go into a damage estimation, ranging from the choice of methodology and data to the exact choice and definitions of variables to be used, it is virtually guaranteed that different economists will arrive at different estimates.

But this does not mean that courts should disregard economic reports to produce their own estimates, based on broad axe or Solomonic solutions instead. On the contrary, we believe courts can use these apparent contradictions to extract relevant insights on the analyses presented, discern their quality, and use them to get the most accurate outcome.

This requires an effort from economists, their clients, the legal advisors, and the courts.

We propose three action points that, in our experience, would improve the use of economic evidence and help achieve the outcome envisaged by the Damages Directive: ensuring that claimants receive compensation commensurate with the harm actually suffered in the case at hand. These action points are:

1. ensuring the credibility of the economic work presented;
2. encouraging constructive dialogues between economists; and
3. challenging the economic experts, forcing them to comply with their duty to the court.

Ensuring the credibility of empirical economic work

Questions around the credibility and robustness of empirical findings are not confined to the courtroom, or even to the discipline of economics. In recent years, several scientific disciplines have been gripped by a “replication crisis” as highly regarded empirical findings turned out to be mirages that could not be replicated by other scientists. The replication crisis has rattled not just economics but also other disciplines, notably including psychology and medicine, a shocking development given that these disciplines largely rely on the gold standard for empirical evidence, the randomised controlled trial.

In a few high-profile cases, important scientific findings could not be replicated because the original data had simply been fabricated.[17] But in many other cases, the explanations were far more innocuous and involved either simple errors made in good faith or a practice known as “p-hacking”, whereby authors would test a wide array of hypotheses but only report those supported by the data. But just as a coin is bound to come up “heads” eventually if flipped often enough, one is bound to detect some associations by chance if one tests enough hypotheses. Therefore, indiscriminately testing a wide range of hypotheses but only reporting those that have been seemingly confirmed cannot uncover meaningful cause-effect relationships.[18]

Fortunately, science not only recognizes the problem, but also offers some solutions.

The first is mandatory disclosure. Many academic journals no longer accept submissions unless accompanied by a comprehensive data and code package that allows reviewers and other scientists to inspect and replicate the work. We strongly believe that this practice should extend to reports submitted to court. Not only does mandatory disclosure allow for opposing experts to see each other’s work, making exchanges between the experts more fruitful; it also, in regimes where courts appoint their own experts, aids court-appointed experts in assessing the parties’ work and generating their own estimations. In our view, the adage in academia that “a paper is nothing but code with a cover on it” should readily extend to economic evidence submitted to courts.

A second, perhaps more controversial, concept borrowed from academia is known as “pre-registration”. To dispel the suspicion that they might have engaged in “p-hacking” and only decided after the fact which empirical results to report and which to withhold, many scientists now “pre-register” their studies. They draft a methodological paper that sets out the econometric models to be estimated before the data has even been collected. This paper is then submitted to a repository where it receives a verifiable time stamp. Later on, the initial methodological paper accompanies the scientific article with the final results, allowing reviewers to assess whether the authors stuck to their original concept for the study.

While we have never seen it in action before, we see no reason why pre-registration could not be adapted to economic expert evidence submitted to court. It would require an expert to commit to certain methodological choices based on first principles before even having received the client’s data, and to put these commitments into a time-stamped document before beginning the empirical work. We see great promises in this approach, as it allows an economic expert to dispel any suspicions that they engaged in an opportunistic “trial and error” approach.

At the same time, we recognise that clients may find the idea of “their” expert tying themselves to an econometric mast without knowing the result difficult to swallow. But it should not be seen that way. Pre-registration commits an expert to a process for quantification, not a specific approach. It does not prevent an expert from changing their view or approach in light of the evidence. However, pre-registration ensures those changes are explicit, so an expert must explain and justify those changes to the court. Ultimately, this makes it more likely the evidence that a party presents to the court is understood and trusted. To overcome the appeal of flexibility, it might help if courts were to encourage pre-registration by making clear that they will assign much more weight to expert evidence that has gone through this process.

Encouraging constructive dialogue between economists

Even when dealing with perfectly transparent reports, it may be difficult for the court to fully understand the reasons for the discrepancies between the results of two opposing experts and, more importantly, how to properly assess the decisions the experts have made and their implications.

This is where constructive dialogue is necessary.

In most courts in Europe, a debate between opposing experts typically takes place through various rounds of rebuttals. In our experience these rebuttals can often be quite useless to the courts, as they provide a comprehensive list of criticisms of the opposing report, no matter how trivial, such that courts may get lost in a myriad of criticisms that have little relevance. Moreover, where two opposing experts respond to each other in standalone documents, it may be difficult for an outsider to form a view of the validity of the arguments from reading the rebuttals alone.[19]

In our opinion, a more useful approach is based on joint expert statements, where the experts from both sides set out what they agree and disagree on. Such statements can have several advantages. First, a joint statement forces economic experts to (i) recognise the key assumptions they disagree on, (ii) make explicit the implications of each of those assumptions and (iii) set out transparently the reasons for the disagreements and their consequences.[20] Second, joint statements provide an opportunity to narrow down the potential issues to the most relevant ones, reducing the attention experts and courts spend on areas of secondary importance. As a result, joint statements put the courts in the best possible position to critically review each expert’s evidence and, if necessary, request additional analyses.

Some legal regimes allow for the appointment of an independent expert, who formulates their own opinions and challenges the work of the expert economists proposed by the parties. In our view, the same approach would be productive regardless of whether a court-appointed expert is present or not. After all, the court-appointed expert will inevitably be challenged by economists retained by the parties. At that stage, all sides will again benefit from a constructive dialogue among economists.

Challenging the economic experts

With all the economic evidence before them, the courts will need to form a view on how to weigh it. This requires a constructive conversation with the economic experts, where courts should not be afraid to challenge their views. While every case and every expert report is different, we think there are three things that courts should request from economic expert evidence.

First, courts should demand clarity. If the reader cannot follow the analysis being presented, it is more likely that the problem is with the analysis, not the reader. Economic evidence may look daunting, with equations and regression tables. But ultimately, the economic concepts involved are usually simple, and experts should be able to state them in plain language. Reports should be written cleanly and succinctly. This also applies to cross-examination.

Second, courts should expect a detailed discussion of the validity of the data used in the analysis. The size and quality of the information available can make all the difference between a credible economic analysis and an unreliable one. Recent advances in computing power have massively improved firms’ capabilities to record detailed information on their transactions. For empirically minded economists such as ourselves, this is a very exciting development, as modern high-quality datasets tend to contain far more transactions, and far more details on each transaction, than datasets previously available. And size matters: not only does it reduce the noise inherent in any statistical analysis, it also allows economists to conduct analyses that are much more credible. For example, if an infringement is known to have come to an end suddenly on a certain date, it may be useful to compare transactions just before to just after the cut-off date. But such a strategy requires data with enough observations on either side of the cut-off to make robust inferences.

Still, in some cases, the data available is far from the ideal we envisage. Even in those situations, useful analyses can be performed. But courts can only rely on these results if there is a detailed discussion of the validity of the information considered and an honest assessment of how this data can approximate the results that one would get with the ideal dataset.

Third, courts should demand transparency. The reality of empirical work is that it involves judgment calls on the sample to use, the variables to include, and the way they enter the econometric model. A credible econometric analysis will come with a comprehensive set of robustness checks that shows that small variations to any of these elements will not overturn the main results. Courts should also be able to instruct experts to conduct additional robustness checks, for example because a court has convinced itself that the analysis presented by one expert is generally reliable but sides with an opposing expert on a particular point of detail.

Not new proposals

None of our proposals is entirely new, and some are already being practised in various legal regimes today. For example, the UK Competition Appeals Tribunal asks economists to formulate joint expert statements, and we have seen procedures for data disclosure to the other experts, to the court and to court-appointed experts in various regimes, including the UK, Italy, Austria, Germany and Spain. Guidance on transparency, replicability and reliability of data has been issued by the European Commission [21] and the Spanish Competition Authority [22], among others.

While parties might be able to implement some of our proposals unilaterally in order to improve the credibility of the economic evidence they are presenting, we think they will be much more likely to do so if they know that courts will reward these efforts and assign their evidence greater weight. On that basis, there seems to be room for a comprehensive set of guidance for courts across Europe that reflects the experience of the past decade, to help them develop processes that facilitate the evaluation of the evidence provided by economic experts. As the Damages Directive turns 10 years old, the time may be ripe for a Practical Guide refresh that covers not just the role of economics but also the role of economists. We have made some suggestions in this paper.

View the PDF version of this article.

References