Managers matter. Here’s some intriguing evidence. From soccer.
Evaluating managerial performance is not an easy task, at least empirically. Part of the problem is that outcome measures of firm performance are not straightforward (short term profits? long term profits? market share?) Plus your typical organization is multi-layered which makes the influence of the top honcho, contrary to adages such as the fish rots from the head, somewhat less easy to identify.
A few years back I kicked around, with a frequent collaborator of mine and over some good Greek wine and ouzo, the idea of using data from soccer (and in particular the Bundesliga, Europe’s best premier league ;-)). Such data have some obvious strengths: For example, coaches are often fired and hired between, and even within, seasons. In fact, in the Bundesliga historically almost half of the coaches were dismissed during the full season. There is hence considerable variation to exploit through coach fixed effects, especially if the coaches have moved a lot. There are also clear-cut performance criteria (e.g., ranking after the season, average points won per season). Obviously that leaves some interesting issues to disentangle. For example, coaches that were hired by top teams ought to be statistically handicapped vis-a-vis those that coach teams at the bottom.
Alas, a bunch of German researchers have done exactly this in an article that I came across only this weekend but which was published a couple of years back. It’s one of these papers that I wish I would have written and it turns out it has aged very well.
Muehlheusser and his colleagues analyzed 21 seasons of Bundesliga data (from 1993–94 to 2013–2014). (They use the term manager when they mean to refer to coaches.) Their unit of observation is a manager-team pair during a half-season which is sensible as each team faces each other exactly once during that time. Consequently, the authors’ basic empirical model regresses points on fixed effects for teams, managers, and half-seasons. They also do several interesting robustness checks (including, intriguingly, a cross validation exercise and an analysis of managerial impact on performance and team style).
Arguably the key result comes out of their estimation of manager fixed effects on performance (as measured by points). Focussing on the 44 mover-coaches in their data set (i.e., managers that moved at least once between clubs), they rank them in Table 5. Turns out the ranking makes indeed a lot of sense, at least to someone like me who has followed the Bundesliga almost as long as it has been in existence. Almost. ;-)
Juergen Klopp, former coach in Mainz and Dortmund (now the manager of FC Liverpool) tops the list with a coefficient of .459. This means, and statistically significant so, that all things equal (“ceteris paribus”) his teams have won on average 0.459 points per match more than a team coached by the manager of median ability (and cognoscenti will appreciate this being Bruno Labbadia). This effect is not just statistically significant but highly economically significant as the authors explain in detail on p. 796. For the season 2012–13, for example, Klopp’s contribution would have pushed a team he trained from rank 13 to 4, which would have entitled the team to join the UEFA Champions League in the following season and hence would have represented a huge financial bonanza for the club that had the smarts to hire Klopp.
There are very few surprises in the list and of course the years since Klopp left Dortmund have provided more cross validation albeit in a different premier league. Talking about which, the authors also present a joint ranking of mover-coaches and non-mover-coaches (72) which is topped by Thomas Tuchel and Pep Guardiola (and has Klopp at number 4). Seeing Guardiola ranked that high won’t surprise many but Tuchel’s top ranking, based on data while he was coaching Mainz and before he joined Dortmund and way before his recent triumphs with FC Chelsea, the authors found so dubious that they qualified their finding saying that Tuchel’s manager effect (0.829!) seemed “excessively high” (p. 798). Ha!
So there, a fabulous paper it is which has aged very well indeed. I have no doubt that a few clubs took notice when it came out and I would also not be surprised if a number of clubs have recently crunched the numbers and identified “undervalued managers” (p. 788). I doubt very much that hiring Nagelsmann as manager of Bayern Munich (and Bayern Munich being willing to pay millions of Euro to its former employer, RB Leipzig) was just based on gut feeling. Although my gut feeling is that Bayern Munich made a big mistake pushing out Hansi Flick. A big win that though for the German national team. There will, undoubtedly, be a follow-up study of the present study and I take bets.
Science. It’s a thing.
