Through the Looking Glass: Understanding Social Science Norms for International Investment Law
Abstract
When social science methods are being employed in a new context — such as the assessment of international investment law — there is value in exploring the underlying assumptions and normative baselines of the enterprise. This article and response address critiques about the methodology of an article in the Harvard International Law Journal by: (1) describing the value of social science in international investment law; (2) replicating the research using new methodologies to conduct more than 20 new tests that were still unable to ascertain the existence of a reliable relationship between development status and outcomes on the basis of pre-2007 data, which demonstrates the initial results were robust; (3) identifying social science norms for conducting and describing research; and (4) exploring opportunities for the evolution of social science research on investment. The article explains that, for investment treaty arbitration, statistical power is often low given the nascent nature of the area and that full statistical power will require between 750-1200 cases to rule out the existence of possible small effects. Therefore, in an area of critical normative debate, it is preferable to report results but acknowledge the limitations; this is appropriate as empirical work, just like any other form of acquiring knowledge, has a risk of error. The article concludes that future research can and should replicate the initial scholarship to ascertain whether the pre-2007 historical snapshot continues to be valid as the population grows. In this way, normative debate can be be grounded in reality rather than rhetoric.