Date created: | Last Updated:

Category: Project

Description: FOCUS was an Intelligence Advanced Research Projects Activity (IARPA) program that sought to develop and empirically evaluate systematic approaches to counterfactual forecasting. This site hosts experimental materials and outcome data from that program with the intention of fostering more research on counterfactual forecasting. FOCUS addressed this research gap by developing and empirically testing alternative approaches to structuring the counterfactual forecasting process in ways that could be used to improve analytic tradecraft in complex domains such as geopolitical analysis. FOCUS was originally conceived and developed by Dr. Paul Lehner. To access FOCUS program-related publications, you can search Google Scholar using the program contract numbers, like this: "140D0419C0048 OR 140D0419C0049 OR 140D0419C0050" IARPA's program website is here: https://www.iarpa.gov/index.php/research-programs/focus-2 Counterfactual forecasts are statements about what would have happened if different circumstances had occurred. For example, a post mortem of an analysis failure may lead to a conclusion that analysts would have avoided the failure if they employed better tradecraft; perhaps by having double checked assumptions, perhaps by having considered a broader range of hypotheses, etc. Counterfactual forecasts about what would have worked in past circumstances are very often the basis for lessons learned for what to do in the future. And such lessons often evolve, over time, into best practices and tradecraft. To date there has been little in the way of research that measures the extent to which different approaches to counterfactual forecasting yield accurate vs. inaccurate counterfactual forecasts. And there is a similar paucity of research on the accuracy of lessons drawn from different lessons-learned approaches. As a result, evidence-based guidance for approaching post mortem lessons learned activities, and for developing the counterfactual forecasts that are the core of such activities, do not exist. Also there is correspondingly little empirical evidence that would support a claim that current lessons learned practices usually yield good or accurate lessons. FOCUS addressed this research gap by developing and empirically testing alternative approaches to structuring the counterfactual forecasting process in ways that could be used to improve analytic tradecraft in complex domains such as geopolitical analysis. This material is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA Contract No. 2017-17032700004-007 under the FoCUS Program. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation. Distribution Statement A - Approved for public release; Distribution is unlimited.