Systems analysis 2015

I attended the Systems Analysis 2015 conference which was organized last week from 11. to 13. November. Systems analysis means the use of modeling to support decision making.

The highlights of the event for me were the chances to get to talk with top scientists of today. It was fascinating to hear about their work and I was very happy to hear that many of them appreciated our ideas.

--

Howard Raiffa session was one of the sessions I found most interesting in the whole conference. The reason is that it dealt with a topic which I have studied a lot: Decision Analysis. Below I will explain some key ideas presented in the talks given in this session. I also provide links to video recordings of the talks:

David E. Bell: Policy, Probability and Preference.

• Modelers accept subjectivity in probabilities but many still hesitate to include subjective preferences in analyses.
• Many models that have been successful in practice are prescriptive and based on simple assumptions. Economics models, mean-variance portfolio optimization, linear programming. (Prescriptive means that they give recommendations).

Ali E. Abbas: The need for a sound decision-making system in systems analysis.

• Decision Analysis is taken seriously in the US. Examples: Former president of Harvard University, Larry Summers, suggests that basic knowledge of decision analysis is as important today as the basics of trigonometry was before. NASA uses decision analysis and saves a lot of money by doing so. Chevron, one of worlds biggest energy companies, uses decision analysis. 
• Approximate methods are reasonable, fundamentally flawed ones are not. If an approximate method is extended and iterated, there is convergence towards the 'true results'. Fundamentally flawed methods do not converge.

Raimo P. Hämäläinen: Behavioral aspects of decision analysis.

• Models by themselves are free of behavioral effects but the behavioral effects are present when we start using the models in practice.
• In early days the behavioral research on decision analysis was concerned with eliciting accurate subjective inputs to the decision analysis model. There is still room for improvement.
• Today, behavioral elements should be considered more broadly. Examples of behavioral effects in modeling are group think, hammer-and-nail syndrome. Challenge is to facilitate the modeling process so that balanced view of the problem is maintained and few people are not allowed to dominate in group situations.
• Howard Raiffa's work exemplifies that student experiments work for developing theory. 
• Even if we think that decision making should be rational, emotions are still important parts of it.

Ralph L. Keeney: Constructing value models for applied systems analysis.

• Policy alternatives should be the input to systems analysis models and output should be the evaluation of various impacts of the alternative courses of action.
• Instead, many systems analysis use various scenarios as inputs and reports outcomes as distributions of some indicator variables.
• There are two gaps in these analyses. Not enough effort put into identifying the alternatives that are put into the models. Not enough effort put into linking the outcomes of the systems analysis model into what are the objectives of the decision makers. 
• Effort needed to think about values and objectives and model them is very small compared to the effort required to build a complicated systems analysis model. Yet, this could increase the value of the analysis to decision makers a lot.

--

I presented a poster on the topic of path dependence in systems analysis. This is joint work with my supervising professor. I was glad to find out that the ideas we present resonated with the experiences of many people. In particular many appreciated the point that modeling process can get locked-in due to behavioral factors. This increased my confidence in our research.

Abstract related to the poster presentation:

Brian Arthur, a IIASA alumni, demonstrated in his seminal paper of 1989 how increasing returns can drive path dependence in technological development and how this can cause an inferior technology to end up in a dominant market position. A similar risk exists in the use of models. The modeling community or problem solving team can become fixed to one approach and only look for refinements in the model that was initially chosen.

We bring path dependence into focus in model-based systems analysis and problem solving. There are usually alternative paths that can be followed in any modeling and problem solving process. Path dependence refers to the impact of the path on the outcomes of the process. The steps of the path include, for example, how the modeling team is formed, the framing and structuring of the problem, the choice of model, the order in which the different parts of the model are specified and solved, and the way in which data or preferences are collected.

We identify and discuss seven possibly interacting origins or drivers of path dependence: systemic origins, learning, procedure, behavior, motivation, uncertainty, and external environment. We provide suggestions on how path dependence can be dealt with.

Awareness of path dependence and its possible consequences is important in systems analysis especially when we are solving complex policy problems related to, for instance, climate change.

Edit 15.11.2015 20:40: Added a sentence in the last paragraph about modeling lock-in. Added the abstract of our poster presentation.