Systems analysis 2015

I attended the Systems Analysis 2015 conference which was organized last week from 11. to 13. November. Systems analysis means the use of modeling to support decision making.

The highlights of the event for me were the chances to get to talk with top scientists of today. It was fascinating to hear about their work and I was very happy to hear that many of them appreciated our ideas.

--

Howard Raiffa session was one of the sessions I found most interesting in the whole conference. The reason is that it dealt with a topic which I have studied a lot: Decision Analysis. Below I will explain some key ideas presented in the talks given in this session. I also provide links to video recordings of the talks:

David E. Bell: Policy, Probability and Preference.

• Modelers accept subjectivity in probabilities but many still hesitate to include subjective preferences in analyses.
• Many models that have been successful in practice are prescriptive and based on simple assumptions. Economics models, mean-variance portfolio optimization, linear programming. (Prescriptive means that they give recommendations).

Ali E. Abbas: The need for a sound decision-making system in systems analysis.

• Decision Analysis is taken seriously in the US. Examples: Former president of Harvard University, Larry Summers, suggests that basic knowledge of decision analysis is as important today as the basics of trigonometry was before. NASA uses decision analysis and saves a lot of money by doing so. Chevron, one of worlds biggest energy companies, uses decision analysis. 
• Approximate methods are reasonable, fundamentally flawed ones are not. If an approximate method is extended and iterated, there is convergence towards the 'true results'. Fundamentally flawed methods do not converge.

Raimo P. Hämäläinen: Behavioral aspects of decision analysis.

• Models by themselves are free of behavioral effects but the behavioral effects are present when we start using the models in practice.
• In early days the behavioral research on decision analysis was concerned with eliciting accurate subjective inputs to the decision analysis model. There is still room for improvement.
• Today, behavioral elements should be considered more broadly. Examples of behavioral effects in modeling are group think, hammer-and-nail syndrome. Challenge is to facilitate the modeling process so that balanced view of the problem is maintained and few people are not allowed to dominate in group situations.
• Howard Raiffa's work exemplifies that student experiments work for developing theory. 
• Even if we think that decision making should be rational, emotions are still important parts of it.

Ralph L. Keeney: Constructing value models for applied systems analysis.

• Policy alternatives should be the input to systems analysis models and output should be the evaluation of various impacts of the alternative courses of action.
• Instead, many systems analysis use various scenarios as inputs and reports outcomes as distributions of some indicator variables.
• There are two gaps in these analyses. Not enough effort put into identifying the alternatives that are put into the models. Not enough effort put into linking the outcomes of the systems analysis model into what are the objectives of the decision makers. 
• Effort needed to think about values and objectives and model them is very small compared to the effort required to build a complicated systems analysis model. Yet, this could increase the value of the analysis to decision makers a lot.

--

I presented a poster on the topic of path dependence in systems analysis. This is joint work with my supervising professor. I was glad to find out that the ideas we present resonated with the experiences of many people. In particular many appreciated the point that modeling process can get locked-in due to behavioral factors. This increased my confidence in our research.

Abstract related to the poster presentation:

Brian Arthur, a IIASA alumni, demonstrated in his seminal paper of 1989 how increasing returns can drive path dependence in technological development and how this can cause an inferior technology to end up in a dominant market position. A similar risk exists in the use of models. The modeling community or problem solving team can become fixed to one approach and only look for refinements in the model that was initially chosen.

We bring path dependence into focus in model-based systems analysis and problem solving. There are usually alternative paths that can be followed in any modeling and problem solving process. Path dependence refers to the impact of the path on the outcomes of the process. The steps of the path include, for example, how the modeling team is formed, the framing and structuring of the problem, the choice of model, the order in which the different parts of the model are specified and solved, and the way in which data or preferences are collected.

We identify and discuss seven possibly interacting origins or drivers of path dependence: systemic origins, learning, procedure, behavior, motivation, uncertainty, and external environment. We provide suggestions on how path dependence can be dealt with.

Awareness of path dependence and its possible consequences is important in systems analysis especially when we are solving complex policy problems related to, for instance, climate change.

Edit 15.11.2015 20:40: Added a sentence in the last paragraph about modeling lock-in. Added the abstract of our poster presentation.

Manuscript submitted, finally!

I have been really busy last couple of weeks polishing the manuscript of my first scientific article (work that has been ongoing for three years, phew!) This is the second journal we sent it to and already received one round of reviewer comments. The work is coauthored with my supervising professor.

Polishing your work is tough. It takes time and mental effort to try see your text through the eyes of an another person and then having to rewrite a lot.

Normally, the saying goes "The first 10% of effort gives 90% of the benefit". I think in scientific publishing it goes the another way around: The last 10% of effort gives 90% of the benefit. 

This is because only at the end you learn the subject so well that you are able to communicate the work fluently and overall make the work and the story compelling.

Here is a link to the manuscript "Path dependence in Operational Research - An Illustration with the Even Swaps Decision Analysis Method": http://sal.aalto.fi/publications/pdf-files/mham15b.pdf

Lack of knowledge, chance, path dependence and ideas for dealing with them

Photo by Dioboss, CC BY-NC-SA 2.0 

There are many uncertain things in the world. Especially, if we consider from the subjective viewpoint of a person. First, she can only know so little about what exists. Second, there is luck. Third, the world is very complex so 'small' choices can lead to big outcomes.

This is the situation we have to cope with and I believe that learning to tolerate uncertainty can be very beneficial. For example, so many good things can happen just because you are in the right place at the right time. Often the cost involved in being at places is so small that it is very worthwhile to try out things and see whether they lead anywhere. In many things there is much more upside than downside potential.

In the survey we conducted with Tommi Pajala (see, earlier post) we saw that people felt uncertainty to be one of the most significant challenges related to their important personal decisions in 2015. In this post I will continue to discuss (what I believe to be) the three major sources of uncertainty: lack of knowledge, chance and path dependence.

I explain these concepts using a simple example. Guy is at a party next to a girl he is attracted to. The girl has already formed an initial impression of Guy. However, as Guy does not know this, he is uncertain about whether he should approach the girl. This lack of knowledge fortunately can be reduced. Guy can, for example, go to a common friend and ask what she thinks.

Lack of knowledge is relevant mainly before one makes his decision. Once Guy has made up his mind and asked the girl for a date, the outcome is out of his control, and will be determined partly by chance. The girl might accept, she might be too busy or maybe she has just met someone else. More generally, I view chance as the kind of uncertainty that we cannot reduce. Throwing dice provides the purest example.

Path dependence is in action after the initial consequences of a decision have taken place.One facet of path dependence is that small decision can set the course of events on a completely different path than if the decision was not made. In the context of our example, it is easy to imagine that, if the girl accepts Guys date proposal (and couple of other things fall into place) the small decision might lead to big things.

What does all of this matter? All the thinking about this topic has led me to the following conclusions:

One should be smart about reducing lack of knowledge. Discovering new information takes up your scarce resources. Prioritize!

Identifying opportunities with little downside and lot of upside potential is particularly useful. This is the case in almost all decisions where you do not need much commitment to find out if things work out or not.

Early steps matter a lot. Small things can lead to big things.

And finally, uncertainty is an unavoidable thing in life. One should not be afraid of it.

Edit (29.4. 21:40): Added "almost" to the third last paragraph.

Dealing with uncertainty – results from a survey about important decisions people are facing in 2015

Last week, me and Tommi Pajala asked you to respond to a survey. We collected, in total, 22 serious decision problems that people are facing in 2015. Here is Tommi's analysis of the same results.

The main finding is: Career choices are important and they are challenging due to trade-offs and uncertainties. See Figure 1 for a distribution of the decisions by category. The decisions are listed in Figure 2.

Besides uncertainty in consequences, five other aspects that were found challenging are uncertainty in preferences, timing, clarifying objectives and specifying alternatives. In this post you find more about my thoughts on these and what to do about them.


Figure 1: Distribution of the decisions in categories.
Categorization based on my judgment.