Navigating the problem solving path in a reflective mode

Me and co-authors wrote a post to Integration and Implementation Insights blog. Topic is reflecting on the path followed in a modelling or a problem solving project. https://i2insights.org/2017/03/28/path-dependence-in-modelling/

Systems analysis 2015

I attended the Systems Analysis 2015 conference which was organized last week from 11. to 13. November. Systems analysis means the use of modeling to support decision making.

The highlights of the event for me were the chances to get to talk with top scientists of today. It was fascinating to hear about their work and I was very happy to hear that many of them appreciated our ideas.

--

Howard Raiffa session was one of the sessions I found most interesting in the whole conference. The reason is that it dealt with a topic which I have studied a lot: Decision Analysis. Below I will explain some key ideas presented in the talks given in this session. I also provide links to video recordings of the talks:

David E. Bell: Policy, Probability and Preference.

• Modelers accept subjectivity in probabilities but many still hesitate to include subjective preferences in analyses.
• Many models that have been successful in practice are prescriptive and based on simple assumptions. Economics models, mean-variance portfolio optimization, linear programming. (Prescriptive means that they give recommendations).

Ali E. Abbas: The need for a sound decision-making system in systems analysis.

• Decision Analysis is taken seriously in the US. Examples: Former president of Harvard University, Larry Summers, suggests that basic knowledge of decision analysis is as important today as the basics of trigonometry was before. NASA uses decision analysis and saves a lot of money by doing so. Chevron, one of worlds biggest energy companies, uses decision analysis. 
• Approximate methods are reasonable, fundamentally flawed ones are not. If an approximate method is extended and iterated, there is convergence towards the 'true results'. Fundamentally flawed methods do not converge.

Raimo P. Hämäläinen: Behavioral aspects of decision analysis.

• Models by themselves are free of behavioral effects but the behavioral effects are present when we start using the models in practice.
• In early days the behavioral research on decision analysis was concerned with eliciting accurate subjective inputs to the decision analysis model. There is still room for improvement.
• Today, behavioral elements should be considered more broadly. Examples of behavioral effects in modeling are group think, hammer-and-nail syndrome. Challenge is to facilitate the modeling process so that balanced view of the problem is maintained and few people are not allowed to dominate in group situations.
• Howard Raiffa's work exemplifies that student experiments work for developing theory. 
• Even if we think that decision making should be rational, emotions are still important parts of it.

Ralph L. Keeney: Constructing value models for applied systems analysis.

• Policy alternatives should be the input to systems analysis models and output should be the evaluation of various impacts of the alternative courses of action.
• Instead, many systems analysis use various scenarios as inputs and reports outcomes as distributions of some indicator variables.
• There are two gaps in these analyses. Not enough effort put into identifying the alternatives that are put into the models. Not enough effort put into linking the outcomes of the systems analysis model into what are the objectives of the decision makers. 
• Effort needed to think about values and objectives and model them is very small compared to the effort required to build a complicated systems analysis model. Yet, this could increase the value of the analysis to decision makers a lot.

--

I presented a poster on the topic of path dependence in systems analysis. This is joint work with my supervising professor. I was glad to find out that the ideas we present resonated with the experiences of many people. In particular many appreciated the point that modeling process can get locked-in due to behavioral factors. This increased my confidence in our research.

Abstract related to the poster presentation:

Brian Arthur, a IIASA alumni, demonstrated in his seminal paper of 1989 how increasing returns can drive path dependence in technological development and how this can cause an inferior technology to end up in a dominant market position. A similar risk exists in the use of models. The modeling community or problem solving team can become fixed to one approach and only look for refinements in the model that was initially chosen.

We bring path dependence into focus in model-based systems analysis and problem solving. There are usually alternative paths that can be followed in any modeling and problem solving process. Path dependence refers to the impact of the path on the outcomes of the process. The steps of the path include, for example, how the modeling team is formed, the framing and structuring of the problem, the choice of model, the order in which the different parts of the model are specified and solved, and the way in which data or preferences are collected.

We identify and discuss seven possibly interacting origins or drivers of path dependence: systemic origins, learning, procedure, behavior, motivation, uncertainty, and external environment. We provide suggestions on how path dependence can be dealt with.

Awareness of path dependence and its possible consequences is important in systems analysis especially when we are solving complex policy problems related to, for instance, climate change.

Edit 15.11.2015 20:40: Added a sentence in the last paragraph about modeling lock-in. Added the abstract of our poster presentation.

Objectives of German students regarding their education

I am (well) alive but not having too much time to write for the blog (although I would like to since I got something I would like to write about...). Anyway, here is a very interesting figure from the working paper "Stimulating the Creation of More and Better Alternatives" by Johannes Siebert and Ralph L. Keeney. (I am very much interested in their work and inspired by it.)

Many of my readers might find it interesting to reflect these possible objectives with the objectives of their own. Could these be also your objectives? Would you add or remove something? In the spirit of the paper, looking at this figure might also stimulate other thoughts such as: Could I do better at achieving these? How?

Figure copied from this work paper.

EURO 2015

Just arrived home from Glasgow, where I attended the European Conference on Operational Research 2015. Bit jet lagged so writing this post at night.

Feelings. My spirits are high from the conference. It was inspiring, fun and I have a lot of ideas.

Behavioral Operational Research stream. Was a success. For the outsiders: There is a movement within Operational Research (OR) to pay more attention to behavioral issues related to use of models. When mathematical models are applied to real decisions (= Operational Research) there are humans involved. This aspect has been studied very little so far. Instead the research has been concerned with the mathematical models.

My presentation. Went very well! There was a big audience (~50 people) and I was able to deliver I think. The presentation can be found here.  I was a little nervous so I was maybe not so relaxed and had a little too serious tone. Positive tone is better to get other people engaged with your ideas. The message went through nonetheless and the term "path dependence" was mentioned several times in later discussions.

Social. Was fun to go out with colleagues and to get to know them better. This is important for work motivation. I also went to see a football games by the Celtics.

Research ideas. I have a lot of them now. However, it is important that I concentrate on those things I am currently doing and do not spend my time on dreaming about future projects.

Manuscript submitted, finally!

I have been really busy last couple of weeks polishing the manuscript of my first scientific article (work that has been ongoing for three years, phew!) This is the second journal we sent it to and already received one round of reviewer comments. The work is coauthored with my supervising professor.

Polishing your work is tough. It takes time and mental effort to try see your text through the eyes of an another person and then having to rewrite a lot.

Normally, the saying goes "The first 10% of effort gives 90% of the benefit". I think in scientific publishing it goes the another way around: The last 10% of effort gives 90% of the benefit. 

This is because only at the end you learn the subject so well that you are able to communicate the work fluently and overall make the work and the story compelling.

Here is a link to the manuscript "Path dependence in Operational Research - An Illustration with the Even Swaps Decision Analysis Method": http://sal.aalto.fi/publications/pdf-files/mham15b.pdf

Steps of the policy analysis process

Recently I have become quite fond of review articles, frameworks, procedures and so forth. Metascience and synthesis of information seem very relevant and important.

Today I came across this 8 step procedure for "Rational-style model-based policy analysis" by Walker (2002)*. Decided that it is worth sharing. Problem formulation, objectives and criteria, scenarios, models... This procedure got them all in nice order!

1. Formulating (and reformulating) the problem.
2.Specifying objectives and deciding upon criteria.
3.Identifying alternatives.
4.Designing scenarios to deal with uncertainty about the future.
5.Building models.
6.Collecting data.
7.Analyzing alternatives and drawing conclusions.
8.Documenting work and communicating results.

(The author notes that this process typically does not proceed in a linear fashion but a significant amount of iteration occurs between the steps.) 

*Walker, 2002. Policy analysis: a systematic approach to supporting policymaking in the public sector. Journal of Multicriteria Decision Analysis.

Lack of knowledge, chance, path dependence and ideas for dealing with them

Photo by Dioboss, CC BY-NC-SA 2.0 

There are many uncertain things in the world. Especially, if we consider from the subjective viewpoint of a person. First, she can only know so little about what exists. Second, there is luck. Third, the world is very complex so 'small' choices can lead to big outcomes.

This is the situation we have to cope with and I believe that learning to tolerate uncertainty can be very beneficial. For example, so many good things can happen just because you are in the right place at the right time. Often the cost involved in being at places is so small that it is very worthwhile to try out things and see whether they lead anywhere. In many things there is much more upside than downside potential.

In the survey we conducted with Tommi Pajala (see, earlier post) we saw that people felt uncertainty to be one of the most significant challenges related to their important personal decisions in 2015. In this post I will continue to discuss (what I believe to be) the three major sources of uncertainty: lack of knowledge, chance and path dependence.

I explain these concepts using a simple example. Guy is at a party next to a girl he is attracted to. The girl has already formed an initial impression of Guy. However, as Guy does not know this, he is uncertain about whether he should approach the girl. This lack of knowledge fortunately can be reduced. Guy can, for example, go to a common friend and ask what she thinks.

Lack of knowledge is relevant mainly before one makes his decision. Once Guy has made up his mind and asked the girl for a date, the outcome is out of his control, and will be determined partly by chance. The girl might accept, she might be too busy or maybe she has just met someone else. More generally, I view chance as the kind of uncertainty that we cannot reduce. Throwing dice provides the purest example.

Path dependence is in action after the initial consequences of a decision have taken place.One facet of path dependence is that small decision can set the course of events on a completely different path than if the decision was not made. In the context of our example, it is easy to imagine that, if the girl accepts Guys date proposal (and couple of other things fall into place) the small decision might lead to big things.

What does all of this matter? All the thinking about this topic has led me to the following conclusions:

One should be smart about reducing lack of knowledge. Discovering new information takes up your scarce resources. Prioritize!

Identifying opportunities with little downside and lot of upside potential is particularly useful. This is the case in almost all decisions where you do not need much commitment to find out if things work out or not.

Early steps matter a lot. Small things can lead to big things.

And finally, uncertainty is an unavoidable thing in life. One should not be afraid of it.

Edit (29.4. 21:40): Added "almost" to the third last paragraph.

Seminaari data-analytiikan soveltamisesta

Olen mukana Suomen Operaatiotutkimusseurassa järjestämässä toukokuun lopulla (26.5. kello 13.00 alkaen) seminaaria Data-analytiikka kasvun ja kannattavuuden moottorina. Aihe saattaa kiinnostaa joitain blogini lukijoista. Tapahtuma on opiskelijoille ilmainen.

Itseäni data-analytiikan ja päätöksenteon yhteys kiinnostaa useastakin syystä. Muutama näistä on,

• Data ei tee päätöksiä. Päätöksentekoon liittyy faktojen lisäksi aina päätöksentekijän tavoitteet tai arvot. Arvoja ja tavoitteita on monenlaisia.
• Data voi johtaa harhaan. 
• Kaikesta ei ole dataa, olemassa oleva data voi olla valikoitunutta. Esimerkiksi kurssipalautteessa korostuvat ääripäät. 
• Datan tulkinta vaatii lisäoletuksia. Jos tietty asiakasryhmä kuluttaa tuotetta X määrän Y niin onko tämä paljon vai vähän? Onko parantamisen varaa vai ei?

Tervetuloa!

Decision research perspectives to sustainable consumption

I re-discovered a presentation which I gave at the Aalto Media Factory (my first paid lecturing gig!) in 2012 titled "Perspectives to sustainable consumption from research in behavioral decision making and decision analysis". I went through the material and still found it current so I decided to share it with you.

The presentation is available here.

Ideas and claims about decision making (collected edition)

A collection of ideas and claims published earlier on this blog. Clicking the picture will lead you to the original post.

Obvious solutions have already been found. However, they can be based on assumptions that do not hold. Questioning those assumptions is an effective means to developing new creative solutions.

Being creative at Systems Thinking Applied meeting.

If one doesn't know what she's after, she may well end up in wrong places.

Structure of the system in which one is acting determines the effects of her decisions. What should fundamentally matter, are the consequences to things that one values.

Career related choices are among the most important decisions for young educated people.

Decision making should be value-focused. Structuring values may help at becoming a better decision maker.

Should you make the decision now or delay it? 

Decision trees provide a valuable tool for structuring one's thinking about a decision. The ease of reversing a decision plays a major part in deciding how to deal with electromagnetic hypersensitivity.

Importance is only half of what should count in prioritization. 

Studying while working may enable better learning.

Threats to educational equality in Finland can be analyzed by considering the likelihoods and impacts of those threats.

Questioning assumptions related to basic education might lead to some big improvements in quality of learning!

Some creative value-focused thinking going on in Finland: http://qz.com/367487/goodbye-math-and-history-finland-wants-to-abandon-teaching-subjects-at-school/

Why we do basic education? Because we want our children to learn skills and gain knowledge about different phenomena.

Some people seem to have been thinking about the useful question: Why teach by subject? 

“Finland is considering its most radical overhaul of basic education yet—abandoning teaching by subject for teaching by phenomenon.”

And arrived at some conclusions:

Improve your creativity - begin with why

The obvious solutions to most problems have already been found. However, they are often based on assumptions that do not hold. 

The question 'Why?' is an effective means to questioning those assumptions.

Two main types of assumptions are: 

• Assumptions about values, objectives, goals - the reasons behind doing things.
• Assumptions about how the world works. Whenever you ask “why should we assume that?” or “why is that so?” and do not get a satisfactory response, there might be something wrong with those assumptions.

It’s not hard to create something but it can be very hard to be creative. In the sense, that you would develop something novel, feasible and useful. I believe that using the question 'Why?' more would help us at achieving this creativity.

Illustration:

Question:  “Why should the lecturer use powerpoint slides?”

Response: “Because that is how lectures are given, to give visual learning aid for students, to provide easily accessible course material, to have a structure for what he is going to present” etc.

Q: “Why lectures have to be given that way?”

R: “Ehm…?”

Creativity point: Lecture do not have to be given that way. What is more, giving lectures is actually not an objective at all. It is just one possible way to try and achieve learning.

Q: “Why is it important to give visual learning aid?”

R: “Because it makes learning cognitively easier”

Creativity point: OK. What other means there would be to deliver even better visual learning aid for the students? Could the lecturer use more informative pictures?

People tend to make quick assumptions intuitively and to accept the narrow framing they are given. This might have been necessary in our evolution so that we could survive. However, stepping out of this 'System 1 thinking' might enable you to be more creative.

My advice is to start using the question 'why' more often. Use it until you understand what you are dealing with. After this, you should be equipped to come up with creative solutions.

Me discussing creativity with value-focused thinking at
Systems Thinking Applied meeting in Ruoholahti.

Structuring decisions: Should electromagnetic hypersensitivity be acknowledged as a medical condition caused by electric fields?

I examine the question whether Electromagnetic hypersensitivity (EHS) should be acknowledged as a medical condition caused by electric fields (EFs) or not.

Figure 1 shows my attempt to structure my thinking about this topic using a decision tree technique. In rest of the post, I explain my thoughts about this decision tree. This type of exercise can help to clarifying thinking, to guide collection of facts and to help at communicating the decision. You can find conclusions at the end of this post.

Figure 1: Decision tree about whether EHS should be acknowledged as medical condition or not. Squares are decision nodes, in which a decision can be taken. Circles are chance nodes where one of the states will be realized. Triangles are consequence nodes.

 

Dealing with uncertainty – results from a survey about important decisions people are facing in 2015

Last week, me and Tommi Pajala asked you to respond to a survey. We collected, in total, 22 serious decision problems that people are facing in 2015. Here is Tommi's analysis of the same results.

The main finding is: Career choices are important and they are challenging due to trade-offs and uncertainties. See Figure 1 for a distribution of the decisions by category. The decisions are listed in Figure 2.

Besides uncertainty in consequences, five other aspects that were found challenging are uncertainty in preferences, timing, clarifying objectives and specifying alternatives. In this post you find more about my thoughts on these and what to do about them.


Figure 1: Distribution of the decisions in categories.
Categorization based on my judgment.

Help us to write better and more influential blog posts! Respond to a short questionnaire about important decisions you'll be facing.

I and Tommi Pajala are doctoral students in decision making related fields and like to blog about the topic.

Please help us create better and more focused content by responding to a short questionnaire about the important decisions you will be facing in 2015.

Link to the questionnaire. Responding takes only a couple of minutes.

We will be posting the interesting results later in our blogs. In general, knowing what people consider to be their important decisions will help us focus on the aspects of decision making that might be relevant for them.

Tuomas and Tommi

Thank you!

Should you decide now or later? -- The pros and cons of delaying a decision

The situation where we can make a decision now or delay it occurs basically every day in our personal lives and at work. Two examples are:

• Should I rent the best apartment available at the moment (e.g. a nice two-room apartment at Helsinginkatu) or search for a better one?
• Should we (a team that builds ad campaigns) decide the sub-elements included in the advertisement campaign now, or should we start with some parts and decide about the rest as the overall picture becomes clearer?

These are obviously important decisions and as in any important decisions the rational procedure is to think of the pros and cons of the possible actions. Here the possible actions are ‘make the decision now’ and ‘delay the decision’.

Opiskeluaikaisen työnteon hyöty oppimisen kannalta

Yksilön näkökulmasta kysymys siitä, kannattaako opiskeluaikana tehdä töitä on liian pelkistetty ollaakseen relevantti. Päätös töiden tekemisestä kannattaa miettiä erikseen jokaisen työmahdollisuuden kohdalla. Päätökseen vaikuttaa niin oma tilanne kuin työtarjouksen ominaisuudet.

Yhteiskunnan näkökulmasta taas hyödyt ja haitat opiskeluaikaisesta työnteosta eivät välttämättä ole samat kuin opiskelijan näkökulmasta. Tämä johtuu siitä, että opiskelijat kilpailevat työpaikoista keskenään. Kahden opiskelijan valmistuessa se, jolla on lisäksi työkokemusta, on etulyöntiasemassa suhteessa siihen, jolla ei ole. Yhteiskunnan näkökulmasta saattaisi olla parempi, että molemmat olisivat keskittyneet nopeaan valmistumiseen.

Opiskelijoiden työnteosta on noussut paljon julkista keskustelua juurikin siitä syystä, että yhteiskunnan ja opiskelijan motiivit eivät välttämättä kohtaa. Opiskelijat argumentoivat hyötyvänsä työnteosta ja toisaalta monet väittävät, että yhteiskunta kokonaisuudessaan hyötyisi, jos opiskelijat tekisivät vähemmän töitä.

Teen jaon kolmen vaihtoehdon välillä: Omaan alaan liittymättömien töiden teko, omaan alaan liittyvien töiden teko ja ei töitä. Alla olevassa kuvassa listaan näihin liittyviä etuja.

How are Systems thinking and Value-focused decision making related?

The idea behind systems thinking is to view problems as systemic phenomena. System is a whole consisting of parts and their interrelations.  To effectively solve a problem one must understand the system around it. The better one understands the system, the more capable she is at developing an effective intervention.

The basic idea in value-focused decision making is that decision making should begin by identifying our values instead of our options. We choose between our options to best achieve what we value. Therefore, we should not narrow our thinking to the options given to us, but think creatively about what would be the best ways to achieve the things that we value.

Helping a friend with value-focused thinking

Last spring I simultaneously worked as an assistant for and took the course ’Value-Focused Thinking and Decision Making’ by Professor Ralph Keeney from Duke University who was visiting Aalto University.

One of the homework assignments for the course was to select a good friend outside the class and to have an in-depth discussion with her about her values related to some important decision she will be facing soon. I had my discussion with Kati and it was about her values related to choosing where to work after she would graduate.

I promised to Kati that I would compile the results and later present them to her in a nice format. This took some time but I got it finally done. I thought the resulting material were quite nice so I asked her if it’s okay to share them on my blog. Thanks for letting me do that Kati!

Uhkakuvia koulutuksen tasa-arvolle

Edellisessä blogipostauksessani pyysin vastaamaan kyselyyn, jossa annettiin arvioita erilaisten uhkakuvien todennäköisyyksistä ja vaikutuksista koulutukselliselle tasa-arvolle Suomessa.

Kyselylinkki johti kunkin vastaajan kohdalla satunnaisesti kahteen vaihtoehtoisesta kyselystä. Näissä oli erilainen kyselyformaatti. Tutkin, miten kyselyformaatti vaikuttaa todennäköisyys ja vaikutusarvioiden korrelaatioon. Vaikutusta ei ollut.

Uhkakuvat ja arviointiasteikot muodostettiin melko nopeasti, koska niillä ei ollut suurta merkitystä varsinaisen tutkimuskysymykseni kannalta.* Raportoin tulokset kuitenkin tässä.

Arvioitavia uhkakuvia oli yhdeksän:

Should most effort be put into what is 'most important'?

Everything could be better but we only have limited capabilities to fix and improve things. It is therefore important to ask how should we prioritize our efforts.

One idea you often hear is that we should put our efforts into what are the 'most important' things. What is considered important varies, for instance, finding a cure to a terrible disease is important because many people die from this disease. Fixing some societal problem is important because the problem causes much inequality among people. Developing friendly A.I. is important because evil A.I. could take over the world. Improving our educational system is important because it could have a huge impact on the productiveness of our society.

What is common for all of these, is that the possible benefits from solving the problem are very large. Although prioritization based on possible benefits can sometimes work as the rule of thumb it includes a pitfall: Even if we put a lot of effort into solving the very important problem, it might be that the problem won't be solved or the state of the world improves only very little.

Taking importance into account is a good start but including a second factor will greatly improve our prioritization ability.

Knowing what you want to stop wasting your time

Imagine that you would have no idea of what you want.

You would be choosing at random! 

The case for knowing what you want (the first cornerstone of good decision making) is simple: the reason for doing things is to get what you want. 

1+1=2, kannattaako Natoon liittyä?

Kannattaako Suomen liittyä Natoon? En tiedä, mutta olen eri mieltä seuraavasta presidentti Sauli Niinistön väitteestä: "Nato-jäsenyyskysymystä ei voi arvioida vain plussia ja miinuksia listaamalla, ikään kuin taulukkolaskentana.” (Suurlähettiläskokous 26.8., lähde: hs.fi).

Four cornerstones of good decision making

Apart from things out of our influence, the consequences of our actions (and omissions) determine how good our lives are. Probably most of our actions are based on routine and instincts. However, many are based on our conscious decision making. Some of these decisions are small, such as what do you choose to have for lunch today, but some are big, such as what education you want to pursue or whether you want to invest your time on writing a blog.

I am a PhD student in Systems Analysis and Decision Making. I created this blog to share my ideas about how to make better decisions to live better lives and to make the world a better place. In this post, I will explain what I see as the four cornerstones of good decision making.