complex problem solving meaning

CONCEPTUAL ANALYSIS article

Complex problem solving: what it is and what it is not.

• 1 Department of Psychology, University of Bamberg, Bamberg, Germany
• 2 Department of Psychology, Heidelberg University, Heidelberg, Germany

Computer-simulated scenarios have been part of psychological research on problem solving for more than 40 years. The shift in emphasis from simple toy problems to complex, more real-life oriented problems has been accompanied by discussions about the best ways to assess the process of solving complex problems. Psychometric issues such as reliable assessments and addressing correlations with other instruments have been in the foreground of these discussions and have left the content validity of complex problem solving in the background. In this paper, we return the focus to content issues and address the important features that define complex problems.

Succeeding in the 21st century requires many competencies, including creativity, life-long learning, and collaboration skills (e.g., National Research Council, 2011 ; Griffin and Care, 2015 ), to name only a few. One competence that seems to be of central importance is the ability to solve complex problems ( Mainzer, 2009 ). Mainzer quotes the Nobel prize winner Simon (1957) who wrote as early as 1957:

The capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problem whose solution is required for objectively rational behavior in the real world or even for a reasonable approximation to such objective rationality. (p. 198)

The shift from well-defined to ill-defined problems came about as a result of a disillusion with the “general problem solver” ( Newell et al., 1959 ): The general problem solver was a computer software intended to solve all kind of problems that can be expressed through well-formed formulas. However, it soon became clear that this procedure was in fact a “special problem solver” that could only solve well-defined problems in a closed space. But real-world problems feature open boundaries and have no well-determined solution. In fact, the world is full of wicked problems and clumsy solutions ( Verweij and Thompson, 2006 ). As a result, solving well-defined problems and solving ill-defined problems requires different cognitive processes ( Schraw et al., 1995 ; but see Funke, 2010 ).

Well-defined problems have a clear set of means for reaching a precisely described goal state. For example: in a match-stick arithmetic problem, a person receives a false arithmetic expression constructed out of matchsticks (e.g., IV = III + III). According to the instructions, moving one of the matchsticks will make the equations true. Here, both the problem (find the appropriate stick to move) and the goal state (true arithmetic expression; solution is: VI = III + III) are defined clearly.

Ill-defined problems have no clear problem definition, their goal state is not defined clearly, and the means of moving towards the (diffusely described) goal state are not clear. For example: The goal state for solving the political conflict in the near-east conflict between Israel and Palestine is not clearly defined (living in peaceful harmony with each other?) and even if the conflict parties would agree on a two-state solution, this goal again leaves many issues unresolved. This type of problem is called a “complex problem” and is of central importance to this paper. All psychological processes that occur within individual persons and deal with the handling of such ill-defined complex problems will be subsumed under the umbrella term “complex problem solving” (CPS).

Systematic research on CPS started in the 1970s with observations of the behavior of participants who were confronted with computer simulated microworlds. For example, in one of those microworlds participants assumed the role of executives who were tasked to manage a company over a certain period of time (see Brehmer and Dörner, 1993 , for a discussion of this methodology). Today, CPS is an established concept and has even influenced large-scale assessments such as PISA (“Programme for International Student Assessment”), organized by the Organization for Economic Cooperation and Development ( OECD, 2014 ). According to the World Economic Forum, CPS is one of the most important competencies required in the future ( World Economic Forum, 2015 ). Numerous articles on the subject have been published in recent years, documenting the increasing research activity relating to this field. In the following collection of papers we list only those published in 2010 and later: theoretical papers ( Blech and Funke, 2010 ; Funke, 2010 ; Knauff and Wolf, 2010 ; Leutner et al., 2012 ; Selten et al., 2012 ; Wüstenberg et al., 2012 ; Greiff et al., 2013b ; Fischer and Neubert, 2015 ; Schoppek and Fischer, 2015 ), papers about measurement issues ( Danner et al., 2011a ; Greiff et al., 2012 , 2015a ; Alison et al., 2013 ; Gobert et al., 2015 ; Greiff and Fischer, 2013 ; Herde et al., 2016 ; Stadler et al., 2016 ), papers about applications ( Fischer and Neubert, 2015 ; Ederer et al., 2016 ; Tremblay et al., 2017 ), papers about differential effects ( Barth and Funke, 2010 ; Danner et al., 2011b ; Beckmann and Goode, 2014 ; Greiff and Neubert, 2014 ; Scherer et al., 2015 ; Meißner et al., 2016 ; Wüstenberg et al., 2016 ), one paper about developmental effects ( Frischkorn et al., 2014 ), one paper with a neuroscience background ( Osman, 2012 ) 1 , papers about cultural differences ( Güss and Dörner, 2011 ; Sonnleitner et al., 2014 ; Güss et al., 2015 ), papers about validity issues ( Goode and Beckmann, 2010 ; Greiff et al., 2013c ; Schweizer et al., 2013 ; Mainert et al., 2015 ; Funke et al., 2017 ; Greiff et al., 2017 , 2015b ; Kretzschmar et al., 2016 ; Kretzschmar, 2017 ), review papers and meta-analyses ( Osman, 2010 ; Stadler et al., 2015 ), and finally books ( Qudrat-Ullah, 2015 ; Csapó and Funke, 2017b ) and book chapters ( Funke, 2012 ; Hotaling et al., 2015 ; Funke and Greiff, 2017 ; Greiff and Funke, 2017 ; Csapó and Funke, 2017a ; Fischer et al., 2017 ; Molnàr et al., 2017 ; Tobinski and Fritz, 2017 ; Viehrig et al., 2017 ). In addition, a new “Journal of Dynamic Decision Making” (JDDM) has been launched ( Fischer et al., 2015 , 2016 ) to give the field an open-access outlet for research and discussion.

This paper aims to clarify aspects of validity: what should be meant by the term CPS and what not? This clarification seems necessary because misunderstandings in recent publications provide – from our point of view – a potentially misleading picture of the construct. We start this article with a historical review before attempting to systematize different positions. We conclude with a working definition.

Historical Review

The concept behind CPS goes back to the German phrase “komplexes Problemlösen” (CPS; the term “komplexes Problemlösen” was used as a book title by Funke, 1986 ). The concept was introduced in Germany by Dörner and colleagues in the mid-1970s (see Dörner et al., 1975 ; Dörner, 1975 ) for the first time. The German phrase was later translated to CPS in the titles of two edited volumes by Sternberg and Frensch (1991) and Frensch and Funke (1995a) that collected papers from different research traditions. Even though it looks as though the term was coined in the 1970s, Edwards (1962) used the term “dynamic decision making” to describe decisions that come in a sequence. He compared static with dynamic decision making, writing:

In dynamic situations, a new complication not found in the static situations arises. The environment in which the decision is set may be changing, either as a function of the sequence of decisions, or independently of them, or both. It is this possibility of an environment which changes while you collect information about it which makes the task of dynamic decision theory so difficult and so much fun. (p. 60)

The ability to solve complex problems is typically measured via dynamic systems that contain several interrelated variables that participants need to alter. Early work (see, e.g., Dörner, 1980 ) used a simulation scenario called “Lohhausen” that contained more than 2000 variables that represented the activities of a small town: Participants had to take over the role of a mayor for a simulated period of 10 years. The simulation condensed these ten years to ten hours in real time. Later, researchers used smaller dynamic systems as scenarios either based on linear equations (see, e.g., Funke, 1993 ) or on finite state automata (see, e.g., Buchner and Funke, 1993 ). In these contexts, CPS consisted of the identification and control of dynamic task environments that were previously unknown to the participants. Different task environments came along with different degrees of fidelity ( Gray, 2002 ).

According to Funke (2012) , the typical attributes of complex systems are (a) complexity of the problem situation which is usually represented by the sheer number of involved variables; (b) connectivity and mutual dependencies between involved variables; (c) dynamics of the situation, which reflects the role of time and developments within a system; (d) intransparency (in part or full) about the involved variables and their current values; and (e) polytely (greek term for “many goals”), representing goal conflicts on different levels of analysis. This mixture of features is similar to what is called VUCA (volatility, uncertainty, complexity, ambiguity) in modern approaches to management (e.g., Mack et al., 2016 ).

In his evaluation of the CPS movement, Sternberg (1995) compared (young) European approaches to CPS with (older) American research on expertise. His analysis of the differences between the European and American traditions shows advantages but also potential drawbacks for each side. He states (p. 301): “I believe that although there are problems with the European approach, it deals with some fundamental questions that American research scarcely addresses.” So, even though the echo of the European approach did not enjoy strong resonance in the US at that time, it was valued by scholars like Sternberg and others. Before attending to validity issues, we will first present a short review of different streams.

Different Approaches to CPS

In the short history of CPS research, different approaches can be identified ( Buchner, 1995 ; Fischer et al., 2017 ). To systematize, we differentiate between the following five lines of research:

(a) The search for individual differences comprises studies identifying interindividual differences that affect the ability to solve complex problems. This line of research is reflected, for example, in the early work by Dörner et al. (1983) and their “Lohhausen” study. Here, naïve student participants took over the role of the mayor of a small simulated town named Lohhausen for a simulation period of ten years. According to the results of the authors, it is not intelligence (as measured by conventional IQ tests) that predicts performance, but it is the ability to stay calm in the face of a challenging situation and the ability to switch easily between an analytic mode of processing and a more holistic one.

(b) The search for cognitive processes deals with the processes behind understanding complex dynamic systems. Representative of this line of research is, for example, Berry and Broadbent’s (1984) work on implicit and explicit learning processes when people interact with a dynamic system called “Sugar Production”. They found that those who perform best in controlling a dynamic system can do so implicitly, without explicit knowledge of details regarding the systems’ relations.

(c) The search for system factors seeks to identify the aspects of dynamic systems that determine the difficulty of complex problems and make some problems harder than others. Representative of this line of research is, for example, work by Funke (1985) , who systematically varied the number of causal effects within a dynamic system or the presence/absence of eigendynamics. He found, for example, that solution quality decreases as the number of systems relations increases.

(d) The psychometric approach develops measurement instruments that can be used as an alternative to classical IQ tests, as something that goes “beyond IQ”. The MicroDYN approach ( Wüstenberg et al., 2012 ) is representative for this line of research that presents an alternative to reasoning tests (like Raven matrices). These authors demonstrated that a small improvement in predicting school grade point average beyond reasoning is possible with MicroDYN tests.

(e) The experimental approach explores CPS under different experimental conditions. This approach uses CPS assessment instruments to test hypotheses derived from psychological theories and is sometimes used in research about cognitive processes (see above). Exemplary for this line of research is the work by Rohe et al. (2016) , who test the usefulness of “motto goals” in the context of complex problems compared to more traditional learning and performance goals. Motto goals differ from pure performance goals by activating positive affect and should lead to better goal attainment especially in complex situations (the mentioned study found no effect).

To be clear: these five approaches are not mutually exclusive and do overlap. But the differentiation helps to identify different research communities and different traditions. These communities had different opinions about scaling complexity.

The Race for Complexity: Use of More and More Complex Systems

In the early years of CPS research, microworlds started with systems containing about 20 variables (“Tailorshop”), soon reached 60 variables (“Moro”), and culminated in systems with about 2000 variables (“Lohhausen”). This race for complexity ended with the introduction of the concept of “minimal complex systems” (MCS; Greiff and Funke, 2009 ; Funke and Greiff, 2017 ), which ushered in a search for the lower bound of complexity instead of the higher bound, which could not be defined as easily. The idea behind this concept was that whereas the upper limits of complexity are unbound, the lower limits might be identifiable. Imagine starting with a simple system containing two variables with a simple linear connection between them; then, step by step, increase the number of variables and/or the type of connections. One soon reaches a point where the system can no longer be considered simple and has become a “complex system”. This point represents a minimal complex system. Despite some research having been conducted in this direction, the point of transition from simple to complex has not been identified clearly as of yet.

Some years later, the original “minimal complex systems” approach ( Greiff and Funke, 2009 ) shifted to the “multiple complex systems” approach ( Greiff et al., 2013a ). This shift is more than a slight change in wording: it is important because it taps into the issue of validity directly. Minimal complex systems have been introduced in the context of challenges from large-scale assessments like PISA 2012 that measure new aspects of problem solving, namely interactive problems besides static problem solving ( Greiff and Funke, 2017 ). PISA 2012 required test developers to remain within testing time constraints (given by the school class schedule). Also, test developers needed a large item pool for the construction of a broad class of problem solving items. It was clear from the beginning that MCS deal with simple dynamic situations that require controlled interaction: the exploration and control of simple ticket machines, simple mobile phones, or simple MP3 players (all of these example domains were developed within PISA 2012) – rather than really complex situations like managerial or political decision making.

As a consequence of this subtle but important shift in interpreting the letters MCS, the definition of CPS became a subject of debate recently ( Funke, 2014a ; Greiff and Martin, 2014 ; Funke et al., 2017 ). In the words of Funke (2014b , p. 495):

It is funny that problems that nowadays come under the term ‘CPS’, are less complex (in terms of the previously described attributes of complex situations) than at the beginning of this new research tradition. The emphasis on psychometric qualities has led to a loss of variety. Systems thinking requires more than analyzing models with two or three linear equations – nonlinearity, cyclicity, rebound effects, etc. are inherent features of complex problems and should show up at least in some of the problems used for research and assessment purposes. Minimal complex systems run the danger of becoming minimal valid systems.

Searching for minimal complex systems is not the same as gaining insight into the way how humans deal with complexity and uncertainty. For psychometric purposes, it is appropriate to reduce complexity to a minimum; for understanding problem solving under conditions of overload, intransparency, and dynamics, it is necessary to realize those attributes with reasonable strength. This aspect is illustrated in the next section.

Importance of the Validity Issue

The most important reason for discussing the question of what complex problem solving is and what it is not stems from its phenomenology: if we lose sight of our phenomena, we are no longer doing good psychology. The relevant phenomena in the context of complex problems encompass many important aspects. In this section, we discuss four phenomena that are specific to complex problems. We consider these phenomena as critical for theory development and for the construction of assessment instruments (i.e., microworlds). These phenomena require theories for explaining them and they require assessment instruments eliciting them in a reliable way.

The first phenomenon is the emergency reaction of the intellectual system ( Dörner, 1980 ): When dealing with complex systems, actors tend to (a) reduce their intellectual level by decreasing self-reflections, by decreasing their intentions, by stereotyping, and by reducing their realization of intentions, (b) they show a tendency for fast action with increased readiness for risk, with increased violations of rules, and with increased tendency to escape the situation, and (c) they degenerate their hypotheses formation by construction of more global hypotheses and reduced tests of hypotheses, by increasing entrenchment, and by decontextualizing their goals. This phenomenon illustrates the strong connection between cognition, emotion, and motivation that has been emphasized by Dörner (see, e.g., Dörner and Güss, 2013 ) from the beginning of his research tradition; the emergency reaction reveals a shift in the mode of information processing under the pressure of complexity.

The second phenomenon comprises cross-cultural differences with respect to strategy use ( Strohschneider and Güss, 1999 ; Güss and Wiley, 2007 ; Güss et al., 2015 ). Results from complex task environments illustrate the strong influence of context and background knowledge to an extent that cannot be found for knowledge-poor problems. For example, in a comparison between Brazilian and German participants, it turned out that Brazilians accept the given problem descriptions and are more optimistic about the results of their efforts, whereas Germans tend to inquire more about the background of the problems and take a more active approach but are less optimistic (according to Strohschneider and Güss, 1998 , p. 695).

The third phenomenon relates to failures that occur during the planning and acting stages ( Jansson, 1994 ; Ramnarayan et al., 1997 ), illustrating that rational procedures seem to be unlikely to be used in complex situations. The potential for failures ( Dörner, 1996 ) rises with the complexity of the problem. Jansson (1994) presents seven major areas for failures with complex situations: acting directly on current feedback; insufficient systematization; insufficient control of hypotheses and strategies; lack of self-reflection; selective information gathering; selective decision making; and thematic vagabonding.

The fourth phenomenon describes (a lack of) training and transfer effects ( Kretzschmar and Süß, 2015 ), which again illustrates the context dependency of strategies and knowledge (i.e., there is no strategy that is so universal that it can be used in many different problem situations). In their own experiment, the authors could show training effects only for knowledge acquisition, not for knowledge application. Only with specific feedback, performance in complex environments can be increased ( Engelhart et al., 2017 ).

These four phenomena illustrate why the type of complexity (or degree of simplicity) used in research really matters. Furthermore, they demonstrate effects that are specific for complex problems, but not for toy problems. These phenomena direct the attention to the important question: does the stimulus material used (i.e., the computer-simulated microworld) tap and elicit the manifold of phenomena described above?

Dealing with partly unknown complex systems requires courage, wisdom, knowledge, grit, and creativity. In creativity research, “little c” and “BIG C” are used to differentiate between everyday creativity and eminent creativity ( Beghetto and Kaufman, 2007 ; Kaufman and Beghetto, 2009 ). Everyday creativity is important for solving everyday problems (e.g., finding a clever fix for a broken spoke on my bicycle), eminent creativity changes the world (e.g., inventing solar cells for energy production). Maybe problem solving research should use a similar differentiation between “little p” and “BIG P” to mark toy problems on the one side and big societal challenges on the other. The question then remains: what can we learn about BIG P by studying little p? What phenomena are present in both types, and what phenomena are unique to each of the two extremes?

Discussing research on CPS requires reflecting on the field’s research methods. Even if the experimental approach has been successful for testing hypotheses (for an overview of older work, see Funke, 1995 ), other methods might provide additional and novel insights. Complex phenomena require complex approaches to understand them. The complex nature of complex systems imposes limitations on psychological experiments: The more complex the environments, the more difficult is it to keep conditions under experimental control. And if experiments have to be run in labs one should bring enough complexity into the lab to establish the phenomena mentioned, at least in part.

There are interesting options to be explored (again): think-aloud protocols , which have been discredited for many years ( Nisbett and Wilson, 1977 ) and yet are a valuable source for theory testing ( Ericsson and Simon, 1983 ); introspection ( Jäkel and Schreiber, 2013 ), which seems to be banned from psychological methods but nevertheless offers insights into thought processes; the use of life-streaming ( Wendt, 2017 ), a medium in which streamers generate a video stream of think-aloud data in computer-gaming; political decision-making ( Dhami et al., 2015 ) that demonstrates error-proneness in groups; historical case studies ( Dörner and Güss, 2011 ) that give insights into the thinking styles of political leaders; the use of the critical incident technique ( Reuschenbach, 2008 ) to construct complex scenarios; and simulations with different degrees of fidelity ( Gray, 2002 ).

The methods tool box is full of instruments that have to be explored more carefully before any individual instrument receives a ban or research narrows its focus to only one paradigm for data collection. Brehmer and Dörner (1993) discussed the tensions between “research in the laboratory and research in the field”, optimistically concluding “that the new methodology of computer-simulated microworlds will provide us with the means to bridge the gap between the laboratory and the field” (p. 183). The idea behind this optimism was that computer-simulated scenarios would bring more complexity from the outside world into the controlled lab environment. But this is not true for all simulated scenarios. In his paper on simulated environments, Gray (2002) differentiated computer-simulated environments with respect to three dimensions: (1) tractability (“the more training subjects require before they can use a simulated task environment, the less tractable it is”, p. 211), correspondence (“High correspondence simulated task environments simulate many aspects of one task environment. Low correspondence simulated task environments simulate one aspect of many task environments”, p. 214), and engagement (“A simulated task environment is engaging to the degree to which it involves and occupies the participants; that is, the degree to which they agree to take it seriously”, p. 217). But the mere fact that a task is called a “computer-simulated task environment” does not mean anything specific in terms of these three dimensions. This is one of several reasons why we should differentiate between those studies that do not address the core features of CPS and those that do.

What is not CPS?

Even though a growing number of references claiming to deal with complex problems exist (e.g., Greiff and Wüstenberg, 2015 ; Greiff et al., 2016 ), it would be better to label the requirements within these tasks “dynamic problem solving,” as it has been done adequately in earlier work ( Greiff et al., 2012 ). The dynamics behind on-off-switches ( Thimbleby, 2007 ) are remarkable but not really complex. Small nonlinear systems that exhibit stunningly complex and unstable behavior do exist – but they are not used in psychometric assessments of so-called CPS. There are other small systems (like MicroDYN scenarios: Greiff and Wüstenberg, 2014 ) that exhibit simple forms of system behavior that are completely predictable and stable. This type of simple systems is used frequently. It is even offered commercially as a complex problem-solving test called COMPRO ( Greiff and Wüstenberg, 2015 ) for business applications. But a closer look reveals that the label is not used correctly; within COMPRO, the used linear equations are far from being complex and the system can be handled properly by using only one strategy (see for more details Funke et al., 2017 ).

Why do simple linear systems not fall within CPS? At the surface, nonlinear and linear systems might appear similar because both only include 3–5 variables. But the difference is in terms of systems behavior as well as strategies and learning. If the behavior is simple (as in linear systems where more input is related to more output and vice versa), the system can be easily understood (participants in the MicroDYN world have 3 minutes to explore a complex system). If the behavior is complex (as in systems that contain strange attractors or negative feedback loops), things become more complicated and much more observation is needed to identify the hidden structure of the unknown system ( Berry and Broadbent, 1984 ; Hundertmark et al., 2015 ).

Another issue is learning. If tasks can be solved using a single (and not so complicated) strategy, steep learning curves are to be expected. The shift from problem solving to learned routine behavior occurs rapidly, as was demonstrated by Luchins (1942) . In his water jar experiments, participants quickly acquired a specific strategy (a mental set) for solving certain measurement problems that they later continued applying to problems that would have allowed for easier approaches. In the case of complex systems, learning can occur only on very general, abstract levels because it is difficult for human observers to make specific predictions. Routines dealing with complex systems are quite different from routines relating to linear systems.

What should not be studied under the label of CPS are pure learning effects, multiple-cue probability learning, or tasks that can be solved using a single strategy. This last issue is a problem for MicroDYN tasks that rely strongly on the VOTAT strategy (“vary one thing at a time”; see Tschirgi, 1980 ). In real-life, it is hard to imagine a business manager trying to solve her or his problems by means of VOTAT.

What is CPS?

In the early days of CPS research, planet Earth’s dynamics and complexities gained attention through such books as “The limits to growth” ( Meadows et al., 1972 ) and “Beyond the limits” ( Meadows et al., 1992 ). In the current decade, for example, the World Economic Forum (2016) attempts to identify the complexities and risks of our modern world. In order to understand the meaning of complexity and uncertainty, taking a look at the worlds’ most pressing issues is helpful. Searching for strategies to cope with these problems is a difficult task: surely there is no place for the simple principle of “vary-one-thing-at-a-time” (VOTAT) when it comes to global problems. The VOTAT strategy is helpful in the context of simple problems ( Wüstenberg et al., 2014 ); therefore, whether or not VOTAT is helpful in a given problem situation helps us distinguish simple from complex problems.

Because there exist no clear-cut strategies for complex problems, typical failures occur when dealing with uncertainty ( Dörner, 1996 ; Güss et al., 2015 ). Ramnarayan et al. (1997) put together a list of generic errors (e.g., not developing adequate action plans; lack of background control; learning from experience blocked by stereotype knowledge; reactive instead of proactive action) that are typical of knowledge-rich complex systems but cannot be found in simple problems.

Complex problem solving is not a one-dimensional, low-level construct. On the contrary, CPS is a multi-dimensional bundle of competencies existing at a high level of abstraction, similar to intelligence (but going beyond IQ). As Funke et al. (2018) state: “Assessment of transversal (in educational contexts: cross-curricular) competencies cannot be done with one or two types of assessment. The plurality of skills and competencies requires a plurality of assessment instruments.”

There are at least three different aspects of complex systems that are part of our understanding of a complex system: (1) a complex system can be described at different levels of abstraction; (2) a complex system develops over time, has a history, a current state, and a (potentially unpredictable) future; (3) a complex system is knowledge-rich and activates a large semantic network, together with a broad list of potential strategies (domain-specific as well as domain-general).

Complex problem solving is not only a cognitive process but is also an emotional one ( Spering et al., 2005 ; Barth and Funke, 2010 ) and strongly dependent on motivation (low-stakes versus high-stakes testing; see Hermes and Stelling, 2016 ).

Furthermore, CPS is a dynamic process unfolding over time, with different phases and with more differentiation than simply knowledge acquisition and knowledge application. Ideally, the process should entail identifying problems (see Dillon, 1982 ; Lee and Cho, 2007 ), even if in experimental settings, problems are provided to participants a priori . The more complex and open a given situation, the more options can be generated (T. S. Schweizer et al., 2016 ). In closed problems, these processes do not occur in the same way.

In analogy to the difference between formative (process-oriented) and summative (result-oriented) assessment ( Wiliam and Black, 1996 ; Bennett, 2011 ), CPS should not be reduced to the mere outcome of a solution process. The process leading up to the solution, including detours and errors made along the way, might provide a more differentiated impression of a person’s problem-solving abilities and competencies than the final result of such a process. This is one of the reasons why CPS environments are not, in fact, complex intelligence tests: research on CPS is not only about the outcome of the decision process, but it is also about the problem-solving process itself.

Complex problem solving is part of our daily life: finding the right person to share one’s life with, choosing a career that not only makes money, but that also makes us happy. Of course, CPS is not restricted to personal problems – life on Earth gives us many hard nuts to crack: climate change, population growth, the threat of war, the use and distribution of natural resources. In sum, many societal challenges can be seen as complex problems. To reduce that complexity to a one-hour lab activity on a random Friday afternoon puts it out of context and does not address CPS issues.

Theories about CPS should specify which populations they apply to. Across populations, one thing to consider is prior knowledge. CPS research with experts (e.g., Dew et al., 2009 ) is quite different from problem solving research using tasks that intentionally do not require any specific prior knowledge (see, e.g., Beckmann and Goode, 2014 ).

More than 20 years ago, Frensch and Funke (1995b) defined CPS as follows:

CPS occurs to overcome barriers between a given state and a desired goal state by means of behavioral and/or cognitive, multi-step activities. The given state, goal state, and barriers between given state and goal state are complex, change dynamically during problem solving, and are intransparent. The exact properties of the given state, goal state, and barriers are unknown to the solver at the outset. CPS implies the efficient interaction between a solver and the situational requirements of the task, and involves a solver’s cognitive, emotional, personal, and social abilities and knowledge. (p. 18)

The above definition is rather formal and does not account for content or relations between the simulation and the real world. In a sense, we need a new definition of CPS that addresses these issues. Based on our previous arguments, we propose the following working definition:

Complex problem solving is a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.

The main differences to the older definition lie in the emphasis on (a) the self-regulation of processes, (b) creativity (as opposed to routine behavior), (c) the bricolage type of solution, and (d) the role of high-stakes challenges. Our new definition incorporates some aspects that have been discussed in this review but were not reflected in the 1995 definition, which focused on attributes of complex problems like dynamics or intransparency.

This leads us to the final reflection about the role of CPS for dealing with uncertainty and complexity in real life. We will distinguish thinking from reasoning and introduce the sense of possibility as an important aspect of validity.

CPS as Combining Reasoning and Thinking in an Uncertain Reality

Leading up to the Battle of Borodino in Leo Tolstoy’s novel “War and Peace”, Prince Andrei Bolkonsky explains the concept of war to his friend Pierre. Pierre expects war to resemble a game of chess: You position the troops and attempt to defeat your opponent by moving them in different directions.

“Far from it!”, Andrei responds. “In chess, you know the knight and his moves, you know the pawn and his combat strength. While in war, a battalion is sometimes stronger than a division and sometimes weaker than a company; it all depends on circumstances that can never be known. In war, you do not know the position of your enemy; some things you might be able to observe, some things you have to divine (but that depends on your ability to do so!) and many things cannot even be guessed at. In chess, you can see all of your opponent’s possible moves. In war, that is impossible. If you decide to attack, you cannot know whether the necessary conditions are met for you to succeed. Many a time, you cannot even know whether your troops will follow your orders…”

In essence, war is characterized by a high degree of uncertainty. A good commander (or politician) can add to that what he or she sees, tentatively fill in the blanks – and not just by means of logical deduction but also by intelligently bridging missing links. A bad commander extrapolates from what he sees and thus arrives at improper conclusions.

Many languages differentiate between two modes of mentalizing; for instance, the English language distinguishes between ‘thinking’ and ‘reasoning’. Reasoning denotes acute and exact mentalizing involving logical deductions. Such deductions are usually based on evidence and counterevidence. Thinking, however, is what is required to write novels. It is the construction of an initially unknown reality. But it is not a pipe dream, an unfounded process of fabrication. Rather, thinking asks us to imagine reality (“Wirklichkeitsfantasie”). In other words, a novelist has to possess a “sense of possibility” (“Möglichkeitssinn”, Robert Musil; in German, sense of possibility is often used synonymously with imagination even though imagination is not the same as sense of possibility, for imagination also encapsulates the impossible). This sense of possibility entails knowing the whole (or several wholes) or being able to construe an unknown whole that could accommodate a known part. The whole has to align with sociological and geographical givens, with the mentality of certain peoples or groups, and with the laws of physics and chemistry. Otherwise, the entire venture is ill-founded. A sense of possibility does not aim for the moon but imagines something that might be possible but has not been considered possible or even potentially possible so far.

Thinking is a means to eliminate uncertainty. This process requires both of the modes of thinking we have discussed thus far. Economic, political, or ecological decisions require us to first consider the situation at hand. Though certain situational aspects can be known, but many cannot. In fact, von Clausewitz (1832) posits that only about 25% of the necessary information is available when a military decision needs to be made. Even then, there is no way to guarantee that whatever information is available is also correct: Even if a piece of information was completely accurate yesterday, it might no longer apply today.

Once our sense of possibility has helped grasping a situation, problem solvers need to call on their reasoning skills. Not every situation requires the same action, and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.

If we are to believe Tuchman’s (1984) book, “The March of Folly”, most politicians and commanders are fools. According to Tuchman, not much has changed in the 3300 years that have elapsed since the misguided Trojans decided to welcome the left-behind wooden horse into their city that would end up dismantling Troy’s defensive walls. The Trojans, too, had been warned, but decided not to heed the warning. Although Laocoön had revealed the horse’s true nature to them by attacking it with a spear, making the weapons inside the horse ring, the Trojans refused to see the forest for the trees. They did not want to listen, they wanted the war to be over, and this desire ended up shaping their perception.

The objective of psychology is to predict and explain human actions and behavior as accurately as possible. However, thinking cannot be investigated by limiting its study to neatly confined fractions of reality such as the realms of propositional logic, chess, Go tasks, the Tower of Hanoi, and so forth. Within these systems, there is little need for a sense of possibility. But a sense of possibility – the ability to divine and construe an unknown reality – is at least as important as logical reasoning skills. Not researching the sense of possibility limits the validity of psychological research. All economic and political decision making draws upon this sense of possibility. By not exploring it, psychological research dedicated to the study of thinking cannot further the understanding of politicians’ competence and the reasons that underlie political mistakes. Christopher Clark identifies European diplomats’, politicians’, and commanders’ inability to form an accurate representation of reality as a reason for the outbreak of World War I. According to Clark’s (2012) book, “The Sleepwalkers”, the politicians of the time lived in their own make-believe world, wrongfully assuming that it was the same world everyone else inhabited. If CPS research wants to make significant contributions to the world, it has to acknowledge complexity and uncertainty as important aspects of it.

For more than 40 years, CPS has been a new subject of psychological research. During this time period, the initial emphasis on analyzing how humans deal with complex, dynamic, and uncertain situations has been lost. What is subsumed under the heading of CPS in modern research has lost the original complexities of real-life problems. From our point of view, the challenges of the 21st century require a return to the origins of this research tradition. We would encourage researchers in the field of problem solving to come back to the original ideas. There is enough complexity and uncertainty in the world to be studied. Improving our understanding of how humans deal with these global and pressing problems would be a worthwhile enterprise.

Author Contributions

JF drafted a first version of the manuscript, DD added further text and commented on the draft. JF finalized the manuscript.

Authors Note

After more than 40 years of controversial discussions between both authors, this is the first joint paper. We are happy to have done this now! We have found common ground!

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank the Deutsche Forschungsgemeinschaft (DFG) for the continuous support of their research over many years. Thanks to Daniel Holt for his comments on validity issues, thanks to Julia Nolte who helped us by translating German text excerpts into readable English and helped us, together with Keri Hartman, to improve our style and grammar – thanks for that! We also thank the two reviewers for their helpful critical comments on earlier versions of this manuscript. Finally, we acknowledge financial support by Deutsche Forschungsgemeinschaft and Ruprecht-Karls-Universität Heidelberg within their funding programme Open Access Publishing .

• ^ The fMRI-paper from Anderson (2012) uses the term “complex problem solving” for tasks that do not fall in our understanding of CPS and is therefore excluded from this list.

Alison, L., van den Heuvel, C., Waring, S., Power, N., Long, A., O’Hara, T., et al. (2013). Immersive simulated learning environments for researching critical incidents: a knowledge synthesis of the literature and experiences of studying high-risk strategic decision making. J. Cogn. Eng. Deci. Mak. 7, 255–272. doi: 10.1177/1555343412468113

CrossRef Full Text | Google Scholar

Anderson, J. R. (2012). Tracking problem solving by multivariate pattern analysis and hidden markov model algorithms. Neuropsychologia 50, 487–498. doi: 10.1016/j.neuropsychologia.2011.07.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Barth, C. M., and Funke, J. (2010). Negative affective environments improve complex solving performance. Cogn. Emot. 24, 1259–1268. doi: 10.1080/02699930903223766

Beckmann, J. F., and Goode, N. (2014). The benefit of being naïve and knowing it: the unfavourable impact of perceived context familiarity on learning in complex problem solving tasks. Instruct. Sci. 42, 271–290. doi: 10.1007/s11251-013-9280-7

Beghetto, R. A., and Kaufman, J. C. (2007). Toward a broader conception of creativity: a case for “mini-c” creativity. Psychol. Aesthetics Creat. Arts 1, 73–79. doi: 10.1037/1931-3896.1.2.73

Bennett, R. E. (2011). Formative assessment: a critical review. Assess. Educ. Princ. Policy Pract. 18, 5–25. doi: 10.1080/0969594X.2010.513678

Berry, D. C., and Broadbent, D. E. (1984). On the relationship between task performance and associated verbalizable knowledge. Q. J. Exp. Psychol. 36, 209–231. doi: 10.1080/14640748408402156

Blech, C., and Funke, J. (2010). You cannot have your cake and eat it, too: how induced goal conflicts affect complex problem solving. Open Psychol. J. 3, 42–53. doi: 10.2174/1874350101003010042

Brehmer, B., and Dörner, D. (1993). Experiments with computer-simulated microworlds: escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Comput. Hum. Behav. 9, 171–184. doi: 10.1016/0747-5632(93)90005-D

Buchner, A. (1995). “Basic topics and approaches to the study of complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 27–63.

Google Scholar

Buchner, A., and Funke, J. (1993). Finite state automata: dynamic task environments in problem solving research. Q. J. Exp. Psychol. 46A, 83–118. doi: 10.1080/14640749308401068

Clark, C. (2012). The Sleepwalkers: How Europe Went to War in 1914 . London: Allen Lane.

Csapó, B., and Funke, J. (2017a). “The development and assessment of problem solving in 21st-century schools,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 19–31.

Csapó, B., and Funke, J. (eds) (2017b). The Nature of Problem Solving. Using Research to Inspire 21st Century Learning. Paris: OECD Publishing.

Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., et al. (2011a). Measuring performance in dynamic decision making. Reliability and validity of the Tailorshop simulation. J. Ind. Differ. 32, 225–233. doi: 10.1027/1614-0001/a000055

CrossRef Full Text

Danner, D., Hagemann, D., Schankin, A., Hager, M., and Funke, J. (2011b). Beyond IQ: a latent state-trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence 39, 323–334. doi: 10.1016/j.intell.2011.06.004

Dew, N., Read, S., Sarasvathy, S. D., and Wiltbank, R. (2009). Effectual versus predictive logics in entrepreneurial decision-making: differences between experts and novices. J. Bus. Ventur. 24, 287–309. doi: 10.1016/j.jbusvent.2008.02.002

Dhami, M. K., Mandel, D. R., Mellers, B. A., and Tetlock, P. E. (2015). Improving intelligence analysis with decision science. Perspect. Psychol. Sci. 10, 753–757. doi: 10.1177/1745691615598511

Dillon, J. T. (1982). Problem finding and solving. J. Creat. Behav. 16, 97–111. doi: 10.1002/j.2162-6057.1982.tb00326.x

Dörner, D. (1975). Wie Menschen eine Welt verbessern wollten [How people wanted to improve a world]. Bild Der Wissenschaft 12, 48–53.

Dörner, D. (1980). On the difficulties people have in dealing with complexity. Simulat. Gam. 11, 87–106. doi: 10.1177/104687818001100108

Dörner, D. (1996). The Logic of Failure: Recognizing and Avoiding Error in Complex Situations. New York, NY: Basic Books.

Dörner, D., Drewes, U., and Reither, F. (1975). “Über das Problemlösen in sehr komplexen Realitätsbereichen,” in Bericht über den 29. Kongreß der DGfPs in Salzburg 1974, Band 1 , ed. W. H. Tack (Göttingen: Hogrefe), 339–340.

Dörner, D., and Güss, C. D. (2011). A psychological analysis of Adolf Hitler’s decision making as commander in chief: summa confidentia et nimius metus. Rev. Gen. Psychol. 15, 37–49. doi: 10.1037/a0022375

Dörner, D., and Güss, C. D. (2013). PSI: a computational architecture of cognition, motivation, and emotion. Rev. Gen. Psychol. 17, 297–317. doi: 10.1037/a0032947

Dörner, D., Kreuzig, H. W., Reither, F., and Stäudel, T. (1983). Lohhausen. Vom Umgang mit Unbestimmtheit und Komplexität. Bern: Huber.

Ederer, P., Patt, A., and Greiff, S. (2016). Complex problem-solving skills and innovativeness – evidence from occupational testing and regional data. Eur. J. Educ. 51, 244–256. doi: 10.1111/ejed.12176

Edwards, W. (1962). Dynamic decision theory and probabiIistic information processing. Hum. Factors 4, 59–73. doi: 10.1177/001872086200400201

Engelhart, M., Funke, J., and Sager, S. (2017). A web-based feedback study on optimization-based training and analysis of human decision making. J. Dynamic Dec. Mak. 3, 1–23.

Ericsson, K. A., and Simon, H. A. (1983). Protocol Analysis: Verbal Reports As Data. Cambridge, MA: Bradford.

Fischer, A., Greiff, S., and Funke, J. (2017). “The history of complex problem solving,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 107–121.

Fischer, A., Holt, D. V., and Funke, J. (2015). Promoting the growing field of dynamic decision making. J. Dynamic Decis. Mak. 1, 1–3. doi: 10.11588/jddm.2015.1.23807

Fischer, A., Holt, D. V., and Funke, J. (2016). The first year of the “journal of dynamic decision making.” J. Dynamic Decis. Mak. 2, 1–2. doi: 10.11588/jddm.2016.1.28995

Fischer, A., and Neubert, J. C. (2015). The multiple faces of complex problems: a model of problem solving competency and its implications for training and assessment. J. Dynamic Decis. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.23945

Frensch, P. A., and Funke, J. (eds) (1995a). Complex Problem Solving: The European Perspective. Hillsdale, NJ: Erlbaum.

Frensch, P. A., and Funke, J. (1995b). “Definitions, traditions, and a general framework for understanding complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Lawrence Erlbaum), 3–25.

Frischkorn, G. T., Greiff, S., and Wüstenberg, S. (2014). The development of complex problem solving in adolescence: a latent growth curve analysis. J. Educ. Psychol. 106, 1004–1020. doi: 10.1037/a0037114

Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung subjektiver Kausalmodelle. Z. Psychol. 193, 435–457.

Funke, J. (1986). Komplexes Problemlösen - Bestandsaufnahme und Perspektiven [Complex Problem Solving: Survey and Perspectives]. Heidelberg: Springer.

Funke, J. (1993). “Microworlds based on linear equation systems: a new approach to complex problem solving and experimental results,” in The Cognitive Psychology of Knowledge , eds G. Strube and K.-F. Wender (Amsterdam: Elsevier Science Publishers), 313–330.

Funke, J. (1995). “Experimental research on complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 243–268.

Funke, J. (2010). Complex problem solving: a case for complex cognition? Cogn. Process. 11, 133–142. doi: 10.1007/s10339-009-0345-0

Funke, J. (2012). “Complex problem solving,” in Encyclopedia of the Sciences of Learning , Vol. 38, ed. N. M. Seel (Heidelberg: Springer), 682–685.

Funke, J. (2014a). Analysis of minimal complex systems and complex problem solving require different forms of causal cognition. Front. Psychol. 5:739. doi: 10.3389/fpsyg.2014.00739

Funke, J. (2014b). “Problem solving: what are the important questions?,” in Proceedings of the 36th Annual Conference of the Cognitive Science Society , eds P. Bello, M. Guarini, M. McShane, and B. Scassellati (Austin, TX: Cognitive Science Society), 493–498.

Funke, J., Fischer, A., and Holt, D. V. (2017). When less is less: solving multiple simple problems is not complex problem solving—A comment on Greiff et al. (2015). J. Intell. 5:5. doi: 10.3390/jintelligence5010005

Funke, J., Fischer, A., and Holt, D. V. (2018). “Competencies for complexity: problem solving in the 21st century,” in Assessment and Teaching of 21st Century Skills , eds E. Care, P. Griffin, and M. Wilson (Dordrecht: Springer), 3.

Funke, J., and Greiff, S. (2017). “Dynamic problem solving: multiple-item testing based on minimally complex systems,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 427–443.

Gobert, J. D., Kim, Y. J., Pedro, M. A. S., Kennedy, M., and Betts, C. G. (2015). Using educational data mining to assess students’ skills at designing and conducting experiments within a complex systems microworld. Think. Skills Creat. 18, 81–90. doi: 10.1016/j.tsc.2015.04.008

Goode, N., and Beckmann, J. F. (2010). You need to know: there is a causal relationship between structural knowledge and control performance in complex problem solving tasks. Intelligence 38, 345–352. doi: 10.1016/j.intell.2010.01.001

Gray, W. D. (2002). Simulated task environments: the role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cogn. Sci. Q. 2, 205–227.

Greiff, S., and Fischer, A. (2013). Measuring complex problem solving: an educational application of psychological theories. J. Educ. Res. 5, 38–58.

Greiff, S., Fischer, A., Stadler, M., and Wüstenberg, S. (2015a). Assessing complex problem-solving skills with multiple complex systems. Think. Reason. 21, 356–382. doi: 10.1080/13546783.2014.989263

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2015b). Sometimes less is more: comparing the validity of complex problem solving measures. Intelligence 50, 100–113. doi: 10.1016/j.intell.2015.02.007

Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., and Martin, R. (2013a). A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence 41, 579–596. doi: 10.1016/j.intell.2013.07.012

Greiff, S., Holt, D. V., and Funke, J. (2013b). Perspectives on problem solving in educational assessment: analytical, interactive, and collaborative problem solving. J. Problem Solv. 5, 71–91. doi: 10.7771/1932-6246.1153

Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., and Csapó, B. (2013c). Complex problem solving in educational contexts—something beyond g: concept, assessment, measurement invariance, and construct validity. J. Educ. Psychol. 105, 364–379. doi: 10.1037/a0031856

Greiff, S., and Funke, J. (2009). “Measuring complex problem solving: the MicroDYN approach,” in The Transition to Computer-Based Assessment. New Approaches to Skills Assessment and Implications for Large-Scale Testing , eds F. Scheuermann and J. Björnsson (Luxembourg: Office for Official Publications of the European Communities), 157–163.

Greiff, S., and Funke, J. (2017). “Interactive problem solving: exploring the potential of minimal complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 93–105.

Greiff, S., and Martin, R. (2014). What you see is what you (don’t) get: a comment on Funke’s (2014) opinion paper. Front. Psychol. 5:1120. doi: 10.3389/fpsyg.2014.01120

Greiff, S., and Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learn. Ind. Diff. 36, 37–48. doi: 10.1016/j.lindif.2014.08.003

Greiff, S., Niepel, C., Scherer, R., and Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: an analysis of behavioral data from computer-generated log files. Comput. Hum. Behav. 61, 36–46. doi: 10.1016/j.chb.2016.02.095

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2017). Sometimes more is too much: a rejoinder to the commentaries on Greiff et al. (2015). J. Intell. 5:6. doi: 10.3390/jintelligence5010006

Greiff, S., and Wüstenberg, S. (2014). Assessment with microworlds using MicroDYN: measurement invariance and latent mean comparisons. Eur. J. Psychol. Assess. 1, 1–11. doi: 10.1027/1015-5759/a000194

Greiff, S., and Wüstenberg, S. (2015). Komplexer Problemlösetest COMPRO [Complex Problem-Solving Test COMPRO]. Mödling: Schuhfried.

Greiff, S., Wüstenberg, S., and Funke, J. (2012). Dynamic problem solving: a new assessment perspective. Appl. Psychol. Measure. 36, 189–213. doi: 10.1177/0146621612439620

Griffin, P., and Care, E. (2015). “The ATC21S method,” in Assessment and Taching of 21st Century Skills , eds P. Griffin and E. Care (Dordrecht, NL: Springer), 3–33.

Güss, C. D., and Dörner, D. (2011). Cultural differences in dynamic decision-making strategies in a non-linear, time-delayed task. Cogn. Syst. Res. 12, 365–376. doi: 10.1016/j.cogsys.2010.12.003

Güss, C. D., Tuason, M. T., and Orduña, L. V. (2015). Strategies, tactics, and errors in dynamic decision making in an Asian sample. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.13131

Güss, C. D., and Wiley, B. (2007). Metacognition of problem-solving strategies in Brazil, India, and the United States. J. Cogn. Cult. 7, 1–25. doi: 10.1163/156853707X171793

Herde, C. N., Wüstenberg, S., and Greiff, S. (2016). Assessment of complex problem solving: what we know and what we don’t know. Appl. Meas. Educ. 29, 265–277. doi: 10.1080/08957347.2016.1209208

Hermes, M., and Stelling, D. (2016). Context matters, but how much? Latent state – trait analysis of cognitive ability assessments. Int. J. Sel. Assess. 24, 285–295. doi: 10.1111/ijsa.12147

Hotaling, J. M., Fakhari, P., and Busemeyer, J. R. (2015). “Dynamic decision making,” in International Encyclopedia of the Social & Behavioral Sciences , 2nd Edn, eds N. J. Smelser and P. B. Batles (New York, NY: Elsevier), 709–714.

Hundertmark, J., Holt, D. V., Fischer, A., Said, N., and Fischer, H. (2015). System structure and cognitive ability as predictors of performance in dynamic system control tasks. J. Dynamic Deci. Mak. 1, 1–10. doi: 10.11588/jddm.2015.1.26416

Jäkel, F., and Schreiber, C. (2013). Introspection in problem solving. J. Problem Solv. 6, 20–33. doi: 10.7771/1932-6246.1131

Jansson, A. (1994). Pathologies in dynamic decision making: consequences or precursors of failure? Sprache Kogn. 13, 160–173.

Kaufman, J. C., and Beghetto, R. A. (2009). Beyond big and little: the four c model of creativity. Rev. Gen. Psychol. 13, 1–12. doi: 10.1037/a0013688

Knauff, M., and Wolf, A. G. (2010). Complex cognition: the science of human reasoning, problem-solving, and decision-making. Cogn. Process. 11, 99–102. doi: 10.1007/s10339-010-0362-z

Kretzschmar, A. (2017). Sometimes less is not enough: a commentary on Greiff et al. (2015). J. Intell. 5:4. doi: 10.3390/jintelligence5010004

Kretzschmar, A., Neubert, J. C., Wüstenberg, S., and Greiff, S. (2016). Construct validity of complex problem solving: a comprehensive view on different facets of intelligence and school grades. Intelligence 54, 55–69. doi: 10.1016/j.intell.2015.11.004

Kretzschmar, A., and Süß, H.-M. (2015). A study on the training of complex problem solving competence. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.15455

Lee, H., and Cho, Y. (2007). Factors affecting problem finding depending on degree of structure of problem situation. J. Educ. Res. 101, 113–123. doi: 10.3200/JOER.101.2.113-125

Leutner, D., Fleischer, J., Wirth, J., Greiff, S., and Funke, J. (2012). Analytische und dynamische Problemlösekompetenz im Lichte internationaler Schulleistungsvergleichsstudien: Untersuchungen zur Dimensionalität. Psychol. Rundschau 63, 34–42. doi: 10.1026/0033-3042/a000108

Luchins, A. S. (1942). Mechanization in problem solving: the effect of einstellung. Psychol. Monogr. 54, 1–95. doi: 10.1037/h0093502

Mack, O., Khare, A., Krämer, A., and Burgartz, T. (eds) (2016). Managing in a VUCA world. Heidelberg: Springer.

Mainert, J., Kretzschmar, A., Neubert, J. C., and Greiff, S. (2015). Linking complex problem solving and general mental ability to career advancement: does a transversal skill reveal incremental predictive validity? Int. J. Lifelong Educ. 34, 393–411. doi: 10.1080/02601370.2015.1060024

Mainzer, K. (2009). Challenges of complexity in the 21st century. An interdisciplinary introduction. Eur. Rev. 17, 219–236. doi: 10.1017/S1062798709000714

Meadows, D. H., Meadows, D. L., and Randers, J. (1992). Beyond the Limits. Vermont, VA: Chelsea Green Publishing.

Meadows, D. H., Meadows, D. L., Randers, J., and Behrens, W. W. (1972). The Limits to Growth. New York, NY: Universe Books.

Meißner, A., Greiff, S., Frischkorn, G. T., and Steinmayr, R. (2016). Predicting complex problem solving and school grades with working memory and ability self-concept. Learn. Ind. Differ. 49, 323–331. doi: 10.1016/j.lindif.2016.04.006

Molnàr, G., Greiff, S., Wüstenberg, S., and Fischer, A. (2017). “Empirical study of computer-based assessment of domain-general complex problem-solving skills,” in The Nature of Problem Solving: Using research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 125–141.

National Research Council (2011). Assessing 21st Century Skills: Summary of a Workshop. Washington, DC: The National Academies Press.

Newell, A., Shaw, J. C., and Simon, H. A. (1959). A general problem-solving program for a computer. Comput. Automat. 8, 10–16.

Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychol. Rev. 84, 231–259. doi: 10.1037/0033-295X.84.3.231

OECD (2014). “PISA 2012 results,” in Creative Problem Solving: Students’ Skills in Tackling Real-Life problems , Vol. 5 (Paris: OECD Publishing).

Osman, M. (2010). Controlling uncertainty: a review of human behavior in complex dynamic environments. Psychol. Bull. 136, 65–86. doi: 10.1037/a0017815

Osman, M. (2012). The role of reward in dynamic decision making. Front. Neurosci. 6:35. doi: 10.3389/fnins.2012.00035

Qudrat-Ullah, H. (2015). Better Decision Making in Complex, Dynamic Tasks. Training with Human-Facilitated Interactive Learning Environments. Heidelberg: Springer.

Ramnarayan, S., Strohschneider, S., and Schaub, H. (1997). Trappings of expertise and the pursuit of failure. Simulat. Gam. 28, 28–43. doi: 10.1177/1046878197281004

Reuschenbach, B. (2008). Planen und Problemlösen im Komplexen Handlungsfeld Pflege. Berlin: Logos.

Rohe, M., Funke, J., Storch, M., and Weber, J. (2016). Can motto goals outperform learning and performance goals? Influence of goal setting on performance, intrinsic motivation, processing style, and affect in a complex problem solving task. J. Dynamic Deci. Mak. 2, 1–15. doi: 10.11588/jddm.2016.1.28510

Scherer, R., Greiff, S., and Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence 48, 37–50. doi: 10.1016/j.intell.2014.10.003

Schoppek, W., and Fischer, A. (2015). Complex problem solving – single ability or complex phenomenon? Front. Psychol. 6:1669. doi: 10.3389/fpsyg.2015.01669

Schraw, G., Dunkle, M., and Bendixen, L. D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Appl. Cogn. Psychol. 9, 523–538. doi: 10.1002/acp.2350090605

Schweizer, F., Wüstenberg, S., and Greiff, S. (2013). Validity of the MicroDYN approach: complex problem solving predicts school grades beyond working memory capacity. Learn. Ind. Differ. 24, 42–52. doi: 10.1016/j.lindif.2012.12.011

Schweizer, T. S., Schmalenberger, K. M., Eisenlohr-Moul, T. A., Mojzisch, A., Kaiser, S., and Funke, J. (2016). Cognitive and affective aspects of creative option generation in everyday life situations. Front. Psychol. 7:1132. doi: 10.3389/fpsyg.2016.01132

Selten, R., Pittnauer, S., and Hohnisch, M. (2012). Dealing with dynamic decision problems when knowledge of the environment is limited: an approach based on goal systems. J. Behav. Deci. Mak. 25, 443–457. doi: 10.1002/bdm.738

Simon, H. A. (1957). Administrative Behavior: A Study of Decision-Making Processes in Administrative Organizations , 2nd Edn. New York, NY: Macmillan.

Sonnleitner, P., Brunner, M., Keller, U., and Martin, R. (2014). Differential relations between facets of complex problem solving and students’ immigration background. J. Educ. Psychol. 106, 681–695. doi: 10.1037/a0035506

Spering, M., Wagener, D., and Funke, J. (2005). The role of emotions in complex problem solving. Cogn. Emot. 19, 1252–1261. doi: 10.1080/02699930500304886

Stadler, M., Becker, N., Gödker, M., Leutner, D., and Greiff, S. (2015). Complex problem solving and intelligence: a meta-analysis. Intelligence 53, 92–101. doi: 10.1016/j.intell.2015.09.005

Stadler, M., Niepel, C., and Greiff, S. (2016). Easily too difficult: estimating item difficulty in computer simulated microworlds. Comput. Hum. Behav. 65, 100–106. doi: 10.1016/j.chb.2016.08.025

Sternberg, R. J. (1995). “Expertise in complex problem solving: a comparison of alternative conceptions,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 295–321.

Sternberg, R. J., and Frensch, P. A. (1991). Complex Problem Solving: Principles and Mechanisms. (eds) R. J. Sternberg and P. A. Frensch. Hillsdale, NJ: Erlbaum.

Strohschneider, S., and Güss, C. D. (1998). Planning and problem solving: differences between brazilian and german students. J. Cross-Cult. Psychol. 29, 695–716. doi: 10.1177/0022022198296002

Strohschneider, S., and Güss, C. D. (1999). The fate of the Moros: a cross-cultural exploration of strategies in complex and dynamic decision making. Int. J. Psychol. 34, 235–252. doi: 10.1080/002075999399873

Thimbleby, H. (2007). Press On. Principles of Interaction. Cambridge, MA: MIT Press.

Tobinski, D. A., and Fritz, A. (2017). “EcoSphere: a new paradigm for problem solving in complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 211–222.

Tremblay, S., Gagnon, J.-F., Lafond, D., Hodgetts, H. M., Doiron, M., and Jeuniaux, P. P. J. M. H. (2017). A cognitive prosthesis for complex decision-making. Appl. Ergon. 58, 349–360. doi: 10.1016/j.apergo.2016.07.009

Tschirgi, J. E. (1980). Sensible reasoning: a hypothesis about hypotheses. Child Dev. 51, 1–10. doi: 10.2307/1129583

Tuchman, B. W. (1984). The March of Folly. From Troy to Vietnam. New York, NY: Ballantine Books.

Verweij, M., and Thompson, M. (eds) (2006). Clumsy Solutions for A Complex World. Governance, Politics and Plural Perceptions. New York, NY: Palgrave Macmillan. doi: 10.1057/9780230624887

Viehrig, K., Siegmund, A., Funke, J., Wüstenberg, S., and Greiff, S. (2017). “The heidelberg inventory of geographic system competency model,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 31–53.

von Clausewitz, C. (1832). Vom Kriege [On war]. Berlin: Dämmler.

Wendt, A. N. (2017). The empirical potential of live streaming beyond cognitive psychology. J. Dynamic Deci. Mak. 3, 1–9. doi: 10.11588/jddm.2017.1.33724

Wiliam, D., and Black, P. (1996). Meanings and consequences: a basis for distinguishing formative and summative functions of assessment? Br. Educ. Res. J. 22, 537–548. doi: 10.1080/0141192960220502

World Economic Forum (2015). New Vsion for Education Unlocking the Potential of Technology. Geneva: World Economic Forum.

World Economic Forum (2016). Global Risks 2016: Insight Report , 11th Edn. Geneva: World Economic Forum.

Wüstenberg, S., Greiff, S., and Funke, J. (2012). Complex problem solving — more than reasoning? Intelligence 40, 1–14. doi: 10.1016/j.intell.2011.11.003

Wüstenberg, S., Greiff, S., Vainikainen, M.-P., and Murphy, K. (2016). Individual differences in students’ complex problem solving skills: how they evolve and what they imply. J. Educ. Psychol. 108, 1028–1044. doi: 10.1037/edu0000101

Wüstenberg, S., Stadler, M., Hautamäki, J., and Greiff, S. (2014). The role of strategy knowledge for the application of strategies in complex problem solving tasks. Technol. Knowl. Learn. 19, 127–146. doi: 10.1007/s10758-014-9222-8

Keywords : complex problem solving, validity, assessment, definition, MicroDYN

Citation: Dörner D and Funke J (2017) Complex Problem Solving: What It Is and What It Is Not. Front. Psychol. 8:1153. doi: 10.3389/fpsyg.2017.01153

Received: 14 March 2017; Accepted: 23 June 2017; Published: 11 July 2017.

Reviewed by:

Copyright © 2017 Dörner and Funke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Joachim Funke, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

How To Solve Complex Problems

In today’s increasingly complex world, we are constantly faced with ill-defined problems that don’t have a clear solution. From poverty and climate change to crime and addiction, complex situations surround us. Unlike simple problems with a pre-defined or “right” answer, complex problems share several basic characteristics that make them hard to solve. While these problems can be frustrating and overwhelming, they also offer an opportunity for growth and creativity. Complex problem-solving skills are the key to addressing these tough issues.

In this article, I will discuss simple versus complex problems, define complex problem solving, and describe why it is so important in complex dynamic environments. I will also explain how to develop problem-solving skills and share some tips for effectively solving complex problems.

How is simple problem-solving different from complex problem-solving?

Solving problems is about getting from a currently undesirable state to an intended goal state. In other words, about bridging the gap between “what is” and “what ought to be”. However, the challenge of reaching a solution varies based on the kind of problem that is being solved. There are generally three different kinds of problems you should consider.

Simple problems have one problem solution. The goal is to find that answer as quickly and efficiently as possible. Puzzles are classic examples of simple problem solving. The objective is to find the one correct solution out of many possibilities.

Problems are different from puzzles in that they don’t have a known problem solution. As such, many people may agree that there is an issue to be solved, but they may not agree on the intended goal state or how to get there. In this type of problem, people spend a lot of time debating the best solution and the optimal way to achieve it.

Messes are collections of interrelated problems where many stakeholders may not even agree on what the issue is. Unlike problems where there is agreement about what the problem is, in messes, there isn’t agreement amongst stakeholders. In other words, even “what is” can’t be taken for granted. Most complex social problems are messes, made up of interrelated social issues with ill-defined boundaries and goals.

Problems and messes can be complicated or complex

Puzzles are simple, but problems and messes exist on a continuum between complicated and complex. Complicated problems are technical in nature. There may be many involved variables, but the relationships are linear. As a result, complicated problems have step-by-step, systematic solutions. Repairing an engine or building a rocket may be difficult because of the many parts involved, but it is a technical problem we call complicated.

On the other hand, solving a complex problem is entirely different. Unlike complicated problems that may have many variables with linear relationships, a complex problem is characterized by connectivity patterns that are harder to understand and predict.

Characteristics of complex problems and messes

So what else makes a problem complex? Here are seven additional characteristics (from Funke and Hester and Adams ).

• Lack of information. There is often a lack of data or information about the problem itself. In some cases, variables are unknown or cannot be measured.
• Many goals. A complex problem has a mix of conflicting objectives. In some sense, every stakeholder involved with the problem may have their own goals. However, with limited resources, not all goals can be simultaneously satisfied.
• Unpredictable feedback loops. In part due to many variables connected by a range of different relationships, a change in one variable is likely to have effects on other variables in the system. However, because we do not know all of the variables it will affect, small changes can have disproportionate system-wide effects. These unexpected events that have big, unpredictable effects are sometimes called Black Swans.
• Dynamic. A complex problem changes over time and there is a significant impact based on when you act. In other words, because the problem and its parts and relationships are constantly changing, an action taken today won’t have the same effects as the same action taken tomorrow.
• Time-delayed. It takes a while for cause and effect to be realized. Thus it is very hard to know if any given intervention is working.
• Unknown unknowns. Building off the previous point about a lack of information, in a complex problem you may not even know what you don’t know. In other words, there may be very important variables that you are not even aware of.
• Affected by (error-prone) humans. Simply put, human behavior tends to be illogical and unpredictable. When humans are involved in a problem, avoiding error may be impossible.

What is complex problem-solving?

“Complex problem solving” is the term for how to address a complex problem or messes that have the characteristics listed above.

Since a complex problem is a different phenomenon than a simple or complicated problem, solving them requires a different approach. Methods designed for simple problems, like systematic organization, deductive logic, and linear thinking don’t work well on their own for a complex problem.

And yet, despite its importance, there isn’t complete agreement about what exactly it is.

How is complex problem solving defined by experts?

Let’s look at what scientists, researchers, and system thinkers have come up with in terms of a definition for solving a complex problem. 

As a series of observations and informed decisions

For many employers, the focus is on making smart decisions. These must weigh the future effects to the company of any given solution. According to Indeed.com , it is defined as “a series of observations and informed decisions used to find and implement a solution to a problem. Beyond finding and implementing a solution, complex problem solving also involves considering future changes to circumstance, resources, and capabilities that may affect the trajectory of the process and success of the solution. Complex problem solving also involves considering the impact of the solution on the surrounding environment and individuals.”

As using information to review options and develop solutions

For others, it is more of a systematic way to consider a range of options. According to O*NET ,  the definition focuses on “identifying complex problems and reviewing related information to develop and evaluate options and implement solutions.”

As a self-regulated psychological process

Others emphasize the broad range of skills and emotions needed for change. In addition, they endorse an inspired kind of pragmatism. For example, Dietrich Dorner and Joachim Funke define it as “a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.”

As a novel way of thinking and reasoning

Finally, some emphasize the multidisciplinary nature of knowledge and processes needed to tackle a complex problem. Patrick Hester and Kevin MacG. Adams have stated that “no single discipline can solve truly complex problems. Problems of real interest, those vexing ones that keep you up at night, require a discipline-agnostic approach…Simply they require us to think systemically about our problem…a novel way of thinking and reasoning about complex problems that encourages increased understanding and deliberate intervention.”

A synthesis definition

By pulling the main themes of these definitions together, we can get a sense of what complex problem-solvers must do:

Gain a better understanding of the phenomena of a complex problem or mess. Use a discipline-agnostic approach in order to develop deliberate interventions. Take into consideration future impacts on the surrounding environment.

Why is complex problem solving important?

Many efforts aimed at complex social problems like reducing homelessness and improving public health – despite good intentions giving more effort than ever before – are destined to fail because their approach is based on simple problem-solving. And some efforts might even unwittingly be contributing to the problems they’re trying to solve. 

Einstein said that “We can’t solve problems by using the same kind of thinking we used when we created them.” I think he could have easily been alluding to the need for more complex problem solvers who think differently. So what skills are required to do this?

What are complex problem-solving skills?

The skills required to solve a complex problem aren’t from one domain, nor are they an easily-packaged bundle. Rather, I like to think of them as a balancing act between a series of seemingly opposite approaches but synthesized. This brings a sort of cognitive dissonance into the process, which is itself informative.

It brings F. Scott Fitzgerald’s maxim to mind: 

“The test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time and still retain the ability to function. One should, for example, be able to see that things are hopeless yet be determined to make them otherwise.” 

To see the problem situation clearly, for example, but also with a sense of optimism and possibility.

Here are the top three dialectics to keep in mind:

Thinking and reasoning

Reasoning is the ability to make logical deductions based on evidence and counterevidence. On the other hand, thinking is more about imagining an unknown reality based on thoughts about the whole picture and how the parts could fit together. By thinking clearly, one can have a sense of possibility that prepares the mind to deduce the right action in the unique moment at hand.

As Dorner and Funke explain: “Not every situation requires the same action,  and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.”

Analysis and reductionism combined with synthesis and holism

It’s important to be able to use scientific processes to break down a complex problem into its parts and analyze them. But at the same time, a complex problem is more than the sum of its parts. In most cases, the relationships between the parts are more important than the parts themselves. Therefore, decomposing problems with rigor isn’t enough. What’s needed, once problems are reduced and understood, is a way of understanding the relationships between various components as well as putting the pieces back together. However, synthesis and holism on their own without deductive analysis can often miss details and relationships that matter.  

What makes this balancing act more difficult is that certain professions tend to be trained in and prefer one domain over the other. Scientists prefer analysis and reductionism whereas most social scientists and practitioners default to synthesis and holism. Unfortunately, this divide of preferences results in people working in their silos at the expense of multi-disciplinary approaches that together can better “see” complexity.

Situational awareness and self-awareness 

Dual awareness is the ability to pay attention to two experiences simultaneously. In the case of complex problems, context really matters. In other words, problem-solving exists in an ecosystem of environmental factors that are not incidental. Personal and cultural preferences play a part as do current events unfolding over time. But as a problem solver, knowing the environment is only part of the equation. 

The other crucial part is the internal psychological process unique to every individual who also interacts with the problem and the environment. Problem solvers inevitably come into contact with others who may disagree with them, or be advancing seemingly counterproductive solutions, and these interactions result in emotions and motivations. Without self-awareness, we can become attached to our own subjective opinions, fall in love with “our” solutions, and generally be driven by the desire to be seen as problem solvers at the expense of actually solving the problem.

By balancing these three dialectics, practitioners can better deal with uncertainty as well as stay motivated despite setbacks. Self-regulation among these seemingly opposite approaches also reminds one to stay open-minded.

How do you develop complex problem-solving skills?

There is no one answer to this question, as the best way to develop them will vary depending on your strengths and weaknesses. However, there are a few general things that you can do to improve your ability to solve problems.

Ground yourself in theory and knowledge

First, it is important to learn about systems thinking and complexity theories. These frameworks will help you understand how complex systems work, and how different parts of a system interact with each other. This conceptual understanding will allow you to identify potential solutions to problems more quickly and effectively.

Practice switching between approaches

Second, practice switching between the dialectics mentioned above. For example, in your next meeting try to spend roughly half your time thinking and half your time reasoning. The important part is trying to get habituated to regularly switching lenses. It may seem disjointed at first, but after a while, it becomes second nature to simultaneously see how the parts interact and the big picture.

Focus on the specific problem phenomena

Third, it may sound obvious, but people often don’t spend very much time studying the problem itself and how it functions. In some sense, becoming a good problem-solver involves becoming a problem scientist. Your time should be spent regularly investigating the phenomena of “what is” rather than “what ought to be”. A holistic understanding of the problem is the required prerequisite to coming up with good solutions.

Stay curious

Finally, after we have worked on a problem for a while, we tend to think we know everything about it, including how to solve it. Even if we’re working on a problem, which may change dynamically from day to day, we start treating it more like a puzzle with a definite solution. When that happens, we can lose our motivation to continue learning about the problem. This is very risky because it closes the door to learning from others, regardless of whether we completely agree with them or not.

As Neils Bohr said, “Two different perspectives or models about a system will reveal truths regarding the system that are neither entirely independent nor entirely compatible.”

By staying curious, we can retain our ability to learn on a daily basis.

Tips for how to solve complex problems

Focus on processes over results.

It’s easy to get lost in utopian thinking. Many people spend so much time on “what ought to be” that they forget that problem solving is about the gap between “what is” and “what ought to be”. It is said that “life is a journey, not a destination.” The same is true for complex problem-solving. To do it well, a problem solver must focus on enjoying the process of gaining a holistic understanding of the problem. 

Adaptive and iterative methods and tools

A variety of adaptive and iterative methods have been developed to address complexity. They share a laser focus on gaining holistic understanding with tools that best match the phenomena of complexity. They are also non-ideological, trans-disciplinary, and flexible. In most cases, your journey through a set of steps won’t be linear. Rather, as you think and reason, analyze and synthesize, you’ll jump around to get a holistic picture.

In my online course , we generally follow a seven-step method:

• Get clear sight with a complex problem-solving frame
• Establish a secure base of operation
• Gain a deep understanding of the problem
• Create an interactive model of the problem
• Develop an impact strategy
• Create an action plan and implement
• Embed systemic solutions

Of course, each of these steps involves testing to see what works and consistently evaluating our process and progress.

Resolution is about systematically managing a problem over time

One last thing to keep in mind. Most social problems are not just solved one day, never to return. In reality,  most complex problems are managed, not solved. For all practical purposes, what this means is that “the solution” is a way of systematically dealing with the problem over time. Some find this disappointing, but it’s actually a pragmatic pointer to think about resolution – a way move problems in the right direction – rather than final solutions.

Problem solvers regularly train and practice

If you need help developing your complex problem-solving skills, I have an online class where you can learn everything you need to know. 

Sign up today and learn how to be successful at making a difference in the world!

SoftwareDominos

Complex Problems: What Does the Nature of the Problem Tell Us About Its Solution

1. Overview

In The 7 Timeless Steps to Guide You Through Complex Problem Solving , we discussed a generic approach that could be systematically applied to solving complex problems. Since not all problems are complex, and many gradations of complexity exist, it is probably a good idea to start by defining what complex problem-solving involves and what categories of problems are most suitable to tackle using that approach. For this reason, understanding complex problems made the top position on the list.

Practically all living organisms deal with complex problems, from single-celled amebas to societies of Homo sapiens , and surprisingly, the solution-creation process can be very similar, at least on the conceptual level. This article will elaborate on this point further, articulating the terminology and ideas often associated with how complex adaptive systems solve complex problems. More specifically, we will answer the following questions:

This article is part of a series on complex problem-solving. The list below will guide you through the different subtopics.

Complex Problem-Solving Guide in 7 Steps

The 7 Timeless Steps to Guide You Through Complex Problem Solving

The Nature of Complex Problems

What Does the Nature of the Problem Tell Us About Its Solution

Gaussian Distributions vs Power Laws

Your Ultimate Guide to Making Sense of Natural and Social Phenomena

Complex Problem-Solving in Groups

An Exploratory Overview of ProbleSolving Processes in Groups

The Power of Critical Thinking

An Essential Guide for Personal and Professional Development

Group-Decision Making

6 Modes That Tell Us How Teams Decide

3. An Intuitive Definition of Complex Problems

We all intuitively grasp the characteristics of challenging problems, at least at their fundamental levels. For instance, we can promptly recognize that fixing a faulty washing machine is relatively simple. First, we need basic technical skills to identify the faulty part. Next, we would read the code on the back, order a spare part, and finally replace it.

In simple problems, there is no uncertainty around the root cause or the solution.

On the other hand, deciding whether or not to accept a job offer is anything but simple. Firstly, you will never have sufficient information to make an optimal decision . Secondly, you cannot predict the consequences of such a decision. Finally, whichever choice you make will change your worldview, rendering any forecasts you have made of the future almost instantly obsolete.

The following characteristics distinguish complex problems.

So, what are complex problems?

Complex problems — an intuitive guide.

Non-triviality

Complex problems generally admit non-trivial solutions. In addition to strong field expertise and solid analytical skills, they require a high cognitive load to formulate.

Uncertainty

Solutions to complex problems cannot be guaranteed as the behaviour of the system to which the solution is applied is always unpredictable.

Diagnosing complex problems is especially challenging because consensus on facts, root causes, and solutions can be difficult to obtain, especially in large groups.

3. Challenges of Working With Complex Problems

Experts like Nassim Taleb, Gerd Gigerenzer, and Daniel Kahnemann insist that solving complex problems is relatively easy once we understand which tools to apply. In their view, failures come from applying engineering methods like optimization rather than intuition , heuristics, biases, imitation, and many other techniques refined over millennia of evolution and accumulated wisdom.

4. Complex Problems in the Literature

Experts have extensively researched topics associated with intuition, cognitive psychology , risk management , organisational behaviour , and decision-making under uncertainty. This has left us with a rich body of knowledge popularized by best-selling authors such as Daniel Kahneman and Nassim Taleb, which will be reviewed next.

4.1 Fooled by Randomness (Taleb, 2001)

Fooled by Randomness is one of Taleb’s best-selling books , and its central story revolves around the hidden role of chance in our lives. In Taleb’s view, we grossly and routinely overestimate our capabilities to forecast future events ( the turkey problem ) and cope with that failure through mechanisms like the narrative fallacy and our ability to reconstruct past events based on new information.

Key takeaways from Fooled By Randomness

• In social , financial, economic, and political systems, Gaussian distributions mislead at best by providing a comforting but shifting ground for modelling events.
• Power laws like Pareto’s provide more suitable models for examining complex systems .
• Time-tested heuristics, formulated through a long knowledge acquisition and refinement period, are more valuable for decision-making under uncertainty than optimisation techniques, which require a well-behaved underlying model (such as the Gaussian).

4.2 Thinking, Fast and Slow (Kahneman, 2011)

Thinking, Fast and Slow is a best-selling book by Daniel Kahneman popularizing his work in cognitive psychology about the mechanism and efficiency of human judgment and decision-making under conditions of uncertainty. His original idea revolves around modelling the human mind as two systems, which he refers to as System 1 and System 2.

Key takeaways from Thinking, Fast and Slow

• Systems 1 and 2 perfectly cover our decision-making needs for simple and complex problems.
• System 1 is fast and inexpensive, allowing us to make critical decisions with imperfect or unreliable data .
• System 1 relies on heuristics and biases to compensate for unreliable information and processing time.
• System 2 is slow and expensive but more accurate, allowing us to make decisions requiring a high cognitive load and processing larger amounts of information.

4.3 Process Consultation (Schein, 1969)

Professor Edgar Schein is a leading authority in organizational behaviour, culture , and psychology. His short but insightful book Process Consultation: Its Role in Organizational Development dedicates a full chapter to group problem-solving and decision-making. Schein explores how leaders and their groups tackle complex problems in this chapter.

Key takeaways from Process Consultation

• Group problem-solving presents challenges and dynamics that differ from those of individuals and is a subject in its own right.
• In both cases, events that cause tension and anxiety trigger a solution-finding process that culminates in applying changes to the environment or the individual or group’s interactions with it.
• However, problem formulation, solution creation and implementation differ significantly between the two cases.

5. The Information Sufficiency Problem

5.1 how much data is enough.

During the Newtonian age, physicists believed that once the initial conditions of a physical system were precisely determined, its future evolution could be predicted with arbitrary precision. For example, the laws of dynamics allow us to calculate the infinite trajectory of a point mass given its initial position and velocity.

What happens when the system consists of innumerable particles, each with a different initial speed and position? For practical reasons, we substitute the individual particles with a unit of volume where its macro properties can be calculated by averaging over its constituent particles. For example, instead of registering the speed and position of every molecule in a gas container, we substitute those numbers with temperature and pressure calculated on a coarse-grained subvolume. This coarse-graining allows us to explore the system’s physical properties without drowning in data.

5.2 The Rise of Statistical Mechanics and Probabilistic Models

The coarse-graining method and the impracticality of precise calculations on the molecular level gave rise to statistical mechanics , which Boltzmann and others pioneered. Under statistical mechanics, physical systems are governed by the laws of thermodynamics. The second law of thermodynamics is the most famous, dictating that a system’s entropy (or disorder) must always increase.

The practical advantages of using coarse-graining came at a cost, as a probabilistic model replaced the classic view of deterministic evolution. In this new paradigm, a physical system is predisposed to evolve into one of numerous states. We can only predict the probability that it will be in a given future state, but we can never be sure which one.

But all is not lost. Even with the probabilistic model, we can still calculate a system’s future state and create contingency plans for each scenario. We might even be able to influence the outcome by applying pressure on known system levers. This assumption forms the basis of Strategic Choice Theory .

Strategic choice theory, in the realm of organizational theory, emphasizes the influence of leaders and decision-makers on an organization’s direction. It contrasts with earlier views that saw organizations solely responding to external forces.

Managing Probabilistic Systems

In probabilistic models, we assume that the system’s future states are well-defined and their probabilities are calculable. Given this information, adequate planning and optimization processes can be applied to maximize a specific utility function.

5.3 Probabilistic Models Cannot Account for Innovation

Any physical, chemical, or biological system that shows innovation cannot, by definition, be analyzed using probability models, as the latter assumes all future states are static and knowable in advance. Also, the probabilities for reaching any of those states are either fixed or vary according to well-specified rules.

Therefore, probabilistic models are not good enough to predict the future behaviour of human systems. This also spells trouble for Strategic Choice Theory, which relies on simple causal relationships between leaders’ interventions and desired consequences to achieve progress or resolve conflicts.

If a system can produce novel behaviour, it is unpredictable and, therefore, hard to manage. Ecologies of living organisms can only be understood through complexity theory and managed by principles that consider that.

Complex systems presenting complex problems will never offer sufficient information, and managers must make choices under uncertain conditions.

Even if we consider every atom (or elementary particle) in the universe , we still would not be able to predict the rich diversity of phenomena (including biodiversity on Earth) that we currently observe. Quantum mechanics and symmetry breaking ensure enough randomness is injected into the system to produce rich but unpredictable results.

The same applies when we try to understand the source of consciousness in our brains. Would it help to incorporate every neuron and synapse in a gigantic mathematical model? Even if this becomes practical someday, experts seem to believe that emerging consciousness in the inanimate matter is far away.

In summary, there seems to be a hard limit on how much useful information, in principle and practice, can be gleaned by observing a complex system .

6. Problem Classification

6.1 maximizing utility functions.

Problems can present themselves in many different ways. However, we are interested in those characterized by a utility function.

A utility function is a concept primarily used in economics, decision theory, and game theory to represent an individual’s preferences over different outcomes or states of the world. It assigns a numerical (or utility) value to each possible outcome or combination, reflecting the individual’s subjective satisfaction or preference associated with those outcomes.

How Are Utility Functions Used?

Here are some key points about utility functions:

Using utility functions, people can compare complex options involving chance or risk and make decisions based on their preferences and risk tolerance.

6.2 Ordered, Chaotic, Complex, and Random Systems

Imagine that you have the following problem. You are required to configure an air conditioning system for a data centre. The system is composed of two machines: a cooling engine and a computer connected to it. The computer has temperature and humidity sensors and various switches and dials that allow operators to set control parameters such as maximum temperature or humidity.

The engineer setting up the system must configure it to minimize power consumption while keeping the room at a given temperature and humidity level. The only issue is that the system does not have an operations guide, and the engineer has to figure out how to set it up using trial and error.

Four scenarios are possible: Ordered, Random, Complex, and Chaotic.

Ordered Systems

• Changes in the switches or dials produce a clear response in the cooling machine.
• Although some settings may impact others, the engineer can, through trial and error, understand the relationship between the controls and the outcomes.
• Ordered systems have direct causal relationships and hard constraints between their components.
• Problems in an ordered system can be resolved through the relationships between control parameters and the utility function.
• Computers, watches, and washing machines are examples of such systems.

Random Systems

• Changes in the switches or dials produce different responses every time. There seems to be no correlation between the settings and the outcomes.
• Random systems have no causal links and no constraints between their components.
• Random systems present problems that cannot be resolved; they are, by definition, unmanageable.
• A reward system based on rolling two dice is a random system.

Chaotic Systems

• Small changes in the switches or dials produce wild responses. Although the system appears random and unpredictable, it shows regular behavioural patterns over the long term.
• Chaotic systems have causal links and hard constraints between their components in addition to non-linear dynamics.
• Chaotic systems are also challenging to manage. However, causes can be linked to effects, and regularities can be leveraged.
• The weather, a turbulent water flow , and three bodies rotating around each other in gravitational fields are examples of chaotic systems.

Complex Systems

• Changes in the switches or dials produce different responses every time. Slight correlations can be measured between the settings and the outcomes.
• Complex systems have indirect causal links and loose constraints between their components.
• Complex systems come in two varieties: adaptive and non-adaptive.
• An example of a non-adaptive complex system is the Brusselator . An example of a complex adaptive system is a microbiome.
• Complex systems present problems that can be resolved through heuristics, safe-to-fail experimentation, and managing in the present rather than towards a desirable future state.

7. Small Worlds, Optimization, and Unknown Unknowns

7.1 leonard j. savage’s “small world”.

Leonard Jimmie Savage (1917-1971) was an American statistician and economist who significantly contributed to statistics , decision theory, and econometrics.

“Savage’s Small World” refers to a thought experiment proposed by the statistician and economist Leonard Jimmie Savage. This concept is often cited in discussions about subjective probability and decision theory.

Decision-Making in a Simple World

L. J. Savage’s “Small World”

• In Savage’s Small World, imagine a small society where everyone knows each other’s preferences, capabilities, and the outcomes of their decisions.
• Within this world, individuals can communicate freely and exchange information about their beliefs, desires, and experiences.
• In such a setting, decision-making becomes more transparent and informed, as individuals can access comprehensive knowledge about each other’s perspectives and choices.

The significance of Savage’s Small World lies in its implications for decision theory. It illustrates an idealized scenario where uncertainty is minimized, and individuals have perfect knowledge about the consequences of their actions. In reality, however, decision-makers often face uncertainty and incomplete information, prompting probabilistic reasoning and subjective judgment.

By contrasting Savage’s Small World with the complexities of real-world decision-making, Savage highlighted the importance of subjective probability for navigating uncertainty and making rational choices. Subjective probability allows individuals to express their beliefs and uncertainty in a formal framework, facilitating reasoned decision-making even when complete information is lacking.

7.2 Optimization Techniques

Optimization techniques can be effectively applied in a Small World scenario where all outcomes and probabilities can be precisely computed beforehand. This is because decision-makers have complete knowledge of the system, allowing them to accurately assess the consequences of their actions and choose the optimal course of action based on predetermined criteria.

In contrast, in the real world, uncertainty, complexity, and incomplete information often make it challenging to compute outcomes and probabilities beforehand precisely. As a result, optimization techniques may not be as effective, as they rely on accurate information to generate optimal solutions. Decision-makers must contend with uncertainty and imperfect knowledge, which can lead to suboptimal outcomes even when applying optimisation techniques.

One example where optimization relies on known outcomes and their probabilities is in the context of inventory management.

In inventory management, a retailer determines the optimal inventory level for each product to minimize costs while ensuring that customer demand is met. In this case, the utility function represents the retailer’s objective, which typically involves minimizing inventory holding costs and stockouts.

Optimisation Process in Inventory Management

Here’s a rigorous breakdown of the optimization process:

• 1 – Identify outcomes and probabilities :
• 2- Define the utility function :
• 3- Formulate the optimization problem :
• 4- Solve the optimization problem :

By incorporating known outcomes (demand scenarios) and their probabilities into the utility function and using optimization techniques, retailers can manage their inventory effectively, minimizing costs while ensuring customer satisfaction and maintaining adequate product availability.

7.3 Optimisation in Complex Worlds

Optimization techniques may encounter challenges in complex situations, particularly those governed by power laws (see discussion on Gaussian Distributions vs Power Laws: Your Ultimate Guide to Making Sense of Natural and Social Phenomena and their impact on our understanding of complex natural phenomena), due to several reasons:

Practical challenges of estimating model parameters in power laws versus Gaussians

Comparing the practical difficulties of estimating model parameters such as mean and variance in power laws versus Gaussians:

• Mean estimation
• Variance estimation

7.4 Unknown Unknowns

The concept of “ unknown unknowns” refers to phenomena or factors that are not only unknown but also unknowable.

In Savage’s “Small World,” which represents an idealized scenario where decision-makers have perfect knowledge of outcomes and their probabilities, the concept of “unknown unknowns” highlights the limitations of this idealization. In a Small World, nothing new ever happens, and there can be no “Unknown Unknowns”.

In contrast, complex adaptive systems constantly display emergent behaviour; patterns that could not have been anticipated. A leader managing such a system cannot list all possible outcomes, let alone assign each probability.

8. Subjectivity and the Role of the Observer

In decision-making, a leader’s subjective experience contrasts with their role as an objective observer. For example, in systems thinking and cybernetics, the leader must diagnose problems based on data and evidence and formulate a logical and rational solution.

A leader working on complex problems in a social group is an integral part of the system. As we have seen in previous sections, the leader is unable to gather sufficient information about the system in principle and practice, and whatever data they gather will be coloured by their subjective experience

Systems Thinking in a Nuntshell

Systems Thinking is a holistic approach to understanding complex systems by examining their interconnectedness, interdependencies, and dynamics. It views the leader’s role in an organization as crucial for effective strategy formulation and decision-making by emphasizing the following principles:

• Holistic Perspective :
• Interconnectedness :
• Feedback Loops :

Systems Thinking addresses the paradox of the leader being part of the system being managed by acknowledging the leader’s dual role as both a participant within the system and an external observer guiding its direction. Several key principles help resolve this conflict:

Exploring problem-solving reveals that not all problems are created equal. Distinguishing between simple and complex problems reveals the underlying nature of the systems they belong to. Simple problems are typically found within ordered systems, whereas complex problems are inherent to complex systems. These systems extend beyond biological ecologies to encompass social groups and organizations, where intricate interactions and emergent behaviours define their complexity.

One defining characteristic of complex systems is their governance by power laws, rendering traditional optimization techniques ineffective. Unlike in ordered systems, where linear solutions may suffice, complex systems defy such neat categorizations. Applying optimization strategies proves futile due to the non-linear, unpredictable dynamics governed by power laws.

Heuristics emerge as promising alternatives to optimization in navigating the labyrinth of complex systems. These intuitive, rule-of-thumb approaches allow for adaptive decision-making, acknowledging complex systems’ inherent uncertainty and non-linearity.

10. References

• Thinking, Fast and Slow — by Daniel Kahneman , 2011
• Fooled by Randomness — by Nassim Nicholas Taleb , 2001
• Process Consultation — by Edgar Schein , 1969
• The Quark and the Jaguar — by Murray Gell-Mann , 1994

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Twitter icon
• Facebook icon
• LinkedIn icon

What are wicked problems? How to tackle complex problems and projects

4. Revisit your principles and values to help guide your approach

Once you’ve defined the underlying issues of the wicked problem, it’s time to begin planning how you’re going to fix it. Before you do, take a moment to revisit your values, as they’ll help guide your approach.

• Align values and outcomes. As a team, take the time to reassess the values of your company and your project team to guide how you’ll make the change. For example, a customer-focused organization may choose a solution that limits any impact on customer operations. Whereas, a commercially focused organization may prioritize the cheapest solutions.

John holds a workshop for the team to review MotorSure’s values and set principles for how the team will move forward. MotorSure are currently focused on establishing their brand in the market, so the team decides to focus on a solution that helps them solve the problem, while also driving positive PR and enhancing their reputation.

5. Build your tolerance for uncertainty and risk

Once you know the guiding principles of your solution, you need to draw out your red lines. This will help you set boundaries you cannot cross and give team members an element of control and certainty around where the solution can and can’t go.

• Set your risk appetite. This article from the Risk Leadership Network provides great tips to help you set your own risk threshold, including how to make effective and considered decisions while still driving fast performance.
• Get a plan in place. From there, you can begin to build out your own risk management plan . If you’re struggling, you can also tap into our guide on the 11 most common project risks and even get a free risk management template.

As the size and the scale of the wicked problem come to life, John is worried that a future solution may have the ability to make the culture worse. To help, he works with his sponsors to determine a risk appetite, establishing the minimum and maximum impacts they’re happy to have on the business. He uses this to identify risks for the project, including a further drop in employee morale and potential impacts on company finances.

6. Solutionize and test with smaller “pilot projects”

Now that you’ve laid all the groundwork, it’s time to start developing and testing potential solutions. This is where you and the team will need to think outside the box to come up with a revolutionary idea to a wicked problem. Then, you’ll test the idea on a small scale to see if it has the desired outcomes.

• Validate ideas. Especially if you’re working in a product-led environment, once you’ve come up with an idea, you need to validate it. The Planio guide to product idea validation can help here, laying out a step-by-step approach to creating, validating and testing left field ideas.
• Build your confidence. If you’re struggling to get those creative juices flowing, it may be because you’re struggling with limiting beliefs . As a team, work through building your confidence to help you overcome those mental blocks.

John’s project team comes up with three ideas to improve culture: focus-time Fridays, investing in more regular socials, and creating an employee voice forum.

They validate these ideas with Sparks, who provide some additional tweaks to add extra value. John and the team implement them within the Operations department and monitor feedback from employees.

7. Implement, review, and go again

Once you’ve tested a solution to a wicked problem on a small sample, it’s time to push on and launch more broadly. This is especially true for wicked problems with a one-shot solution, where you simply have to bite the bullet and trust that the solution will work!

• Go for it. Launches of any type are nerve wracking moments, but none more so than new product launches. Check out the Planio guide to product launches to help you get inspired by different ways to launch a new solution out to your target audience.
• Adapt and iterate. Like all things Agile project management , the solution may not be 100% on the first try. Keep in touch with stakeholders to gather feedback and look for ways to continually improve for your next implementation.

How to overcome your unconscious biases (12 examples)

How to create better action items (with free templates)

How to protect your energy (and remove chaos from your work)

Working with planio, see how our customers use planio.

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

To Solve a Tough Problem, Reframe It

• Julia Binder
• Michael D. Watkins

Research shows that companies devote too little effort to examining problems before trying to solve them. By jumping immediately into problem-solving, teams limit their ability to design innovative solutions.

The authors recommend that companies spend more time up front on problem-framing, a process for understanding and defining a problem. Exploring different frames is like looking at a scene through various camera lenses while adjusting your angle, aperture, and focus. A wide-angle lens gives you a very different photo from that taken with a telephoto lens, and shifting your angle and depth of focus yields distinct images. Effective problem-framing is similar: Looking at a problem from a variety of perspectives helps you uncover new insights and generate fresh ideas.

This article introduces a five-phase approach to problem-framing: In the expand phase, the team identifies all aspects of a problem; in examine, it dives into root causes; in empathize, it considers key stakeholders’ perspectives; in elevate, it puts the problem into a broader context; and in envision, it creates a road map toward the desired outcome.

Five steps to ensure that you don’t jump to solutions

Idea in Brief

The problem.

Research shows that most companies devote too little effort to examining problems from all angles before trying to solve them. That limits their ability to come up with innovative ways to address them.

The Solution

Companies need a structured approach for understanding and defining complex problems to uncover new insights and generate fresh ideas.

The Approach

This article introduces a five-phase approach to problem-framing: In the expand phase, the team identifies all aspects of a problem; in examine, it dives into root causes; in empathize, it considers key stakeholders’ perspectives; in elevate, it puts the problem into a broader context; and in envision, it creates a road map toward the desired outcome.

When business leaders confront complex problems, there’s a powerful impulse to dive right into “solving” mode: You gather a team and then identify potential solutions. That’s fine for challenges you’ve faced before or when proven methods yield good results. But what happens when a new type of problem arises or aspects of a familiar one shift substantially? Or if you’re not exactly sure what the problem is?

Research conducted by us and others shows that leaders and their teams devote too little effort to examining and defining problems before trying to solve them. A study by Paul Nutt of Ohio State University, for example, looked at 350 decision-making processes at medium to large companies and found that more than half failed to achieve desired results, often because perceived time pressure caused people to pay insufficient attention to examining problems from all angles and exploring their complexities. By jumping immediately into problem-solving, teams limit their ability to design innovative and durable solutions.

When we work with organizations and teams, we encourage them to spend more time up front on problem-framing, a process for understanding and defining a problem. Exploring frames is like looking at a scene through various camera lenses while adjusting your angle, aperture, and focus. A wide-angle lens will give you a very different photo from that taken with a telephoto lens, and shifting your angle and depth of focus yields distinct images. Effective problem-framing is similar: Looking at a problem from a variety of perspectives lets you uncover new insights and generate fresh ideas.

As with all essential processes, it helps to have a methodology and a road map. This article introduces the E5 approach to problem-framing—expand, examine, empathize, elevate, and envision—and offers tools that enable leaders to fully explore the problem space.

Phase 1: Expand

In the first phase, set aside preconceptions and open your mind. We recommend using a tool called frame-storming, which encourages a comprehensive exploration of an issue and its nuances. It is a neglected precursor to brainstorming, which typically focuses on generating many different answers for an already framed challenge. Frame-storming helps teams identify assumptions and blind spots, mitigating the risk of pursuing inadequate or biased solutions. The goal is to spark innovation and creativity as people dig into—or as Tina Seelig from Stanford puts it, “fall in love with”—the problem.

Begin by assembling a diverse team, encompassing a variety of types of expertise and perspectives. Involving outsiders can be helpful, since they’re often coming to the issue cold. A good way to prompt the team to consider alternative scenarios is by asking “What if…?” and “How might we…?” questions. For example, ask your team, “What if we had access to unlimited resources to tackle this issue?” or “How might better collaboration between departments or teams help us tackle this issue?” The primary objective is to generate many alternative problem frames, allowing for a more holistic understanding of the issue. Within an open, nonjudgmental atmosphere, you deliberately challenge established thinking—what we call “breaking” the frame.

It may be easy to eliminate some possibilities, and that’s exactly what you should do. Rather than make assumptions, generate alternative hypotheses and then test them.

Consider the problem-framing process at a company we’ll call Omega Soundscapes, a midsize producer of high-end headphones. (Omega is a composite of several firms we’ve worked with.) Omega’s sales had declined substantially over the past two quarters, and the leadership team’s initial diagnosis, or reference frame, was that recent price hikes to its flagship product made it too expensive for its target market. Before acting on this assumption, the team convened knowledgeable representatives from sales, marketing, R&D, customer service, and external consultants to do some frame-storming. Team members were asked:

• What if we lowered the price of our flagship product? How would that impact sales and profitability?
• How might we identify customers in new target markets who could afford our headphones at the current price?
• What if we offered financing or a subscription-based model for our headphones? How would that change perceptions of affordability?
• How might we optimize our supply chain and production processes to reduce manufacturing costs without compromising quality?

In playing out each of those scenarios, the Omega team generated several problem frames:

• The target market’s preferences have evolved.
• New competitors have entered the market.
• Product quality has decreased.
• Something has damaged perceptions of the brand.
• Something has changed in the priorities of our key distributors.

Each of the frames presented a unique angle from which to approach the problem of declining sales, setting the stage for the development of diverse potential solutions. At this stage, it may be relatively easy to eliminate some possibilities, and that’s exactly what you should do. Rather than make assumptions, generate alternative hypotheses and then test them.

See more HBR charts in Data & Visuals

Phase 2: Examine

If the expand phase is about identifying all the facets of a problem, this one is about diving deep to identify root causes. The team investigates the issue thoroughly, peeling back the layers to understand underlying drivers and systemic contributors.

A useful tool for doing this is the iceberg model, which guides the team through layers of causation: surface-level events, the behavioral patterns that drive them, underlying systematic structures, and established mental models. As you probe ever deeper and document your findings, you begin to home in on the problem’s root causes. As is the case in the expand phase, open discussions and collaborative research are crucial for achieving a comprehensive analysis.

Let’s return to our Omega Soundscapes example and use the iceberg model to delve into the issues surrounding the two quarters of declining sales. Starting with the first layer beneath the surface, the behavioral pattern, the team diligently analyzed customer feedback. It discovered a significant drop in brand loyalty. This finding validated the problem frame of a “shifting brand perception,” prompting further investigation into what might have been causing it.

Phase 3: Empathize

In this phase, the focus is on the stakeholders—employees, customers, clients, investors, supply chain partners, and other parties—who are most central to and affected by the problem under investigation. The core objective is to understand how they perceive the issue: what they think and feel, how they’re acting, and what they want.

First list all the people who are directly or indirectly relevant to the problem. It may be helpful to create a visual representation of the network of relationships in the ecosystem. Prioritize the stakeholders according to their level of influence on and interest in the problem, and focus on understanding the roles, demographics, behavior patterns, motivations, and goals of the most important ones.

Now create empathy maps for those critical stakeholders. Make a template divided into four sections: Say, Think, Feel, and Do. Conduct interviews or surveys to gather authentic data. How do various users explain the problem? How do they think about the issue, and how do their beliefs inform that thinking? What emotions are they feeling and expressing? How are they behaving? Populate each section of the map with notes based on your observations and interactions. Finally, analyze the completed empathy maps. Look for pain points, inconsistencies, and patterns in stakeholder perspectives.

Returning to the Omega case study, the team identified its ecosystem of stakeholders: customers (both current and potential); retail partners and distributors; the R&D, marketing, and sales teams; suppliers of headphone components; investors and shareholders; and new and existing competitors. They narrowed the list to a few key stakeholders related to the declining-sales problem: customers, retail partners, and investors/shareholders; Omega created empathy maps for representatives from each.

Here’s what the empathy maps showed about what the stakeholders were saying, thinking, feeling, and doing:

Sarah, the customer, complained on social media about the high price of her favorite headphones. Dave, the retailer, expressed concerns about unsold inventory and the challenge of convincing customers to buy the expensive headphones. Alex, the shareholder, brought up Omega’s declining financial performance during its annual investor day.

Sarah thought that Omega was losing touch with its loyal customer base. Dave was considering whether to continue carrying Omega’s products in his store or explore other brands. Alex was contemplating diversifying his portfolio into other consumer-tech companies.

As a longtime supporter of the brand, Sarah felt frustrated and slightly betrayed. Dave was feeling anxious about the drop in sales and the impact on his store’s profitability. Alex was unhappy with the declining stock value.

Sarah was looking for alternatives to the headphones, even though she loves the product’s quality. Dave was scheduling a call with Omega to negotiate pricing and terms. Alex was planning to attend Omega’s next shareholder meeting to find out more information from the leadership team.

When Omega leaders analyzed the data in the maps, they realized that pricing wasn’t the only reason for declining sales. A more profound issue was customers’ dissatisfaction with the perceived price-to-quality ratio, especially when compared with competitors’ offerings. That insight prompted the team to consider enhancing the headphones with additional features, offering more-affordable alternatives, and possibly switching to a service model.

Phase 4: Elevate

This phase involves exploring how the problem connects to broader organizational issues. It’s like zooming out on a map to understand where a city lies in relation to the whole country or continent. This bird’s-eye view reveals interconnected issues and their implications.

For this analysis, we recommend the four-frame model developed by Lee Bolman and Terrence Deal, which offers distinct lenses through which to view the problem at a higher level. The structural frame helps you explore formal structures (such as hierarchy and reporting relationships); processes (such as workflow); and systems, rules, and policies. This frame examines efficiency, coordination, and alignment of activities.

The human resources frame focuses on people, relationships, and social dynamics. This includes teamwork, leadership, employee motivation, engagement, professional development, and personal growth. In this frame, the organization is seen as a community or a family that recognizes that talent is its most valuable asset. The political frame delves into power dynamics, competing interests, conflicts, coalitions, and negotiations. From this perspective, organizations are arenas where various stakeholders vie for resources and engage in political struggles to influence decisions. It helps you see how power is distributed, used, and contested.

The symbolic frame highlights the importance of symbols, rituals, stories, and shared values in shaping group identity and culture. In it, organizations are depicted as theaters through which its members make meaning.

Using this model, the Omega team generated the following insights in the four frames:

Structural.

A deeper look into the company’s structure revealed siloing and a lack of coordination between the R&D and marketing departments, which had led to misaligned messaging to customers. It also highlighted a lack of collaboration between the two functions and pointed to the need to communicate with the target market about the product’s features and benefits in a coherent and compelling way.

Human resources.

This frame revealed that the declining sales and price hikes had ramped up pressure on the sales team, damaging morale. The demotivated team was struggling to effectively promote the product, making it harder to recover from declining sales. Omega realized it was lacking adequate support, training, and incentives for the team.

The key insight from this frame was that the finance team’s reluctance to approve promotions in the sales group to maintain margins was exacerbating the morale problem. Omega understood that investing in sales leadership development while still generating profits was crucial for long-term success and that frank discussions about the issue were needed.

This frame highlighted an important misalignment in perception: The company believed that its headphones were of “top quality,” while customers reported in surveys that they were “overpriced.” This divergence raised alarm that branding, marketing, and pricing strategies, which were all predicated on the central corporate value of superior quality, were no longer resonating with customers. Omega realized that it had been paying too little attention to quality assurance and functionality.

Phase 5: Envision

In this phase, you transition from framing the problem to actively imagining and designing solutions. This involves synthesizing the insights gained from earlier phases and crafting a shared vision of the desired future state.

Here we recommend using a technique known as backcasting. First, clearly define your desired goal. For example, a team struggling with missed deadlines and declining productivity might aim to achieve on-time completion rates of 98% for its projects and increase its volume of projects by 5% over the next year. Next, reverse engineer the path to achieving your goal. Outline key milestones required over both the short term and the long term. For each one, pinpoint specific interventions, strategies, and initiatives that will propel you closer to your goal. These may encompass changes in processes, policies, technologies, and behaviors. Synthesize the activities into a sequenced, chronological, prioritized road map or action plan, and allocate the resources, including time, budget, and personnel, necessary to implement your plan. Finally, monitor progress toward your goal and be prepared to adjust the plan in response to outcomes, feedback, or changing circumstances. This approach ensures that the team’s efforts in implementing the insights from the previous phases are strategically and purposefully directed toward a concrete destination.

Applying the Approach

Albert Einstein once said, “If I had one hour to solve a problem, I would spend 55 minutes thinking about the problem and five minutes thinking about the solution.” That philosophy underpins our E5 framework, which provides a structured approach for conscientiously engaging with complex problems before leaping to solutions.

As teams use the methodology, they must understand that problem-framing in today’s intricate business landscape is rarely a linear process. While we’re attempting to provide a structured path, we also recognize the dynamic nature of problems and the need for adaptability. Invariably, as teams begin to implement solutions, new facets of a problem may come to light, unforeseen challenges may arise, or external circumstances may evolve. Your team should be ready to loop back to previous phases—for instance, revisiting the expand phase to reassess the problem’s frame, delving deeper into an overlooked root cause in another examine phase, or gathering fresh insights from stakeholders in a new empathize phase. Ultimately, the E5 framework is intended to foster a culture of continuous improvement and innovation.

• JB Julia Binder is the director of the Center for Sustainable and Inclusive Business and a professor of sustainable innovation at IMD.
• Michael D. Watkins is a professor of leadership and organizational change at IMD , a cofounder of Genesis Advisers , and the author of The Six Disciplines of Strategic Thinking .

Partner Center

Complex Problem Solving

• Reference work entry
• Cite this reference work entry

• Joachim Funke 2  

1336 Accesses

51 Citations

3 Altmetric

Dealing with uncertainty ; Dynamic decision making ; Problem solving in dynamic microworlds

Complex problem solving takes place for reducing the barrier between a given start state and an intended goal state with the help of cognitive activities and behavior. Start state, intended goal state, and barriers prove complexity, change dynamically over time, and can be partially intransparent. In contrast to solving simple problems, with complex problems at the beginning of a problem solution the exact features of the start state, of the intended goal state, and of the barriers are unknown. Complex problem solving expects the efficient interaction between the problem-solving person and situational conditions that depend on the task. It demands the use of cognitive, emotional, and social resources as well as knowledge (see Frensch and Funke 1995 ).

Theoretical Background

Since 1975 there has been started a new movement in the psychology of thinking that is engaged in complex...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Brehmer, B., & Dörner, D. (1993). Experiments with computer-simulated microworlds: Escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior, 9 , 171–184.

Google Scholar  

Dörner, D. (1997). The logic of failure. Recognizing and avoiding error in complex situations . New York: Basic Books.

Frensch, P. A., & Funke, J. (Eds.). (1995). Complex problem solving: The European perspective . Hillsdale: Lawrence Erlbaum Associates.

Funke, J. (2003). Problemlösendes Denken . Stuttgart: Kohlhammer.

Osman, M. (2010). Controlling uncertainty: A review of human behavior in complex dynamic environments. Psychological Bulletin, 136 , 65–86.

Wenke, D., Frensch, P. A., & Funke, J. (2005). Complex problem solving and intelligence: Empirical relation and causal direction. In R. J. Sternberg & J. E. Pretz (Eds.), Cognition and intelligence: Identifying the mechanisms of the mind (pp. 160–187). New York: Cambridge University Press.

Download references

Author information

Authors and affiliations.

Department of Psychology, Heidelberg University, Hauptstr. 47-51, Heidelberg, 69117, Germany

Dr. Joachim Funke

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Joachim Funke .

Editor information

Editors and affiliations.

Faculty of Economics and Behavioral Sciences, Department of Education, University of Freiburg, 79085, Freiburg, Germany

Norbert M. Seel

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this entry

Cite this entry.

Funke, J. (2012). Complex Problem Solving. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_685

Download citation

DOI : https://doi.org/10.1007/978-1-4419-1428-6_685

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4419-1427-9

Online ISBN : 978-1-4419-1428-6

eBook Packages : Humanities, Social Sciences and Law Reference Module Humanities and Social Sciences Reference Module Education

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Analysing Complex Problem-Solving Strategies from a Cognitive Perspective: The Role of Thinking Skills

Gyöngyvér molnár.

• Author information
• Article notes
• Copyright and License information

Correspondence: [email protected]

Received 2022 May 31; Accepted 2022 Jul 21; Collection date 2022 Sep.

Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( https://creativecommons.org/licenses/by/4.0/ ).

Complex problem solving (CPS) is considered to be one of the most important skills for successful learning. In an effort to explore the nature of CPS, this study aims to investigate the role of inductive reasoning (IR) and combinatorial reasoning (CR) in the problem-solving process of students using statistically distinguishable exploration strategies in the CPS environment. The sample was drawn from a group of university students (N = 1343). The tests were delivered via the eDia online assessment platform. Latent class analyses were employed to seek students whose problem-solving strategies showed similar patterns. Four qualitatively different class profiles were identified: (1) 84.3% of the students were proficient strategy users, (2) 6.2% were rapid learners, (3) 3.1% were non-persistent explorers, and (4) 6.5% were non-performing explorers. Better exploration strategy users showed greater development in thinking skills, and the roles of IR and CR in the CPS process were varied for each type of strategy user. To sum up, the analysis identified students’ problem-solving behaviours in respect of exploration strategy in the CPS environment and detected a number of remarkable differences in terms of the use of thinking skills between students with different exploration strategies.

Keywords: complex problem solving, thinking skills, logfile analysis, process data

1. Introduction

Problem solving is part and parcel of our daily activities, for instance, in determining what to wear in the morning, how to use our new electronic devices, how to reach a restaurant by public transport, how to arrange our schedule to achieve the greatest work efficiency and how to communicate with people in a foreign country. In most cases, it is essential to solve the problems that recur in our study, work and daily lives. These situations require problem solving. Generally, problem solving is the thinking that occurs if we want “to overcome barriers between a given state and a desired goal state by means of behavioural and/or cognitive, multistep activities” ( Frensch and Funke 1995, p. 18 ). It has also been considered as one of the most important skills for successful learning in the 21st century. This study focuses on one specific kind of problem solving, complex problem solving (CPS). (Numerous other terms are also used ( Funke et al. 2018 ), such as interactive problem solving ( Greiff et al. 2013 ; Wu and Molnár 2018 ), and creative problem solving ( OECD 2010 ), etc.).

CPS is a transversal skill ( Greiff et al. 2014 ), operating several mental activities and thinking skills (see Molnár et al. 2013 ). In order to explore the nature of CPS, some studies have focused on detecting its component skills ( Wu and Molnár 2018 ), whereas others have analysed students’ behaviour during the problem-solving process ( Greiff et al. 2018 ; Wu and Molnár 2021 ). This study aims to link these two fields by investigating the role of thinking skills in learning by examining students’ use of statistically distinguishable exploration strategies in the CPS environment.

1.1. Complex Problem Solving: Definition, Assessment and Relations to Intelligence

According to a widely accepted definition proposed by Buchner ( 1995 ), CPS is “the successful interaction with task environments that are dynamic (i.e., change as a function of users’ intervention and/or as a function of time) and in which some, if not all, of the environment’s regularities can only be revealed by successful exploration and integration of the information gained in that process” ( Buchner 1995, p. 14 ). A CPS process is split into two phases, knowledge acquisition and knowledge application. In the knowledge acquisition (KAC) phase of CPS, the problem solver understands the problem itself and stores the acquired information ( Funke 2001 ; Novick and Bassok 2005 ). In the knowledge application (KAP) phase, the problem solver applies the acquired knowledge to bring about the transition from a given state to a goal state ( Novick and Bassok 2005 ).

Problem solving, especially CPS, has frequently been compared or linked to intelligence in previous studies (e.g., Beckmann and Guthke 1995 ; Stadler et al. 2015 ; Wenke et al. 2005 ). Lotz et al. ( 2017 ) observed that “intelligence and [CPS] are two strongly overlapping constructs” (p. 98). There are many similarities and commonalities that can be detected between CPS and intelligence. For instance, CPS and intelligence share some of the same key features, such as the integration of information ( Stadler et al. 2015 ). Furthermore, Wenke et al. ( 2005 ) stated that “the ability to solve problems has featured prominently in virtually every definition of human intelligence” (p. 9); meanwhile, from the opposite perspective, intelligence has also been considered as one of the most important predictors of the ability to solve problems ( Wenke et al. 2005 ). Moreover, the relation between CPS and intelligence has also been discussed from an empirical perspective. A meta-analysis conducted by Stadler et al. ( 2015 ) selected 47 empirical studies (total sample size N = 13,740) which focused on the correlation between CPS and intelligence. The results of their analysis confirmed that a correlation between CPS and intelligence exists with a moderate effect size of M(g) = 0.43.

Due to the strong link between CPS and intelligence, assessments of these two domains have been connected and have overlapped to a certain extent. For instance, Beckmann and Guthke ( 1995 ) observed that some of the intelligence tests “capture something akin to an individual’s general ability to solve problems (e.g., Sternberg 1982 )” (p. 184). Nowadays, some widely used CPS assessment methods are related to intelligence but still constitute a distinct construct ( Schweizer et al. 2013 ), such as the MicroDYN approach ( Greiff and Funke 2009 ; Greiff et al. 2012 ; Schweizer et al. 2013 ). This approach uses the minimal complex system to simulate simplistic, artificial but still complex problems following certain construction rules ( Greiff and Funke 2009 ; Greiff et al. 2012 ).

The MicroDYN approach has been widely employed to measure problem solving in a well-defined problem context (i.e., “problems have a clear set of means for reaching a precisely described goal state”, Dörner and Funke 2017, p. 1 ). To complete a task based on the MicroDYN approach, the problem solver engages in dynamic interaction with the task to acquire relevant knowledge. It is not possible to create this kind of test environment with the traditional paper-and-pencil-based method. Therefore, it is currently only possible to conduct a MicroDYN-based CPS assessment within the computer-based assessment framework. In the context of computer-based assessment, the problem-solvers’ operations were recorded and logged by the assessment platform. Thus, except for regular achievement-focused result data, logfile data are also available for analysis. This provides the option of exploring and monitoring problem solvers’ behaviour and thinking processes, specifically, their exploration strategies, during the problem-solving process (see, e.g., Chen et al. 2019 ; Greiff et al. 2015a ; Molnár and Csapó 2018 ; Molnár et al. 2022 ; Wu and Molnár 2021 ).

Problem solving, in the context of an ill-defined problem (i.e., “problems have no clear problem definition, their goal state is not defined clearly, and the means of moving towards the (diffusely described) goal state are not clear”, Dörner and Funke 2017, p. 1), involved a different cognitive process than that in the context of a well-defined problem ( Funke 2010 ; Schraw et al. 1995 ), and it cannot be measured with the MicroDYN approach. The nature of ill-defined problem solving has been explored and discussed in numerous studies (e.g., Dörner and Funke 2017 ; Hołda et al. 2020 ; Schraw et al. 1995 ; Welter et al. 2017 ). This will not be discussed here as this study focuses on well-defined problem solving.

1.2. Inductive and Combinatorial Reasoning as Component Skills of Complex Problem Solving

Frensch and Funke ( 1995 ) constructed a theoretical framework that summarizes the basic components of CPS and the interrelations among the components. The framework contains three separate components: problem solver, task and environment. The impact of the problem solver is mainly relevant to three main categories, which are memory contents, dynamic information processing and non-cognitive variables. Some thinking skills have been reported to play an important role in dynamic information processing. We can thus describe them as component skills of CPS. Inductive reasoning (IR) and combinatorial reasoning (CR) are the two thinking skills that have been most frequently discussed as component skills of CPS.

IR is the reasoning skill that has been covered most commonly in the literature. Currently, there is no universally accepted definition. Molnár et al. ( 2013 ) described it as the cognitive process of acquiring general regularities by generalizing single and specific observations and experiences, whereas Klauer ( 1990 ) defined it as the discovery of regularities that relies upon the detection of similarities and/or dissimilarities as concerns attributes of or relations to or between objects. Sandberg and McCullough ( 2010 ) provided a general conclusion of the definitions of IR: it is the process of moving from the specific to the general.

Csapó ( 1997 ) pointed out that IR is a basic component of thinking and that it forms a central aspect of intellectual functioning. Some studies have also discussed the role of IR in a problem-solving environment. For instance, Mayer ( 1998 ) stated that IR will be applied in information processing during the process of solving general problems. Gilhooly ( 1982 ) also pointed out that IR plays a key role in some activities in the problem-solving process, such as hypothesis generation and hypothesis testing. Moreover, the influence of IR on both KAC and KAP has been analysed and demonstrated in previous studies ( Molnár et al. 2013 ).

Empirical studies have also provided evidence that IR and CPS are related. Based on the results of a large-scale assessment (N = 2769), Molnár et al. ( 2013 ) showed that IR significantly correlated with 9–17-year-old students’ domain-general problem-solving achievement (r = 0.44–0.52). Greiff et al. ( 2015b ) conducted a large-scale assessment project (N = 2021) in Finland to explore the links between fluid reasoning skills and domain-general CPS. The study measured fluid reasoning as a two-dimensional model which consisted of deductive reasoning and scientific reasoning and included inductive thinking processes ( Greiff et al. 2015b ). The results drawing on structural equation modelling indicated that fluid reasoning which was partly based on IR had significant and strong predictive effects on both KAC (β = 0.51) and KAP (β = 0.55), the two phases of problem solving. Such studies have suggested that IR is one of the component skills of CPS.

According to Adey and Csapó ’s ( 2012 ) definition, CR is the process of creating complex constructions out of a set of given elements that satisfy the conditions explicitly given in or inferred from the situation. In this process, some cognitive operations, such as combinations, arrangements, permutations, notations and formulae, will be employed ( English 2005 ). CR is one of the basic components of formal thinking ( Batanero et al. 1997 ). The relationship between CR and CPS has frequently been discussed. English ( 2005 ) demonstrated that CR has an essential meaning in several types of problem situations, such as problems requiring the systematic testing of alternative solutions. Moreover, Newell ( 1993 ) pointed out that CR is applied in some key activities of problem-solving information processing, such as strategy generation and application. Its functions include, but are not limited to, helping problem solvers to discover relationships between certain elements and concepts, promoting their fluency of thinking when they are considering different strategies ( Csapó 1999 ) and identifying all possible alternatives ( OECD 2014 ). Moreover, Wu and Molnár ’s ( 2018 ) empirical study drew on a sample (N = 187) of 11–13-year-old primary school students in China. Their study built a structural equation model between CPS, IR and CR, and the result indicated that CR showed a strong and statistically significant predictive power for CPS (β = 0.55). Thus, the results of the empirical study also support the argument that CR is one of the component skills of CPS.

1.3. Behaviours and Strategies in a Complex Problem-Solving Environment

Wüstenberg et al. ( 2012 ) stated that the creation and implementation of strategic exploration are core actions of the problem-solving task. Exploring and generating effective information are key to successfully solving a problem. Wittmann and Hattrup ( 2004 ) illustrated that “riskier strategies [create] a learning environment with greater opportunities to discover and master the rules and boundaries [of a problem]” (p. 406). Thus, when gathering information about a complex problem, there may be differences between exploration strategies in terms of efficacy. The MicroDYN scenarios, a simplification and simulation of the real-world problem-solving context, will also be influenced by the adoption and implementation of exploration strategies.

The effectiveness of the isolated variation strategy (or “Vary-One-Thing-At-A-Time” strategy—VOTAT; Vollmeyer et al. 1996 ) in a CPS environment has been hotly debated ( Chen et al. 2019 ; Greiff et al. 2018 ; Molnár and Csapó 2018 ; Molnár et al. 2022 ; Wu and Molnár 2021 ; Wüstenberg et al. 2014 ). To use the VOTAT strategy, a problem solver “systematically varies only one input variable, whereas the others remain unchanged. This way, the effect of the variable that has just been changed can be observed directly by monitoring the changes in the output variables” ( Molnár and Csapó 2018, p. 2 ). Understanding and using VOTAT effectively is the foundation for developing more complex strategies for coordinating multiple variables and the basis for some phases of scientific thinking (i.e., inquiry, analysis, inference and argument; Kuhn 2010 ; Kuhn et al. 1995 ).

Some previous studies have indicated that students who are able to apply VOTAT are more likely to achieve higher performance in a CPS assessment ( Greiff et al. 2018 ), especially if the problem is a well-defined minimal complex system (such as MicroDYN) ( Fischer et al. 2012 ; Molnár and Csapó 2018 ; Wu and Molnár 2021 ). For instance, Molnár and Csapó ( 2018 ) conducted an empirical study to explore how students’ exploration strategies influence their performance in an interactive problem-solving environment. They measured a group (N = 4371) of 3rd- to 12th-grade (aged 9–18) Hungarian students’ problem-solving achievement and modelled students’ exploration strategies. This result confirmed that students’ exploration strategies influence their problem-solving performance. For example, conscious VOTAT strategy users proved to be the best problem-solvers. Furthermore, other empirical studies (e.g., Molnár et al. 2022 ; Wu and Molnár 2021 ) achieved similar results, thus confirming the importance of VOTAT in a MicroDYN-based CPS environment.

Lotz et al. ( 2017 ) illustrated that effective use of VOTAT is associated with higher levels of intelligence. Their study also pointed out that intelligence has the potential to facilitate successful exploration behaviour. Reasoning skills are an important component of general intelligence. Based on Lotz et al. ’s ( 2017 ) statements, the roles IR and CR play in the CPS process might vary due to students’ different strategy usage patterns. However, there is still a lack of empirical studies in this regard.

2. Research Aims and Questions

Numerous studies have explored the nature of CPS, some of them discussing and analysing it from behavioural or cognitive perspectives. However, there have barely been any that have merged these two perspectives. From the cognitive perspective, this study explores the role of thinking skills (including IR and CR) in the cognition process of CPS. From the behavioural perspective, the study focuses on students’ behaviour (i.e., their exploration strategy) in the CPS assessment process. More specifically, the research aims to fill this gap and examine students’ use of statistically distinguishable exploration strategies in CPS environments and to detect the connection between the level of students’ thinking skills and their behaviour strategies in the CPS environment. The following research questions were thus formed.

What exploration strategy profiles characterise the various problem-solvers at the university level?

Can developmental differences in CPS, IR and CR be detected among students with different exploration strategy profiles?

What are the similarities and differences in the roles IR and CR play in the CPS process as well as in the two phases of CPS (i.e., KAC and KAP) among students with different exploration strategy profiles?

3.1. Participants and Procedure

The sample was drawn from one of the largest universities in Hungary. Participation was voluntary, but students were able to earn one course credit for taking part in the assessment. The participants were students who had just started their studies there (N = 1671). 43.4% of the first-year students took part in the assessment. 50.9% of the participants were female, and 49.1% were male. We filtered the sample and excluded those who had more than 80% missing data on any of the tests. After the data were cleaned, data from 1343 students were available for analysis. The test was designed and delivered via the eDia online assessment system ( Csapó and Molnár 2019 ). The assessment was held in the university ICT room and divided into two sessions. The first session involved the CPS test, whereas the second session entailed the IR and CR tests. Each session lasted 45 min. The language of the tests was Hungarian, the mother tongue of the students.

3.2. Instruments

3.2.1. complex problem solving (cps).

The CPS assessment instrument adopted the MicroDYN approach. It contains a total of twelve scenarios, and each scenario consisted of two items (one item in the KAC phase and one item in the KAP phase in each problem scenario). Twelve KAC items and twelve KAP items were therefore delivered on the CPS test for a total of twenty-four items. Each scenario has a fictional cover story. For instance, students found a sick cat in front of their house, and they were expected to feed the cat with two different kinds of cat food to help it recover.

Each item contains up to three input and three output variables. The relations between the input and output variables were formulated with linear structural equations ( Funke 2001 ). Figure 1 shows a MicroDYN sample structure containing three input variables (A, B and C), three output variables (X, Y and Z) and a number of possible relations between the variables. The complexity of the item was defined by the number of input and output variables, and the number of relations between the variables. The test began with the item with the lowest complexity. The complexity of each item gradually increased as the test progressed.

A typical MicroDYN structure with three input variables and three output variables ( Greiff and Funke 2009 ).

The interface of each item displays the value of each variable in both numerical and figural forms (See Figure 2 ). Each of the input variables has a controller, which makes it possible to vary and set the value between +2 (+ +) and −2 (− −). To operate the system, students need to click the “+” or “−” button or use the slider directly to select the value they want to be added to or subtracted from the current value of the input variable. After clicking the “Apply” button in the interface, the input variables will add or subtract the selected value, and the output variables will show the corresponding changes. The history of the values for the input and output variables within the same problem scenario is displayed on screen. If students want to withdraw all the changes and set all the variables to their original status, they can click the “Reset” button.

Screenshot of the MicroDYN item Cat—first phase (knowledge acquisition). (The items were administered in Hungarian.)

In the first phase of the problem-solving process, the KAC phase, students are asked to interact with the system by changing the value of the input variables and observing and analysing the corresponding changes in the output variables. They are then expected to determine the relationship between the input and output variables and draw it in the form of (an) arrow(s) on the concept map at the bottom of the interface. To avoid item dependence in the second phase of the problem-solving process, the students are provided with a concept map during the KAP phase (see Figure 3 ), which shows the correct connections between the input and output variables. The students are expected to interact with the system by manipulating the input variables to make the output variables reach the given target values in four steps or less. That is, they cannot click on the “Apply” button more than four times. The first phase had a 180 s time limit, whereas the second had a 90 s time limit.

Screenshot of the MicroDYN item Cat—second phase (knowledge application). (The items were administered in Hungarian).

3.2.2. Inductive Reasoning (IR)

The IR instrument (see Figure 4 ) was originally designed and developed in Hungary ( Csapó 1997 ). In the last 25 years, the instrument has been further developed and scaled for a wide age range ( Molnár and Csapó 2011 ). In addition, figural items have been added, and the assessment method has evolved from paper-and-pencil to computer-based ( Pásztor 2016 ). Currently, the instrument is widely employed in a number of countries (see, e.g., Mousa and Molnár 2020 ; Pásztor et al. 2018 ; Wu et al. 2022 ; Wu and Molnár 2018 ). In the present study, four types of items were included after test adaptation: figural series, figural analogies, number analogies and number series. Students were expected to ascertain the correct relationship between the given figures and numbers and select a suitable figure or number as their answer. Students used the drag-and-drop operation to provide their answers. In total, 49 inductive reasoning items were delivered to the participating students.

Sample items for the IR test. (The items were administered in Hungarian.).

3.2.3. Combinatorial Reasoning (CR)

The CR instrument (see Figure 5 ) was originally designed by Csapó ( 1988 ). The instrument was first developed in paper-and-pencil format and then modified for computer use ( Pásztor and Csapó 2014 ). Each item contained figural or verbal elements and a clear requirement for combing through the elements. Students were asked to list every single combination based on a given rule they could find. For the figural items, students provided their answers using the drag-and-drop operation; for the verbal items, they were asked to type their answers in a text box provided on screen. The test consisted of eight combinatorial reasoning items in total.

Sample item for the CR test. (The items were administered in Hungarian).

3.3. Scoring

Students’ performance was automatically scored via the eDia platform. Items on the CPS and IR tests were scored dichotomously. In the first phase (KAC) of the CPS test, if a student drew all the correct relations on the concept map provided on screen within the given timeframe, his/her performance was assigned a score of 1 or otherwise a score of 0. In the second phase (KAP) of the CPS test, if the student successfully reached the given target values of the output variables by manipulating the level of the input variables within no more than four steps and the given timeframe, then his/her performance earned a score of 1 or otherwise a score of 0. On the IR test items, if a student selected the correct figure or number as his/her answer, then he or she received a score of 1; otherwise, the score was 0.

Students’ performance on the CR test items was scored according to a special J index, which was developed by Csapó ( 1988 ). The J index ranges from 0 to 1, where 1 means that the student provided all the correct combinations without any redundant combinations on the task. The formula for computing the J index is the following:

x stands for the number of correct combinations in the student’s answer,

T stands for the number of all possible correct combinations, and

y stands for the number of redundant combinations in the student’s answer.

Furthermore, according to Csapó ’s ( 1988 ) design, if y is higher than T, then the J index will be counted as 0.

3.4. Coding and Labelling the Logfile Data

Beyond concrete answer data, students’ interaction and manipulation behaviour were also logged in the assessment system. This made it possible to analyse students’ exploration behaviour in the first phase of the CPS process (KAC phase). Toward this aim, we adopted a labelling system developed by Molnár and Csapó ( 2018 ) to transfer the raw logfile data to structured data files for analysis. Based on the system, each trial (i.e., the sum of manipulations within the same problem scenario which was applied and tested by clicking the “Apply” button) was modelled as a single data entity. The sum of these trials within the same problem was defined as a strategy. In our study, we only consider the trials which were able to provide useful and new information for the problem-solvers, whereas the redundant or operations trials were excluded.

In this study, we analysed students’ trials to determine the extent to which they used the VOTAT strategy: fully, partially or not at all. This strategy is the most successful exploration strategy for such problems; it is the easiest to interpret and provides direct information about the given variable without any mediation effects ( Fischer et al. 2012 ; Greiff et al. 2018 ; Molnár and Csapó 2018 ; Wüstenberg et al. 2014 ; Wu and Molnár 2021 ). Based on the definition of VOTAT noted in Section 1.3 , we checked students’ trials to ascertain if they systematically varied one input variable while keeping the others unchanged, or applied a different, less successful strategy. We considered the following three types of trials:

“Only one single input variable was manipulated, whose relationship to the output variables was unknown (we considered a relationship unknown if its effect cannot be known from previous settings), while the other variables were set at a neutral value like zero […]

One single input variable was changed, whose relationship to the output variables was unknown. The others were not at zero, but at a setting used earlier. […]

One single input variable was changed, whose relationship to the output variables was unknown, and the others were not at zero; however, the effect of the other input variable(s) was known from earlier settings. Even so, this combination was not attempted earlier” ( Molnár and Csapó 2018, p. 8 )

We used the numbers 0, 1 and 2 to distinguish the level of students’ use of the most effective exploration strategy (i.e., VOTAT). If a student applied one or more of the above trials for every input variable within the same scenario, we considered that they had used the full VOTAT strategy and labelled this behaviour 2. If a student had only employed VOTAT on some but not all of the input variables, we concluded that they had used a partial VOTAT strategy for that problem scenario and labelled it 1. If a student had used none of the trials noted above in their problem exploration, then we determined that they had not used VOTAT at all and thus gave them a label of 0.

3.5. Data Analysis Plan

We used LCA (latent class analysis) to explore students’ exploration strategy profiles. LCA is a latent variable modelling approach that can be used to identify unmeasured (latent) classes of samples with similarly observed variables. LCA has been widely used in analysing logfile data for CPS assessment and in exploring students’ behaviour patterns (see, e.g., Gnaldi et al. 2020 ; Greiff et al. 2018 ; Molnár et al. 2022 ; Molnár and Csapó 2018 ; Mustafić et al. 2019 ; Wu and Molnár 2021 ). The scores for the use of VOTAT in the KAC phase (0, 1, 2; see Section 3.4 ) were used for the LCA analysis. We used Mplus ( Muthén and Muthén 2010 ) to run the LCA analysis. Several indices were used to measure the model fit: AIC (Akaike information criterion), BIC (Bayesian information criterion) and aBIC (adjusted Bayesian information criterion). With these three indicators, lower values indicate a better model fit. Entropy (ranging from 0 to 1, with values close to 1 indicating high certainty in the classification). The Lo–Mendell–Rubin adjusted likelihood ratio was used to compare the model containing n latent classes with the model containing n − 1 latent classes, and the p value was the indicator for whether a significant difference could be detected ( Lo et al. 2001 ). The results of the Lo–Mendell–Rubin adjusted likelihood ratio analysis were used to decide the correct number of latent classes in LCA models.

ANOVA was used to analyse the performance differences for CPS, IR and CR across the students from the different class profiles. The analysis was run using SPSS. A path analysis (PA) was employed in the structural equation modelling (SEM) framework to investigate the roles of CR and IR in CPS and the similarities and differences across the students from the different exploration strategy profiles. The PA models were carried out with Mplus. The Tucker–Lewis index (TLI), the comparative fit index (CFI) and the root-mean-square error of approximation (RMSEA) were used as indicators for the model fit. A TLI and CFI larger than 0.90 paired with a RMSEA less than 0.08 are commonly considered as an acceptable model fit ( van de Schoot et al. 2012 ).

4.1. Descriptive Results

All three tests showed good reliability (Cronbach’s α: CPS: 0.89; IR: 0.87; CR: 0.79). Furthermore, the two sub-dimensions of the CPS test, KAC and KAP, also showed satisfactory reliability (Cronbach’s α: KAC: 0.86; KAP: 0.78). The tests thus proved to be reliable. The means and standard deviations of students’ performance (in percentage) on each test are provided in Table 1 .

The means and standard deviations of students’ performance on each test.

4.2. Four Qualitatively Different Exploration Strategy Profiles Can Be Distinguished in CPS

Based on the labelled logfile data for CPS, we applied latent class analyses to identify the behaviour patterns of the students in the exploration phase of the problem-solving process. The model fits for the LCA analysis are listed in Table 2 . Compared with the 2 or 3 latent class models, the 4 latent class model has a lower AIC, BIC and aBIC, and the likelihood ratio statistical test (the Lo–Mendell–Rubin adjusted likelihood ratio test) confirmed it has a significantly better model fit. The 5 and 6 latent class models did not show a better model fit than the 4 latent class model. Therefore, based on the results, four qualitatively different exploration strategy profiles can be distinguished, which covered 96% of the students.

Fit indices for latent class analyses.

The patterns for the four qualitatively different exploration strategy profiles are shown in Figure 6 . In total, 84.3% of the students were proficient exploration strategy users, who were able to use VOTAT in each problem scenario independent of its difficulty level (represented by the red line in Figure 5 ). In total, 6.2% of the students were rapid learners. They were not able to apply VOTAT at the beginning of the test on the easiest problems but managed to learn quickly, and, after a rapid learning curve by the end of the test, they reached the level of proficient exploration strategy users, even though the problems became much more complex (represented by the blue line). In total, 3.1% of the students proved to be non-persistent explorers, and they employed VOTAT on the easiest problems but did not transfer this knowledge to the more complex problems. Finally, they were no longer able to apply VOTAT when the complexity of the problems increased (represented by the green line). In total, 6.5% of the students were non-performing explorers; they barely used any VOTAT strategy during the whole test (represented by the pink line) independent of problem complexity.

Four qualitatively different exploration strategy profiles.

4.3. Better Exploration Strategy Users Showed Better Performance in Reasoning Skills

Students with different exploration strategy profiles showed different kinds of performance in each reasoning skill under investigation. Results (see Table 3 ) showed that more proficient strategy users tended to have higher achievement in all the domains assessed as well as in the two sub-dimensions in CPS (i.e., KAC and KAP; ANOVA: CPS: F(3, 1339) = 187.28, p < 0.001; KAC: F(3, 1339) = 237.15, p < 0.001; KAP: F(3, 1339) = 74.91, p < 0.001; IR: F(3, 1339) = 48.10, p < 0.001; CR: F(3, 1339) = 28.72, p < 0.001); specifically, students identified as “proficient exploration strategy users” achieved the highest level on the reasoning skills tests independent of the domains. On average, they were followed by rapid learners, non-persistent explorers and, finally, non-performing explorers. Tukey’s post hoc tests revealed more details on the performance differences of students with different exploration profiles in each of the domains being measured. Proficient strategy users proved to be significantly more skilled in each of the reasoning domains. They were followed by rapid learners, who outperformed non-persistent explorers and non-performing explorers in CPS. In the domains of IR and CR, there were no achievement differences between rapid learners and non-persistent explorers, who significantly outperformed non-performing strategy explorers.

Students’ performance on each test—grouped according to the different exploration strategy profiles.

4.4. The Roles of IR and CR in CPS and Its Processes Were Different for Each Type of Exploration Strategy User

Path analysis was used to explore the predictive power of IR and CR for CPS and its processes, knowledge acquisition and knowledge application, for each group of students with different exploration strategy profiles. That is, four path analysis models were built to indicate the predictive power of IR and CR for CPS (see Figure 7 ), and another four path analyses models were developed to monitor the predictive power of IR and CR for the two empirically distinguishable phases of CPS (i.e., KAC and KAP) (see Figure 8 ). All eight models had good model fits, the fit indices TLI and CFI were above 0.90, and RMSEA was less than 0.08.

Path analysis models (with CPS, IR and CR) for each type of strategy user; * significant at 0.05 ( p   <  0.05); ** significant at 0.01 ( p   <  0.01); N.S.: no significant effect can be found.

Path analysis models (with KAC, KAP, IR and CR) for each type of strategy user; * significant at 0.05 ( p  <  0.05); ** significant at 0.01 ( p  <  0.01); N.S.: no significant effect can be found.

Students’ level of IR significantly predicted their level of CPS in all four path analysis models independent of their exploration strategy profile ( Figure 7 ; proficient strategy users: β = 0.432, p < 0.01; rapid learners: β = 0.350, p < 0.01; non-persistent explorers: β = 0.309, p < 0.05; and non-performing explorers: β = 0.386, p < 0.01). This was not the case for CR, which only proved to have predictive power for CPS among proficient strategy users (β = 0.104, p < 0.01). IR and CR were significantly correlated in all four models.

After examining the roles of IR and CR in the CPS process, we went further to explore the roles of these two reasoning skills in the distinguishable phases of CPS. The path analysis models ( Figure 8 ) showed that the predictive power of IR and CR for KAC and KAP was varied in each group. Levels of IR and CR among non-persistent explorers and non-performing explorers failed to predict their achievement in the KAC phase of the CPS process. Moreover, rapid learners’ level of IR significantly predicted their achievement in the KAC phase (β = 0.327, p < 0.01), but their level of CR did not have the same predictive power. Furthermore, the proficient strategy users’ levels of both reasoning skills had significant predictive power for KAC (IR: β = 0.363, p < 0.01; CR: β = 0.132, p < 0.01). In addition, in the KAP phase of the CPS problems, IR played a significant role for all types of strategy users, although with different power (proficient strategy users: β = 0.408, p < 0.01; rapid learners: β = 0.339, p < 0.01; non-persistent explorers: β = 0.361, p < 0.01; and non-performing explorers: β = 0.447, p < 0.01); by contrast, CR did not have significant predictive power for the KAP phase in any of the models.

5. Discussion

The study aims to investigate the role of IR and CR in CPS and its phases among students using statistically distinguishable exploration strategies in different CPS environments. We examined 1343 Hungarian university students and assessed their CPS, IR and CR skills. Both achievement data and logfile data were used in the analysis. The traditional achievement indicators formed the foundation for analysing the students’ CPS, CR and IR performance, whereas process data extracted from logfile data were used to explore students’ exploration behaviour in various CPS environments.

Four qualitatively different exploration strategy profiles were distinguished: proficient strategy users, rapid learners, non-persistent explorers and non-performing explorers (RQ1). The four profiles were consistent with the result of another study conducted at university level (see Molnár et al. 2022 ), and the frequencies of these four profiles in these two studies were very similar. The two studies therefore corroborate and validate each other’s results. The majority of the participants were identified as proficient strategy users. More than 80% of the university students were able to employ effective exploration strategies in various CPS environments. Of the remaining students, some performed poorly in exploration strategy use in the early part of the test (rapid learners), some in the last part (non-persistent explorers) and some throughout the test (non-performing explorers). However, students with these three exploration strategy profiles only constituted small portions of the total sample (with proportions ranging from 3.1% to 6.5%). The university students therefore exhibited generally good performance in terms of exploration strategy use in a CPS environment, especially compared with previous results among younger students (e.g., primary school students, see Greiff et al. 2018 ; Wu and Molnár 2021 ; primary to secondary students, see Molnár and Csapó 2018 ).

The results have indicated that better exploration strategy users achieved higher CPS performance and had better development levels of IR and CR (RQ2). First, the results have confirmed the importance of VOTAT in a CPS environment. This finding is consistent with previous studies (e.g., Greiff et al. 2015a ; Molnár and Csapó 2018 ; Mustafić et al. 2019 ; Wu and Molnár 2021 ). Second, the results have confirmed that effective use of VOTAT is strongly tied to the level of IR and CR development. Reasoning forms an important component of human intelligence, and the level of development in reasoning was an indicator of the level of intelligence ( Klauer et al. 2002 ; Sternberg and Kaufman 2011 ). Therefore, this finding has supplemented empirical evidence for the argument that effective use of VOTAT is associated with levels of intelligence to a certain extent.

The roles of IR and CR proved to be varied for each type of exploration strategy user (RQ3). For instance, the level of CPS among the best exploration strategy users (i.e., the proficient strategy users) was predicted by both the levels of IR and CR, but this was not the case for students with other profiles. In addition, the results have indicated that IR played important roles in both the KAC and KAP phases for the students with relatively good exploration strategy profiles (i.e., proficient strategy users and rapid learners) but only in the KAP phase for the rest of the students (non-persistent explorers and non-performing explorers); moreover, the predictive power of CR can only be detected in the KAC phase of the proficient strategy users. To sum up, the results suggest a general trend of IR and CR playing more important roles in the CPS process among better exploration strategy users.

Combining the answers to RQ2 and RQ3, we can gain further insights into students’ exploration strategy use in a CPS environment. Our results have confirmed that the use of VOTAT is associated with the level of IR and CR development and that the importance of IR and CR increases with proficiency in exploration strategy use. Based on these findings, we can make a reasonable argument that IR and CR are essential skills for using VOTAT and that underdeveloped IR and CR will prevent students from using effective strategies in a CPS environment. Therefore, if we want to encourage students to become better exploration strategy users, it is important to first enhance their IR and CR skills. Previous studies have suggested that establishing explicit training in using effective strategies in a CPS environment is important for students’ CPS development ( Molnár et al. 2022 ). Our findings have identified the importance of IR and CR in exploration strategy use, which has important implications for designing training programmes.

The results have also provided a basis for further studies. Future studies have been suggested to further link the behavioural and cognitive perspectives in CPS research. For instance, IR and CR were considered as component skills of CPS (see Section 1.2 ). The results of the study have indicated the possibility of not only discussing the roles of IR and CR in the cognitive process of CPS, but also exploration behaviour in a CPS environment. The results have thus provided a new perspective for exploring the component skills of CPS.

6. Limitations

There are some limitations in the study. All the tests were low stake; therefore, students might not be sufficiently motivated to do their best. This feature might have produced the missing values detected in the sample. In addition, some students’ exploration behaviour shown in this study might theoretically be below their true level. However, considering that data cleaning was adopted in this study (see Section 3.1 ), we believe this phenomenon will not have a remarkable influence on the results. Moreover, the CPS test in this study was based on the MicroDYN approach, which is a well-established and widely used artificial model with a limited number of variables and relations. However, it does not have the power to cover all kinds of complex and dynamic problems in real life. For instance, the MicroDYN approach cannot measure ill-defined problem solving. Thus, this study can only demonstrate the influence of IR and CR on problem solving in well-defined MicroDYN-simulated problems. Furthermore, VOTAT is helpful with minimally complex problems under well-defined laboratory conditions, but it may not be that helpful with real-world, ill-defined complex problems ( Dörner and Funke 2017 ; Funke 2021 ). Therefore, the generalizability of the findings is limited.

7. Conclusions

In general, the results have shed new light on students’ problem-solving behaviours in respect of exploration strategy in a CPS environment and explored differences in terms of the use of thinking skills between students with different exploration strategies. Most studies discuss students’ problem-solving strategies from a behavioural perspective. By contrast, this paper discusses them from both behavioural and cognitive perspectives, thus expanding our understanding in this area. As for educational implications, the study contributes to designing and revising training methods for CPS by identifying the importance of IR and CR in exploration behaviour in a CPS environment. To sum up, the study has investigated the nature of CPS from a fresh angle and provided a sound basis for future studies.

Author Contributions

Conceptualization, H.W. and G.M.; methodology, H.W. and G.M.; formal analysis, H.W.; writing—original draft preparation, H.W.; writing—review and editing, G.M.; project administration, G.M.; funding acquisition, G.M. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Ethical approval was not required for this study in accordance with the national and institutional guidelines. The assessments which provided data for this study were integrated parts of the educational processes of the participating university. The participation was voluntary.

Informed Consent Statement

All of the students in the assessment turned 18, that is, it was not required or possible to request and obtain written informed parental consent from the participants.

Data Availability Statement

The data used to support the findings cannot be shared at this time as it also forms part of an ongoing study.

Conflicts of Interest

Authors declare no conflict of interest.

Funding Statement

This study has been conducted with support provided by the National Research, Development and Innovation Fund of Hungary, financed under the OTKA K135727 funding scheme and supported by the Research Programme for Public Education Development, Hungarian Academy of Sciences (KOZOKT2021-16).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Adey Philip, Csapó Benő. Developing and Assessing Scientific Reasoning. In: Csapó Benő, Szabó Gábor., editors. Framework for Diagnostic Assessment of Science. Nemzeti Tankönyvkiadó; Budapest: 2012. pp. 17–53. [ Google Scholar ]
• Batanero Carmen, Navarro-Pelayo Virginia, Godino Juan D. Effect of the implicit combinatorial model on combinatorial reasoning in secondary school pupils. Educational Studies in Mathematics. 1997;32:181–99. doi: 10.1023/A:1002954428327. [ DOI ] [ Google Scholar ]
• Beckmann Jens F., Guthke Jürgen. Complex problem solving, intelligence, and learning ability. In: Frensch Peter A., Funke Joachim., editors. Complex Problem Solving: The European Perspective. Erlbaum; Hillsdale: 1995. pp. 177–200. [ Google Scholar ]
• Buchner Axel. Basic topics and approaches to the study of complex problem solving. In: Frensch Peter A., Funke Joachim., editors. Complex Problem Solving: The European Perspective. Erlbaum; Hillsdale: 1995. pp. 27–63. [ Google Scholar ]
• Chen Yunxiao, Li Xiaoou, Liu Jincheng, Ying Zhiliang. Statistical analysis of complex problem-solving process data: An event history analysis approach. Frontiers in Psychology. 2019;10:486. doi: 10.3389/fpsyg.2019.00486. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Csapó Benő. A kombinatív képesség struktúrája és fejlődése. Akadémiai Kiadó; Budapest: 1988. [ Google Scholar ]
• Csapó Benő. The development of inductive reasoning: Cross-sectional assessments in an educational context. International Journal of Behavioral Development. 1997;20:609–26. doi: 10.1080/016502597385081. [ DOI ] [ Google Scholar ]
• Csapó Benő. Teaching and Learning Thinking Skills. Swets & Zeitlinger; Lisse: 1999. Improving thinking through the content of teaching; pp. 37–62. [ Google Scholar ]
• Csapó Benő, Molnár Gyöngyvér. Online diagnostic assessment in support of personalized teaching and learning: The eDia System. Frontiers in Psychology. 2019;10:1522. doi: 10.3389/fpsyg.2019.01522. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Dörner Dietrich, Funke Joachim. Complex problem solving: What it is and what it is not. Frontiers in Psychology. 2017;8:1153. doi: 10.3389/fpsyg.2017.01153. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• English Lyn D. Combinatorics and the development of children’s combinatorial reasoning. In: Jones Graham A., editor. Exploring Probability in School: Challenges for Teaching and Learning. Springer; New York: 2005. pp. 121–41. [ Google Scholar ]
• Fischer Andreas, Greiff Samuel, Funke Joachim. The process of solving complex problems. Journal of Problem Solving. 2012;4:19–42. doi: 10.7771/1932-6246.1118. [ DOI ] [ Google Scholar ]
• Frensch Peter A., Funke Joachim. Complex Problem Solving: The European Perspective. Psychology Press; New York: 1995. [ Google Scholar ]
• Funke Joachim. Dynamic systems as tools for analysing human judgement. Thinking and Reasoning. 2001;7:69–89. doi: 10.1080/13546780042000046. [ DOI ] [ Google Scholar ]
• Funke Joachim. Complex problem solving: A case for complex cognition? Cognitive Processing. 2010;11:133–42. doi: 10.1007/s10339-009-0345-0. [ DOI ] [ PubMed ] [ Google Scholar ]
• Funke Joachim. It Requires More Than Intelligence to Solve Consequential World Problems. Journal of Intelligence. 2021;9:38. doi: 10.3390/jintelligence9030038. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Funke Joachim, Fischer Andreas, Holt Daniel V. Competencies for complexity: Problem solving in the twenty-first century. In: Care Esther, Griffin Patrick, Wilson Mark., editors. Assessment and Teaching of 21st Century Skills. Springer; Dordrecht: 2018. pp. 41–53. [ Google Scholar ]
• Gilhooly Kenneth J. Thinking: Directed, Undirected and Creative. Academic Press; London: 1982. [ Google Scholar ]
• Gnaldi Michela, Bacci Silvia, Kunze Thiemo, Greiff Samuel. Students’ complex problem solving profiles. Psychometrika. 2020;85:469–501. doi: 10.1007/s11336-020-09709-2. [ DOI ] [ PubMed ] [ Google Scholar ]
• Greiff Samuel, Funke Joachim. Measuring complex problem solving-the MicroDYN approach. In: Scheuermann Friedrich, Björnsson Julius., editors. The Transition to Computer-Based Assessment. Office for Official Publications of the European Communities; Luxembourg: 2009. pp. 157–63. [ Google Scholar ]
• Greiff Samuel, Holt Daniel V., Funke Joachim. Perspectives on problem solving in educational assessment: Analytical, interactive, and collaborative problem solving. Journal of Problem Solving. 2013;5:71–91. doi: 10.7771/1932-6246.1153. [ DOI ] [ Google Scholar ]
• Greiff Samuel, Molnár Gyöngyvér, Martina Romain, Zimmermann Johannes, Csapó Benő. Students’ exploration strategies in computer-simulated complex problem environments: A latent class approach. Computers & Education. 2018;126:248–63. [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Avvisati Francesco. Computer-generated log-file analyses as a window into students’ minds? A showcase study based on the PISA 2012 assessment of problem solving. Computers & Education. 2015a;91:92–105. [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Funke Joachim. Dynamic problem solving: A new measurement perspective. Applied Psychological Measurement. 2012;36:189–213. doi: 10.1177/0146621612439620. [ DOI ] [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Csapó Benő, Demetriou Andreas, Hautamäki Jarkko, Graesser Arthur C., Martin Romain. Domain-general problem solving skills and education in the 21st century. Educational Research Review. 2014;13:74–83. doi: 10.1016/j.edurev.2014.10.002. [ DOI ] [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Goetz Thomas, Vainikainen Mari-Pauliina, Hautamäki Jarkko, Bornstein Marc H. A longitudinal study of higher-order thinking skills: Working memory and fluid reasoning in childhood enhance complex problem solving in adolescence. Frontiers in Psychology. 2015b;6:1060. doi: 10.3389/fpsyg.2015.01060. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Hołda Małgorzata, Głodek Anna, Dankiewicz-Berger Malwina, Skrzypińska Dagna, Szmigielska Barbara. Ill-defined problem solving does not benefit from daytime napping. Frontiers in Psychology. 2020;11:559. doi: 10.3389/fpsyg.2020.00559. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Klauer Karl Josef. Paradigmatic teaching of inductive thinking. Learning and Instruction. 1990;2:23–45. [ Google Scholar ]
• Klauer Karl Josef, Willmes Klaus, Phye Gary D. Inducing inductive reasoning: Does it transfer to fluid intelligence? Contemporary Educational Psychology. 2002;27:1–25. doi: 10.1006/ceps.2001.1079. [ DOI ] [ Google Scholar ]
• Kuhn Deanna. What is scientific thinking and how does it develop? In: Goswami Usha., editor. The Wiley-Blackwell Handbook of Childhood Cognitive Development. Wiley-Blackwell; Oxford: 2010. pp. 371–93. [ Google Scholar ]
• Kuhn Deanna, Garcia-Mila Merce, Zohar Anat, Andersen Christopher, Sheldon H. White, Klahr David, Carver Sharon M. Strategies of knowledge acquisition. Monographs of the Society for Research in Child Development. 1995;60:1–157. doi: 10.2307/1166059. [ DOI ] [ Google Scholar ]
• Lo Yungtai, Mendell Nancy R., Rubin Donald B. Testing the number of components in a normal mixture. Biometrika. 2001;88:767–78. doi: 10.1093/biomet/88.3.767. [ DOI ] [ Google Scholar ]
• Lotz Christin, Scherer Ronny, Greiff Samuel, Sparfeldt Jörn R. Intelligence in action—Effective strategic behaviors while solving complex problems. Intelligence. 2017;64:98–112. doi: 10.1016/j.intell.2017.08.002. [ DOI ] [ Google Scholar ]
• Mayer Richard E. Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science. 1998;26:49–63. doi: 10.1023/A:1003088013286. [ DOI ] [ Google Scholar ]
• Molnár Gyöngyvér, Csapó Benő. Az 1–11 évfolyamot átfogó induktív gondolkodás kompetenciaskála készítése a valószínűségi tesztelmélet alkalmazásával. Magyar Pedagógia. 2011;111:127–40. [ Google Scholar ]
• Molnár Gyöngyvér, Csapó Benő. The efficacy and development of students’ problem-solving strategies during compulsory schooling: Logfile analyses. Frontiers in Psychology. 2018;9:302. doi: 10.3389/fpsyg.2018.00302. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Molnár Gyöngyvér, Alrababah Saleh Ahmad, Greiff Samuel. How we explore, interpret, and solve complex problems: A cross-national study of problem-solving processes. Heliyon. 2022;8:e08775. doi: 10.1016/j.heliyon.2022.e08775. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Molnár Gyöngyvér, Greiff Samuel, Csapó Benő. Inductive reasoning, domain specific and complex problem solving: Relations and development. Thinking Skills and Creativity. 2013;9:35–45. doi: 10.1016/j.tsc.2013.03.002. [ DOI ] [ Google Scholar ]
• Mousa Mojahed, Molnár Gyöngyvér. Computer-based training in math improves inductive reasoning of 9- to 11-year-old children. Thinking Skills and Creativity. 2020;37:100687. doi: 10.1016/j.tsc.2020.100687. [ DOI ] [ Google Scholar ]
• Mustafić Maida, Yu Jing, Stadler Matthias, Vainikainen Mari-Pauliina, Bornstein Marc H., Putnick Diane L., Greiff Samuel. Complex problem solving: Profiles and developmental paths revealed via latent transition analysis. Developmental Psychology. 2019;55:2090–101. doi: 10.1037/dev0000764. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Muthén Linda K., Muthén Bengt O. Mplus User’s Guide. Muthén & Muthén; Los Angeles: 2010. [ Google Scholar ]
• Newell Allen. Reasoning, Problem Solving, and Decision Processes: The Problem Space as a Fundamental Category. MIT Press; Boston: 1993. [ Google Scholar ]
• Novick Laura R., Bassok Miriam. Problem solving. In: Holyoak Keith James, Morrison Robert G., editors. The Cambridge Handbook of Thinking and Reasoning. Cambridge University Press; New York: 2005. pp. 321–49. [ Google Scholar ]
• OECD . PISA 2012 Field Trial Problem Solving Framework. OECD Publishing; Paris: 2010. [ Google Scholar ]
• OECD . Results: Creative Problem Solving—Students’ Skills in Tackling Real-Life Problems (Volume V) OECD Publishing; Paris: 2014. [ Google Scholar ]
• Pásztor Attila. Ph.D. thesis. Doctoral School of Education, University of Szeged; Szeged, Hungary: 2016. Technology-Based Assessment and Development of Inductive Reasoning. [ Google Scholar ]
• Pásztor Attila, Csapó Benő. Improving Combinatorial Reasoning through Inquiry-Based Science Learning; Paper presented at the Science and Mathematics Education Conference; Dublin, Ireland. June 24–25; 2014. [ Google Scholar ]
• Pásztor Attila, Kupiainen Sirkku, Hotulainen Risto, Molnár Gyöngyvér, Csapó Benő. Comparing Finnish and Hungarian Fourth Grade Students’ Inductive Reasoning Skills; Paper presented at the EARLI SIG 1 Conference; Helsinki, Finland. August 29–31; 2018. [ Google Scholar ]
• Sandberg Elisabeth Hollister, McCullough Mary Beth. The development of reasoning skills. In: Sandberg Elisabeth Hollister, Spritz Becky L., editors. A Clinician’s Guide to Normal Cognitive Development in Childhood. Routledge; New York: 2010. pp. 179–89. [ Google Scholar ]
• Schraw Gregory, Dunkle Michael E., Bendixen Lisa D. Cognitive processes in well-defined and ill-defined problem solving. Applied Cognitive Psychology. 1995;9:523–38. doi: 10.1002/acp.2350090605. [ DOI ] [ Google Scholar ]
• Schweizer Fabian, Wüstenberg Sascha, Greiff Samuel. Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences. 2013;24:42–52. doi: 10.1016/j.lindif.2012.12.011. [ DOI ] [ Google Scholar ]
• Stadler Matthias, Becker Nicolas, Gödker Markus, Leutner Detlev, Greiff Samuel. Complex problem solving and intelligence: A meta-analysis. Intelligence. 2015;53:92–101. doi: 10.1016/j.intell.2015.09.005. [ DOI ] [ Google Scholar ]
• Sternberg Robert J. Handbook of Human Intelligence. Cambridge University Press; New York: 1982. [ Google Scholar ]
• Sternberg Robert J., Kaufman Scott Barry. The Cambridge Handbook of Intelligence. Cambridge University Press; New York: 2011. [ Google Scholar ]
• van de Schoot Rens, Lugtig Peter, Hox Joop. A checklist for testing measurement invariance. European Journal of Developmental Psychology. 2012;9:486–92. doi: 10.1080/17405629.2012.686740. [ DOI ] [ Google Scholar ]
• Vollmeyer Regina, Burns Bruce D., Holyoak Keith J. The impact of goal specificity on strategy use and the acquisition of problem structure. Cognitive Science. 1996;20:75–100. doi: 10.1207/s15516709cog2001_3. [ DOI ] [ Google Scholar ]
• Welter Marisete Maria, Jaarsveld Saskia, Lachmann Thomas. Problem space matters: The development of creativity and intelligence in primary school children. Creativity Research Journal. 2017;29:125–32. doi: 10.1080/10400419.2017.1302769. [ DOI ] [ Google Scholar ]
• Wenke Dorit, Frensch Peter A., Funke Joachim. Complex Problem Solving and intelligence: Empirical relation and causal direction. In: Sternberg Robert J., Pretz Jean E., editors. Cognition and Intelligence: Identifying the Mechanisms of the Mind. Cambridge University Press; New York: 2005. pp. 160–87. [ Google Scholar ]
• Wittmann Werner W., Hattrup Keith. The relationship between performance in dynamic systems and intelligence. Systems Research and Behavioral Science. 2004;21:393–409. doi: 10.1002/sres.653. [ DOI ] [ Google Scholar ]
• Wu Hao, Molnár Gyöngyvér. Interactive problem solving: Assessment and relations to combinatorial and inductive reasoning. Journal of Psychological and Educational Research. 2018;26:90–105. [ Google Scholar ]
• Wu Hao, Molnár Gyöngyvér. Logfile analyses of successful and unsuccessful strategy use in complex problem-solving: A cross-national comparison study. European Journal of Psychology of Education. 2021;36:1009–32. doi: 10.1007/s10212-020-00516-y. [ DOI ] [ Google Scholar ]
• Wu Hao, Saleh Andi Rahmat, Molnár Gyöngyvér. Inductive and combinatorial reasoning in international educational context: Assessment, measurement invariance, and latent mean differences. Asia Pacific Education Review. 2022;23:297–310. doi: 10.1007/s12564-022-09750-z. [ DOI ] [ Google Scholar ]
• Wüstenberg Sascha, Greiff Samuel, Funke Joachim. Complex problem solving—More than reasoning? Intelligence. 2012;40:1–14. doi: 10.1016/j.intell.2011.11.003. [ DOI ] [ Google Scholar ]
• Wüstenberg Sascha, Greiff Samuel, Molnár Gyöngyvér, Funke Joachim. Cross-national gender differences in complex problem solving and their determinants. Learning and Individual Differences. 2014;29:18–29. doi: 10.1016/j.lindif.2013.10.006. [ DOI ] [ Google Scholar ]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

• View on publisher site
• PDF (3.0 MB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections