Integrated use of simulation science tools, system dynamics modeling and wargaming, to cultivate a larger analytic process to develop evaluation criteria for game design and systems thinking leaders

Johnson, Anne

Etd

Integrated use of simulation science tools, system dynamics modeling and wargaming, to cultivate a larger analytic process to develop evaluation criteria for game design and systems thinking leaders

Öffentlich Deposited

This research investigated overlap of wargaming design and development, system dynamics modeling and systems thinking to baseline an integrated modeling, simulation, and analysis (MSA) framework to explore behavioral characteristics and potential policy spaces for leaders to understand and manage the complex systems that create messy problems of interest to national security. Wargaming is used by military leaders as a tool to manage messy problems of interest to national security but there are no criteria to evaluate game design or learning objectives. A simulation/game was developed using the publicly available Battle for Moscow wargame as a laboratory tool for data collection. Expanded policy space was incorporated to allow exploration of counter-intuitive behaviors and unintended consequences that could impact the problem behavior beyond the invasion policy captured in the Battle for Moscow wargame. The simulation/game provided visuals as part of the interface to allow participants to observe changes in system output behaviors due to policy decisions they made during the simulation/game. The simulation/game provided consistency of system behavioral response to policy decisions made by participants exploring several policy spaces. Consistency of response behaviors could be leveraged to compare results of policy choices across in future research. A rubric was developed to evaluate change to eight (8) dimensions of systems thinking when using the simulation/game as an intervention during a pre-test/post-test survey. Results include a quantifiable simulation/game of a messy land force resistance problem with measurable variables linking interacting policy choice decisions made during simulation/game play to changes in problem behavior, and a rubric to evaluate change to dimensions of systems thinking using the simulation/game as an intervention tool. Findings from the investigation of integrating system dynamics modeling and wargaming indicate that a simulation/game can be used to inform criteria to evaluate sufficiency in capturing the complex system and messy problem space during game design in wargaming. Findings from rubric development and pre-test/post-test survey show statistically significant change to dimensions of systems thinking when using the simulation/game as an intervention tool which can be used to inform criteria for learning objective for game participants in wargame design. Implications include a better understanding by leaders for impacts due to their policy decisions by providing them learning objectives and tools to explore and manipulate the messy problem spaces they face with consistent repeatability. This research fills gaps in the literature by providing quantitative measures, repeatability, and linking policy decisions to wargame output and learning objectives for wargame participants. The limited scope of the Battle for Moscow wargame transferred to the developed simulation/game. Incorporating additional policy space to explore changes to system output may uncover additional counterintuitive behaviors and unintended consequences resulting in different ranges of solution sets. It is anticipated that any alternative solutions will be directly comparable to each other if model dimensions remain consistent with each other. Evaluation of responses to unguided open-ended questions during the survey was conducted by one investigator; multiple investigators conducting response evaluations would offer robustness to the process. Guided questions or response options may reduce the need for qualitative responses, though behavioral bias of participants should be considered when offering prompts. Further research could build on our results to develop best practices for design criteria and learning objectives of wargames to develop systems thinking leaders.

Creator