Experiment Design and Verification for Assessing the Acquisition of Strategic Planning Ability

Exposing healthy subjects to the Tower of London (ToL) task several times enables the observation of the evolution of skill level over time as well as its reflections on eye gaze patterns. In order to identify such effects, one needs a carefully designed set of ToL tasks (i.e. problem sets). In particular, problem complexity should be gradually increasing within each problem set and comparable across different problem sets, as assessed in terms of objective measures, to enable smooth progress and the accumulation of experience. In that respect, this study uses goal hierarchy, search depth, and the number of optimal solutions for assessing complexity and establishing the gradual relation. Namely, two problem sets, Base-1 and Base-2, are designed. To satisfy the comparability condition, six color permutations are derived from Base-1 (experiments-1 to -6) and delivered to the subjects on two consecutive days. Finally, another problem set derived from Base-2 with the exact same structural properties is delivered on the last day (experiment-7). We conducted experiments with 15 participants and examined the evolution of their planning, trial, and execution times, as well as rule violations and the number of moves beyond optimal. The results suggest that there are indeed improvements in speed and accuracy in experiment-7 compared to experiment-1, even though both are novel to the participants. Additionally, these improvements are shown to be similar and consistent for different (initial) problem sets. Moreover, the proposed problem sets' difficulty and planning time for all experiments have a strong positive correlation, averaging 0.86 (σ=0.06). In that respect, we believe that the eye-gaze patterns also have the potential to withhold important information and are worth further investigation in future studies.