Eye movements are like gestures in the creation of informal algorithms
- by Sunny Khemlani
- in News
- posted May 7, 2024
Robert Mackiewicz led a project (with Monica Bucciarelli, myself, and Phil Johnson-Laird) on how people comprehend informal programs, like the pseudo-program you might use to set a dinner table (all the placemats first, all the dining plates second, and so on). Our previous work shows that people construct kinematic mental simulations to create such algorithms. But Mackiewicz examined this idea further in two studies on eye movements, now published in the proceedings of CogSci 2024. Here’s the abstract:
People who have no experience with programming can create informal programs to rearrange the order of cars in trains. To find out whether they rely on kinematic mental simulations, the current studies examined participants’ eye movements in two experiments in which participants performed various moves and rearrangements on a railway consisting of a main track running from left to right and a siding entered from and exited to the left track. In Experiment 1, they had to imagine different sorts of single moves of cars on the railway. The sequences of their fixations resembled iconic gestures: they tended to look at the starting location of the imagined move, and then at its final location. In Experiment 2, the task was to create descriptions of how to solve four sorts of rearrangements that differ in their Kolmogorov complexity. It predicted the time to find the correct solution and the relative number and duration of fixations recorded during the description of each move for rearrangements of different complexity. Participants were more likely to fixate on the symbols on the cars than anything else, and they fixated longer when the rearrangement was more difficult. They also tended to fixate regions of the tracks where a car’s movement began or ended, as if they were imagining a car moving along the tracks. The results suggest that humans rely on a kinematic mental simulation when creating informal algorithms.