Comparing Video Based Shoulder Surfing with Live Simulation

We analyze the claims that video recreations of shoulder surfing attacks offer a suitable alternative and a baseline compared to eval- uating in a live setting by recreating a subset of the factors of an experiment originally conducted by Aviv et al. (ACSAC 2017) using live participants (n = 36) with both the victim and attacker present as opposed to participants viewing videos of a victim. The exper- iment confirmed that for the Android graphical patterns, video simulation was consistent with live settings for attacker success rates. However, both 4- and 6-digit PINs demonstrated statistically significant differences in attack performance with live attackers performing as much 1.9x better. The security benefits gained from removing feedback lines in Android’s graphical patterns are also greatly diminished in the live setting, particularly under multi- ple view conditions, but overall, video recreations can provide a baseline measure for attacker success rate as we never observed a significantly different result in the live setting that under-performed compared to video recreation. However, we caution that researchers should consider that these baselines may greatly underestimate the threat of an attacker in live settings.