User perceptions of player substitution with Deep Player Behavior Models

Many online games suffer when players drop off due to lost connections or quitting prematurely, which leads to match terminations or game-play imbalances. While rule-based outcome evaluations or substitutions with bots are frequently used to mitigate such disruptions, these techniques are often perceived as unsatisfactory.

Deep learning methods have successfully been used in deep player behavior modelling (DPBM) to produce non-player characters or bots which show more complex behavior patterns than those modelled using traditional AI techniques.

Motivated by these findings, we present an investigation of the player-perceived awareness, believability and representativeness, when substituting disconnected players with DPBM agents in an online-multiplayer action game. Both quantitative and qualitative outcomes indicate that DPBM agent substitutes perform similarly to human players and that players were unable to detect substitutions. Notably, players were in fact able to detect substitution with agents driven by more traditional heuristics.

A purpose-built online multiplayer game online Korona:Nemesis, was released on Steam for this study.

Takeaways

Since unintentional, as well as deliberate disconnects, dropoffs or client terminations are unlikely to disappear with conventional, stability-improving hardware and software methods, we demonstrated an alternative approach that bridges (temporary) player absence by substituting them with Deep Player Behavior Models (DPBM).

An ecologically valid online field study (n = 312) with a duration of four weeks simulated the replacement of a human player in the online multi-player fighting platformer Korona:Nemesis, assessing the remaining players’ awareness, the believability of the substitution, and the performance-related representativeness.

We conclude that players were not able to distinguish between DPBM bots and original human players, but notably managed to detect bots based on heuristic behavior.

Perceived performance and predictability changes did not differ between players who did detect DPBM bots and players who indicated that they thought that they had been playing against other human players only.

All together, we implemented and evaluated a novel approach to tackling online match disruptions and lay ground for further evaluations spanning additional games, genres and integrations.