In the latest article I published at Frontiers in Computer Science, titled Validation of User Preferences and Effects of Personalized Gamification on Task Performance, I studied how users would react to being able to customize their gameful experience. Additionally, the study also showed that task performance improved for the customizable application in comparison to the generic one, with some caveats.
In the study with 252 participants, we asked them to classify between 50 and 100 images by typing as many tags as they could think of for the image (e.g., balloon, sky, clouds, blue, etc.).
The digital application featured several gameful elements: points, levels, challenges, badges, progress feedback, power-ups, random rewards, unlockable content (avatars), leaderboards, and moderating role. Half of the participants were assigned to use a generic version of the app, in which all of the game elements were turned on. The other half used a customizable version of the app. In this version, they could see a list of the game elements and select which ones they wanted to use and which ones they did not want.
Selection of Gameful Design Elements
In general, participants who used the customizable version enjoyed the possibility of controlling their own gameful experience. Some examples of comments from participants are “I felt like I had control and like what I was doing mattered”, “It was interesting because not many games allow you to do this”, “It’s a good idea, everyone can choose what they prefer, so every [sic] can play and be motivated with something they are interesting [sic] in.”
Additionally, we compared participants’ Hexad user type scores with their actual selection. Logically, we would expect a correlation. For example, participants with high Achiever scores should select elements like Challenges, Badges, etc.; participants with high Socialiser scores should select Leaderboards, etc.
In practice, some of these correlations were observed, but not all of them. Thus, it seems that the user type scores indeed have some influence on what elements people want to use if given the choice. But this correspondence is not perfect, i.e., some of the selections we observed could not be explained only by the user type scores. So, there are also other factors that we still don’t know, which also influence what game elements people select when customizing their gameful experience.
I suspect that some of these differences can be explained by the context of the application. The Hexad user types scale asks people about their preferences in general. This shows each person’s general tendencies. However, these tendencies may be affected by the reality of each system. For example, even if a person generally likes leaderboards, it does not mean that she will always want to use leaderboards. There may be a particular app for which said person does not feel motivated by the leaderboard because of the context. I believe that this is what happened in this case. In our app, the leaderboards were not very meaningful because participants did not know each other and the interaction was rather simple. So, a leaderboard in this context may not be as motivating as a leaderboard in a competitive context.
Task Performance
When comparing measures of task performance such as the number of images tagged, the number of tags, and the time spent in the platform, we observed significant differences between the generic and customizable versions. Overall, participants tagged more images in the customizable version than the generic one. However, they did not spend more time on the platform. So the increase in the number of images tagged was apparently achieved by tagging images a bit faster and writing a little fewer tags per image.
In our application, the number of images tagged was the metric that led to better rewards. The base number of points earned by participants was 10 points per image, plus one additional point per tag (so, for example, writing three tags for an image would yield 13 points: 10 base points for the image, plus three points for the three tags). Thus, participants should have quickly learned that they could maximize their scores by tagging more images instead of spending additional time writing more tags for the same image.
Therefore, the caveat revealed in this study is that participant performance did not increase for all the metrics. Instead, performance increase for the metric that was more rewarding in the gameful experience. Well, this sounds logical. When you gamify something, users will pay more attention to the metrics and figure out which tasks offer the biggest rewards. So, the design of a gameful application needs to be carefully balanced to incentivize users to carry out the right tasks. In the example of our study, if the goal of the design was to have users tagging as many images as they can, then our design worked perfectly. If we instead wanted users to write more tags for each image, we could for example decrease the number of points given for each image tagged and increase the number of points given for each additional tag per image.
What does this mean for gameful design?
There are several takeaways from this study to gameful design. First, it seems that at least in the correct context, users enjoy selecting which game elements they will use for their gameful experience. Therefore, this can be an easy and inexpensive way to implement a personalized gameful system.
Second, we showed that the Hexad user type scores are somewhat related to what game element users select if given the choice. So, the user types can be used as an aid in the design and personalization. If we let users select the elements for their experience, that’s fine. However, if someone wants to build a system that automatically tries to identify which game elements are going to be more engaging for each user, it may not be enough to use just user type scores.
Finally, allowing users to personalize or customize their gameful experience may lead to increased performance. I believe that this happens because users are more engaged with the gameful experience if they participated in its design, whereas it is easier to ignore the game elements if they are just there. Users are more invested if you give them choices. On the other hand, the performance increase of personalized gamification will probably be concentrated on the tasks that are better rewarded. Therefore, designers need to carefully balance the game elements and rewards to incentivize a performance increase for the tasks that most need to be improved.
Original Article: Gustavo F. Tondello and Lennart E. Nacke. Validation of User Preferences and Effects of Personalized Gamification on Task Performance. Frontiers in Computer Science, 2, pp. 29, 2020. DOI:10.3389/fcomp.2020.00029
And if you’re interested in utilizing the application used in this study as an inspiration, it is also available on GitHub.
