Evaluating narrative-driven movie recommendations on reddit

Lukas Eberhard, Lisa Posch, Simon Walk, Denis Helic

Research output: Contribution to conferencePaperResearchpeer-review

Abstract

Recommender systems have become omni-present tools that are used by a wide variety of users in everyday life tasks, such as finding products in Web stores or online movie streaming portals. However, in situations where users already have an idea of what they are looking for (e.g., 'The Lord of the Rings', but in space with a dark vibe), most traditional recommender algorithms struggle to adequately address such a priori defined requirements. Therefore, users have built dedicated discussion boards to ask peers for suggestions, which ideally fulfill the stated requirements. In this paper, we set out to determine the utility of well-established recommender algorithms for calculating recommendations when provided with such a narrative. To that end, we first crowdsource a reference evaluation dataset from human movie suggestions. We use this dataset to evaluate the potential of five recommendation algorithms for incorporating such a narrative into their recommendations. Further, we make the dataset available for other researchers to advance the state of research in the field of narrative-driven recommendations. Finally, we use our evaluation dataset to improve not only our algorithmic recommendations, but also existing empirical recommendations of IMDb. Our findings suggest that the implemented recommender algorithms yield vastly different suggestions than humans when presented with the same a priori requirements. However, with carefully configured post-filtering techniques, we can outperform the baseline by up to 100%. This represents an important first step towards more refined algorithmic narrative-driven recommendations.

Original languageEnglish
Pages1-11
Number of pages11
DOIs
Publication statusPublished - 1 Jan 2019
Event24th ACM International Conference on Intelligent User Interfaces, IUI 2019 - Marina del Ray, United States
Duration: 17 Mar 201920 Mar 2019

Conference

Conference24th ACM International Conference on Intelligent User Interfaces, IUI 2019
CountryUnited States
CityMarina del Ray
Period17/03/1920/03/19

Fingerprint

Recommender systems
World Wide Web

Keywords

  • Crowdsourcing
  • Dataset
  • Narrative-driven recommendations

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction

Cite this

Eberhard, L., Posch, L., Walk, S., & Helic, D. (2019). Evaluating narrative-driven movie recommendations on reddit. 1-11. Paper presented at 24th ACM International Conference on Intelligent User Interfaces, IUI 2019, Marina del Ray, United States. https://doi.org/10.1145/3301275.3302287

Evaluating narrative-driven movie recommendations on reddit. / Eberhard, Lukas; Posch, Lisa; Walk, Simon; Helic, Denis.

2019. 1-11 Paper presented at 24th ACM International Conference on Intelligent User Interfaces, IUI 2019, Marina del Ray, United States.

Research output: Contribution to conferencePaperResearchpeer-review

Eberhard, L, Posch, L, Walk, S & Helic, D 2019, 'Evaluating narrative-driven movie recommendations on reddit' Paper presented at 24th ACM International Conference on Intelligent User Interfaces, IUI 2019, Marina del Ray, United States, 17/03/19 - 20/03/19, pp. 1-11. https://doi.org/10.1145/3301275.3302287
Eberhard L, Posch L, Walk S, Helic D. Evaluating narrative-driven movie recommendations on reddit. 2019. Paper presented at 24th ACM International Conference on Intelligent User Interfaces, IUI 2019, Marina del Ray, United States. https://doi.org/10.1145/3301275.3302287
Eberhard, Lukas ; Posch, Lisa ; Walk, Simon ; Helic, Denis. / Evaluating narrative-driven movie recommendations on reddit. Paper presented at 24th ACM International Conference on Intelligent User Interfaces, IUI 2019, Marina del Ray, United States.11 p.
@conference{202fbbf0267b408daf236d07b177b363,
title = "Evaluating narrative-driven movie recommendations on reddit",
abstract = "Recommender systems have become omni-present tools that are used by a wide variety of users in everyday life tasks, such as finding products in Web stores or online movie streaming portals. However, in situations where users already have an idea of what they are looking for (e.g., 'The Lord of the Rings', but in space with a dark vibe), most traditional recommender algorithms struggle to adequately address such a priori defined requirements. Therefore, users have built dedicated discussion boards to ask peers for suggestions, which ideally fulfill the stated requirements. In this paper, we set out to determine the utility of well-established recommender algorithms for calculating recommendations when provided with such a narrative. To that end, we first crowdsource a reference evaluation dataset from human movie suggestions. We use this dataset to evaluate the potential of five recommendation algorithms for incorporating such a narrative into their recommendations. Further, we make the dataset available for other researchers to advance the state of research in the field of narrative-driven recommendations. Finally, we use our evaluation dataset to improve not only our algorithmic recommendations, but also existing empirical recommendations of IMDb. Our findings suggest that the implemented recommender algorithms yield vastly different suggestions than humans when presented with the same a priori requirements. However, with carefully configured post-filtering techniques, we can outperform the baseline by up to 100{\%}. This represents an important first step towards more refined algorithmic narrative-driven recommendations.",
keywords = "Crowdsourcing, Dataset, Narrative-driven recommendations",
author = "Lukas Eberhard and Lisa Posch and Simon Walk and Denis Helic",
year = "2019",
month = "1",
day = "1",
doi = "10.1145/3301275.3302287",
language = "English",
pages = "1--11",
note = "24th ACM International Conference on Intelligent User Interfaces, IUI 2019 ; Conference date: 17-03-2019 Through 20-03-2019",

}

TY - CONF

T1 - Evaluating narrative-driven movie recommendations on reddit

AU - Eberhard, Lukas

AU - Posch, Lisa

AU - Walk, Simon

AU - Helic, Denis

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Recommender systems have become omni-present tools that are used by a wide variety of users in everyday life tasks, such as finding products in Web stores or online movie streaming portals. However, in situations where users already have an idea of what they are looking for (e.g., 'The Lord of the Rings', but in space with a dark vibe), most traditional recommender algorithms struggle to adequately address such a priori defined requirements. Therefore, users have built dedicated discussion boards to ask peers for suggestions, which ideally fulfill the stated requirements. In this paper, we set out to determine the utility of well-established recommender algorithms for calculating recommendations when provided with such a narrative. To that end, we first crowdsource a reference evaluation dataset from human movie suggestions. We use this dataset to evaluate the potential of five recommendation algorithms for incorporating such a narrative into their recommendations. Further, we make the dataset available for other researchers to advance the state of research in the field of narrative-driven recommendations. Finally, we use our evaluation dataset to improve not only our algorithmic recommendations, but also existing empirical recommendations of IMDb. Our findings suggest that the implemented recommender algorithms yield vastly different suggestions than humans when presented with the same a priori requirements. However, with carefully configured post-filtering techniques, we can outperform the baseline by up to 100%. This represents an important first step towards more refined algorithmic narrative-driven recommendations.

AB - Recommender systems have become omni-present tools that are used by a wide variety of users in everyday life tasks, such as finding products in Web stores or online movie streaming portals. However, in situations where users already have an idea of what they are looking for (e.g., 'The Lord of the Rings', but in space with a dark vibe), most traditional recommender algorithms struggle to adequately address such a priori defined requirements. Therefore, users have built dedicated discussion boards to ask peers for suggestions, which ideally fulfill the stated requirements. In this paper, we set out to determine the utility of well-established recommender algorithms for calculating recommendations when provided with such a narrative. To that end, we first crowdsource a reference evaluation dataset from human movie suggestions. We use this dataset to evaluate the potential of five recommendation algorithms for incorporating such a narrative into their recommendations. Further, we make the dataset available for other researchers to advance the state of research in the field of narrative-driven recommendations. Finally, we use our evaluation dataset to improve not only our algorithmic recommendations, but also existing empirical recommendations of IMDb. Our findings suggest that the implemented recommender algorithms yield vastly different suggestions than humans when presented with the same a priori requirements. However, with carefully configured post-filtering techniques, we can outperform the baseline by up to 100%. This represents an important first step towards more refined algorithmic narrative-driven recommendations.

KW - Crowdsourcing

KW - Dataset

KW - Narrative-driven recommendations

UR - http://www.scopus.com/inward/record.url?scp=85065576517&partnerID=8YFLogxK

U2 - 10.1145/3301275.3302287

DO - 10.1145/3301275.3302287

M3 - Paper

SP - 1

EP - 11

ER -