Enhanced Assessment Item Generation with Multimedia for Learning

Automatic item generation (AIG) is an emerging research topic in the domain of technology-based assessment. AIG has demonstrated extensive cost savings in item development. Automatic item generation has gained significant interest over the last years in order to overcome the adaptivity and scalability issues of the current item production model. Current approaches for AIG lack of using multimedia elements in test items, feedback elements are not generated and the generation process do not use external resources, like Open Educational Resources (OER) or educational models available on the Web. The goal of this project is to investigate the generation of assessment items with external content sources, including multimedia and educational resources in order to a) increase the diversity of test items, their authenticity, as well as their potential to attract language learners to use them, and b) to extend the generation process with a feedback element while reducing the effort and time to develop large item banks for different use cases. The new AIG approach will improve the process of variable creation in assessment items and in assessment item generation in particular by supporting the identification of semantic and multimedia variables rather than numeric and linguistic variables. Further, the AIG approach will adapt the classical three step generation process by incorporation the retrieval and reuse of multimedia elements as auxiliary information, as supporting or as contextual information in the assessment item. Finally, it mechanism are developed to retrieve and select educational resources to include them as feedback hence to support the formative assessment process. This includes OER, which are becoming widely available in a number of repositories. The new AIG is in-line with several key evolutions in the field of technology-enhanced learning, including the development of Open Educational Resources (OER) and the availability of personalized learning services, which include also computer adapted tests (CAT). Besides a qualitative evaluation of the multimedia items by assessment experts, several field studies with stakeholders will serve to detect potential quality issues and collect first psychometric properties in three scenarios: a) Language learning while shopping, b) Language learning while using public administrative online services or using services provided to tourists, c) language learning while visiting a museum.