VocabulARy 2.0 is a generative augmented reality system that supports vocabulary learning by enabling learners to create personalised 3D mnemonics anchored to objects in their surroundings. The system is developed by Karolina Trajkovska as part of her doctoral research at HICUP Lab, under the supervision of Matjaž Kljun, Maheshya Weerasinghe and Klen Čopič Pucihar. The work appears in the IEEE ISMAR 2026 Adjunct Proceedings.
Demo Video: Watch the demo on YouTube
Abstract:
Mnemonics support vocabulary learning by linking unfamiliar words to familiar concepts through memorable verbal and visual associations. We present a generative AR system that enables learners to create personalised 3D mnemonics by transforming familiar objects in their surroundings into situated mnemonic anchors. The system scans the environment, detects visible objects, and generates bilingual flashcards. After selecting a word, the learner describes a personal association. The system combines this description with an image of the selected object to generate a textured 3D model using generative AI. Learners can then place the model relative to the real object, transforming it into a situated mnemonic anchor.
How It Works:
VocabulARy 2.0 follows a four-stage workflow running on a Meta Quest 3 headset, Apple Vision Pro, and Android tablets:
- Scan the scene. The headset captures a frame of the learner’s surroundings and sends it to the server. Gemini 3.5 Flash identifies visible objects and translates their names into the target language. Grounding DINO verifies each detection. The result is a set of bilingual flashcards displayed in AR above the workspace.
- Select a word. The learner grabs a flashcard for the word they want to learn.
- Describe a mnemonic. The learner types a personal association (e.g. a phonetic link or a visual image) and scans the real object. Gemini 3 Pro Image integrates the mnemonic into the object photo, isolates the subject, and passes the result to TRELLIS.2, which generates a textured 3D mesh (.glb).
- Place and anchor. The generated model is downloaded into the Unity scene. The learner resizes, repositions, and aligns it with the real object — turning it into a situated mnemonic anchor they can return to.
The backend runs on a local workstation with two NVIDIA RTX PRO 6000 Blackwell GPUs (96 GB VRAM each) and a 32-core AMD Ryzen Threadripper PRO 7975WX.
Publications:
- Trajkovska, K., Kljun, M., Weerasinghe, M., & Pucihar, K. C. (2026). Personalised 3D Mnemonics for Vocabulary Learning in Context. In 2026 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMARAdjunct). PDF (arXiv)
- Trajkovska, K., Kljun, M., Weerasinghe, M., & Pucihar, K. C. (2026). VocabulARy 2.0: A generative AR system enabling personalised 3D mnemonics for vocabulary learning in context. In 2026 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMARAdjunct)
(Links to the papers will be added here once the conference proceedings become publicly available.)
Citation:
If you use or build on this work, please cite both contributions:
@inproceedings{trajkovska2026vocabularyPoster,
title = {Personalised {3D} Mnemonics for Vocabulary Learning in Context},
author = {Trajkovska, Karolina and Kljun, Matja{\v{z}} and Weerasinghe, Maheshya and {\v{C}}opi{\v{c}} Pucihar, Klen},
booktitle = {2026 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
year = {2026},
publisher = {IEEE}
}@inproceedings{trajkovska2026vocabularyDemo,
title = {{VocabulARy} 2.0: A Generative {AR} System Enabling Personalised {3D} Mnemonics for Vocabulary Learning in Context},
author = {Trajkovska, Karolina and Kljun, Matja{\v{z}} and Weerasinghe, Maheshya and {\v{C}}opi{\v{c}} Pucihar, Klen},
booktitle = {2026 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
year = {2026},
publisher = {IEEE}
}Follow HICUP Lab in its official channels for more updates:
For feedback regarding our media content, you may contact our Social media and web manager.