---
_id: '12048'
abstract:
- lang: eng
text: Interactive stories can be an effective approach for teaching purposes. One
shortcoming is the effort necessary to author and create these stories, especially
complex storylines with choices for the readers. Based on recent advances in Natural
Language Processing (NLP), new opportunities arise for assistance systems in the
context of interactive stories. In our work, we present an authoring approach
and prototypical tool for the creation of visual comic-strip like interactive
stories, a type of hypercomics, that integrate an Artificial Intelligence (AI)
assistance. Such comics are already used in our Gekonnt hanDeln web platform.
The AI assistance provides suggestions for the overall story outline as well as
how to design and write individual story frames. We provide a detailed description
about the approach and its prototypical implementation. Furthermore, we present
a study evaluating the prototype with student groups and how the prototype evolved
in an iterative style based on the students’ feedback.
author:
- first_name: Valentin
full_name: Grimm, Valentin
id: '74000'
last_name: Grimm
- first_name: Jessica
full_name: Rubart, Jessica
id: '45672'
last_name: Rubart
citation:
ama: Grimm V, Rubart J. Authoring Educational Hypercomics Assisted by Large Language
Models. Association for Computing Machinery (ACM); 2024:88-97. doi:10.1145/3648188.3675124
apa: 'Grimm, V., & Rubart, J. (2024). Authoring Educational Hypercomics assisted
by Large Language Models. In HT ’24: Proceedings of the 35th ACM Conference
on Hypertext and Social Media (pp. 88–97). Association for Computing Machinery
(ACM). https://doi.org/10.1145/3648188.3675124'
bjps: 'Grimm V and Rubart J (2024) Authoring Educational Hypercomics Assisted
by Large Language Models. New York, USA: Association for Computing Machinery
(ACM).'
chicago: 'Grimm, Valentin, and Jessica Rubart. Authoring Educational Hypercomics
Assisted by Large Language Models. HT ’24: Proceedings of the 35th ACM
Conference on Hypertext and Social Media. New York, USA: Association for Computing
Machinery (ACM), 2024. https://doi.org/10.1145/3648188.3675124.'
chicago-de: 'Grimm, Valentin und Jessica Rubart. 2024. Authoring Educational
Hypercomics assisted by Large Language Models. HT ’24: Proceedings of the
35th ACM Conference on Hypertext and Social Media. New York, USA: Association
for Computing Machinery (ACM). doi:10.1145/3648188.3675124,
.'
din1505-2-1: 'Grimm, Valentin ; Rubart, Jessica: Authoring Educational
Hypercomics assisted by Large Language Models. New York, USA : Association
for Computing Machinery (ACM), 2024'
havard: V. Grimm, J. Rubart, Authoring Educational Hypercomics assisted by Large
Language Models, Association for Computing Machinery (ACM), New York, USA, 2024.
ieee: 'V. Grimm and J. Rubart, Authoring Educational Hypercomics assisted by
Large Language Models. New York, USA: Association for Computing Machinery
(ACM), 2024, pp. 88–97. doi: 10.1145/3648188.3675124.'
mla: 'Grimm, Valentin, and Jessica Rubart. “Authoring Educational Hypercomics Assisted
by Large Language Models.” HT ’24: Proceedings of the 35th ACM Conference on
Hypertext and Social Media, Association for Computing Machinery (ACM), 2024,
pp. 88–97, https://doi.org/10.1145/3648188.3675124.'
short: V. Grimm, J. Rubart, Authoring Educational Hypercomics Assisted by Large
Language Models, Association for Computing Machinery (ACM), New York, USA, 2024.
ufg: 'Grimm, Valentin/Rubart, Jessica: Authoring Educational Hypercomics
assisted by Large Language Models, New York, USA 2024.'
van: 'Grimm V, Rubart J. Authoring Educational Hypercomics assisted by Large Language
Models. HT ’24: Proceedings of the 35th ACM Conference on Hypertext and Social
Media. New York, USA: Association for Computing Machinery (ACM); 2024.'
conference:
end_date: 2024-09-13
location: Poznan, Poland
name: 35th ACM Conference on Hypertext and Social Media
start_date: 2024-09-10
date_created: 2024-11-22T12:25:29Z
date_updated: 2025-06-25T13:06:37Z
department:
- _id: DEP8008
- _id: DEP8008
doi: 10.1145/3648188.3675124
keyword:
- Storytelling
- Authoring
- GPT
- Hypercomics
- Large Language Models
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://dl.acm.org/doi/10.1145/3648188.3675124
oa: '1'
page: 88-97
place: New York, USA
publication: 'HT ''24: Proceedings of the 35th ACM Conference on Hypertext and Social
Media'
publication_identifier:
isbn:
- '979-8-4007-0595-3 '
publication_status: published
publisher: Association for Computing Machinery (ACM)
quality_controlled: '1'
status: public
title: Authoring Educational Hypercomics assisted by Large Language Models
type: conference_editor_article
user_id: '83781'
year: '2024'
...
---
_id: '4094'
abstract:
- lang: eng
text: Projection-based assitive systems that guide users through assembly work are
on their way to industrial application. Previous research work investigated how
people can be supported with such systems. However, there has been little work
on the question on how to generate and author sequential instructions for assitive
systems. In this paper, we present a new concept and a prototypical implementation
of an assitive system that can be taught by demonstrating an assembly process.
By using a combination of RGB and depth cameras, we can generate an assembly instruction
of Lego Duplo bricks based on the demonstration of a user. This generated manual
can later on be used for assisting other users in the assembly process. By our
prototype system, we show the technological feasibility of assistive systems that
can learn from users.
author:
- first_name: Sebastian
full_name: Büttner, Sebastian
id: '61868'
last_name: Büttner
- first_name: Andreas
full_name: Peda, Andreas
id: '51144'
last_name: Peda
- first_name: Mario
full_name: Heinz, Mario
id: '68913'
last_name: Heinz
- first_name: Carsten
full_name: Röcker, Carsten
id: '61525'
last_name: Röcker
citation:
ama: 'Büttner S, Peda A, Heinz M, Röcker C. Teaching by Demonstrating – How Smart
Assistive Systems Can Learn from Users. In: 22nd International Conference on
Human-Computer Interaction. Vol 12203. Lecture Notes in Computer Science.
Springer; 2020:153-163. doi:https://doi.org/10.1007/978-3-030-50344-4_12'
apa: Büttner, S., Peda, A., Heinz, M., & Röcker, C. (2020). Teaching by Demonstrating
– How Smart Assistive Systems Can Learn from Users. 22nd International Conference
on Human-Computer Interaction, 12203, 153–163. https://doi.org/10.1007/978-3-030-50344-4_12
bjps: 'Büttner S et al. (2020) Teaching by Demonstrating – How Smart
Assistive Systems Can Learn from Users. 22nd International Conference on Human-Computer
Interaction, vol. 12203. Berlin: Springer, pp. 153–163.'
chicago: 'Büttner, Sebastian, Andreas Peda, Mario Heinz, and Carsten Röcker. “Teaching
by Demonstrating – How Smart Assistive Systems Can Learn from Users.” In 22nd
International Conference on Human-Computer Interaction, 12203:153–63. Lecture
Notes in Computer Science. Berlin: Springer, 2020. https://doi.org/10.1007/978-3-030-50344-4_12.'
chicago-de: 'Büttner, Sebastian, Andreas Peda, Mario Heinz und Carsten Röcker. 2020.
Teaching by Demonstrating – How Smart Assistive Systems Can Learn from Users.
In: 22nd International Conference on Human-Computer Interaction, 12203:153–163.
Lecture Notes in Computer Science. Berlin: Springer. doi:https://doi.org/10.1007/978-3-030-50344-4_12,
.'
din1505-2-1: 'Büttner, Sebastian ;
Peda, Andreas ; Heinz,
Mario ; Röcker, Carsten:
Teaching by Demonstrating – How Smart Assistive Systems Can Learn from Users.
In: 22nd International Conference on Human-Computer Interaction, Lecture
Notes in Computer Science. Bd. 12203. Berlin : Springer, 2020, S. 153–163'
havard: 'S. Büttner, A. Peda, M. Heinz, C. Röcker, Teaching by Demonstrating – How
Smart Assistive Systems Can Learn from Users, in: 22nd International Conference
on Human-Computer Interaction, Springer, Berlin, 2020: pp. 153–163.'
ieee: 'S. Büttner, A. Peda, M. Heinz, and C. Röcker, “Teaching by Demonstrating
– How Smart Assistive Systems Can Learn from Users,” in 22nd International
Conference on Human-Computer Interaction, Copenhagen, Denmark, 2020, vol.
12203, pp. 153–163. doi: https://doi.org/10.1007/978-3-030-50344-4_12.'
mla: Büttner, Sebastian, et al. “Teaching by Demonstrating – How Smart Assistive
Systems Can Learn from Users.” 22nd International Conference on Human-Computer
Interaction, vol. 12203, Springer, 2020, pp. 153–63, https://doi.org/10.1007/978-3-030-50344-4_12.
short: 'S. Büttner, A. Peda, M. Heinz, C. Röcker, in: 22nd International Conference
on Human-Computer Interaction, Springer, Berlin, 2020, pp. 153–163.'
ufg: 'Büttner, Sebastian u. a.: Teaching by Demonstrating – How Smart Assistive
Systems Can Learn from Users, in: o. Hg.: 22nd International Conference on Human-Computer
Interaction, Bd. 12203, Berlin 2020 (Lecture Notes in Computer Science), S. 153–163.'
van: 'Büttner S, Peda A, Heinz M, Röcker C. Teaching by Demonstrating – How Smart
Assistive Systems Can Learn from Users. In: 22nd International Conference on Human-Computer
Interaction. Berlin: Springer; 2020. p. 153–63. (Lecture Notes in Computer Science;
vol. 12203).'
conference:
end_date: 2020-07-24
location: Copenhagen, Denmark
name: 22nd International Conference on Human-Computer Interaction
start_date: 2020-07-19
date_created: 2020-11-26T14:10:00Z
date_updated: 2025-06-26T08:59:53Z
department:
- _id: DEP5023
doi: https://doi.org/10.1007/978-3-030-50344-4_12
intvolume: ' 12203'
keyword:
- Assitive system
- Authoring
- Instruction generation
- Computer vision
- Teaching by demonstration
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://link.springer.com/chapter/10.1007/978-3-030-50344-4_12
oa: '1'
page: 153-163
place: Berlin
publication: 22nd International Conference on Human-Computer Interaction
publication_identifier:
eisbn:
- 978-3-030-50344-4
isbn:
- 978-3-030-50343-7
publication_status: published
publisher: Springer
series_title: Lecture Notes in Computer Science
status: public
title: Teaching by Demonstrating – How Smart Assistive Systems Can Learn from Users
type: conference
user_id: '83781'
volume: 12203
year: '2020'
...