---
_id: '12993'
abstract:
- lang: eng
text: In computer science and related technical fields, researchers, educators,
and practitioners are continuously automating recurring tasks for high efficiency
in a wide variety of fields. In higher education, such tasks that educators face
are the recurring review and assessment process of students' programming coursework.
Thus, various attempts exist to automate the assessment and feedback generation
for course homework and practicals in higher education. Those approaches for automated
programming task assessment often comprise running automated tests to check for
limited functional correctness and potentially style checking for various violations
(LINTing). Educators familiar with large-scale automated task assessment are likely
used to seeing hard-coded solutions specifically or accidentally designed to just
pass the required tests, ignoring or misinterpreting the actual task requirements.
Detecting such issues in arbitrary code is non-trivial and an ongoing research
topic in software engineering. Software engineering research has yielded various
semantic analysis frameworks, such as GitHub's CodeQL, which can be adapted for
programming task assessment. We present a work-in-progress programming task analysis
framework which employs CodeQL's analysis technology to identify the actual use
of task-description-mandated syntactic and semantic elements such as loop structures
or the use of mandated data blocks in branching conditions. This allows extending
existing course work analysis frameworks to include a semantic check of an uploaded
program which exceeds the relatively simple set of input-output test cases provided
by unit tests. We use a running example of entry level programming tasks and several
solution attempts to introduce and explain our proposed control flow and data
flow -based analysis method. We discuss the benefits of including semantic analysis
as an additional method in the automated programming task assessment toolbox.
Our main contribution is the adaptation of an semantic analysis code framework
to analyse syntactic and semantic components in students' programming coursework.
author:
- first_name: Leon
full_name: Wehmeier, Leon
id: '81257'
last_name: Wehmeier
- first_name: Sebastian
full_name: Eilermann, Sebastian
last_name: Eilermann
- first_name: Oliver
full_name: Niggemann, Oliver
id: '10876'
last_name: Niggemann
- first_name: Andreas
full_name: Deuter, Andreas
id: '62088'
last_name: Deuter
orcid: 0000-0002-6529-6215
citation:
ama: Wehmeier L, Eilermann S, Niggemann O, Deuter A. Task-Fidelity Assessment
for Programming Tasks Using Semantic Code Analysis. (IEEE ASEE Frontiers in
Education Conference, Institute of Electrical and Electronics Engineers, American
Society for Engineering Education, eds.). IEEE; 2024. doi:10.1109/fie58773.2023.10342916
apa: 'Wehmeier, L., Eilermann, S., Niggemann, O., & Deuter, A. (2024). Task-fidelity
Assessment for Programming Tasks Using Semantic Code Analysis. In IEEE ASEE Frontiers
in Education Conference, Institute of Electrical and Electronics Engineers, &
American Society for Engineering Education (Eds.), FIE 2023 : College Station,
TX, USA, October 18-21, 2023 : conference proceedings / 2023 IEEE Frontiers in
Education Conference (FIE). IEEE. https://doi.org/10.1109/fie58773.2023.10342916'
bjps: 'Wehmeier L et al. (2024) Task-Fidelity Assessment for Programming
Tasks Using Semantic Code Analysis, IEEE ASEE Frontiers in Education Conference,
Institute of Electrical and Electronics Engineers, and American Society for Engineering
Education (eds). [Piscataway, NJ]: IEEE.'
chicago: 'Wehmeier, Leon, Sebastian Eilermann, Oliver Niggemann, and Andreas Deuter.
Task-Fidelity Assessment for Programming Tasks Using Semantic Code Analysis.
Edited by IEEE ASEE Frontiers in Education Conference, Institute of Electrical
and Electronics Engineers, and American Society for Engineering Education. FIE
2023 : College Station, TX, USA, October 18-21, 2023 : Conference Proceedings
/ 2023 IEEE Frontiers in Education Conference (FIE). [Piscataway, NJ]: IEEE,
2024. https://doi.org/10.1109/fie58773.2023.10342916.'
chicago-de: 'Wehmeier, Leon, Sebastian Eilermann, Oliver Niggemann und Andreas Deuter.
2024. Task-fidelity Assessment for Programming Tasks Using Semantic Code Analysis.
Hg. von IEEE ASEE Frontiers in Education Conference, Institute of Electrical and
Electronics Engineers, und American Society for Engineering Education. FIE
2023 : College Station, TX, USA, October 18-21, 2023 : conference proceedings
/ 2023 IEEE Frontiers in Education Conference (FIE). [Piscataway, NJ]: IEEE.
doi:10.1109/fie58773.2023.10342916,
.'
din1505-2-1: 'Wehmeier, Leon ; Eilermann, Sebastian ; Niggemann,
Oliver ; Deuter, Andreas
; IEEE ASEE Frontiers in Education Conference
; Institute of Electrical and Electronics
Engineers ; American Society for
Engineering Education (Hrsg.): Task-fidelity Assessment for Programming
Tasks Using Semantic Code Analysis. [Piscataway, NJ] : IEEE, 2024'
havard: L. Wehmeier, S. Eilermann, O. Niggemann, A. Deuter, Task-fidelity Assessment
for Programming Tasks Using Semantic Code Analysis, IEEE, [Piscataway, NJ], 2024.
ieee: 'L. Wehmeier, S. Eilermann, O. Niggemann, and A. Deuter, Task-fidelity
Assessment for Programming Tasks Using Semantic Code Analysis. [Piscataway,
NJ]: IEEE, 2024. doi: 10.1109/fie58773.2023.10342916.'
mla: 'Wehmeier, Leon, et al. “Task-Fidelity Assessment for Programming Tasks Using
Semantic Code Analysis.” FIE 2023 : College Station, TX, USA, October 18-21,
2023 : Conference Proceedings / 2023 IEEE Frontiers in Education Conference (FIE),
edited by IEEE ASEE Frontiers in Education Conference et al., IEEE, 2024, https://doi.org/10.1109/fie58773.2023.10342916.'
short: L. Wehmeier, S. Eilermann, O. Niggemann, A. Deuter, Task-Fidelity Assessment
for Programming Tasks Using Semantic Code Analysis, IEEE, [Piscataway, NJ], 2024.
ufg: 'Wehmeier, Leon u. a.: Task-fidelity Assessment for Programming Tasks
Using Semantic Code Analysis, hg. von IEEE ASEE Frontiers in Education Conference/Institute
of Electrical and Electronics Engineers, American Society for Engineering Education,
[Piscataway, NJ] 2024.'
van: 'Wehmeier L, Eilermann S, Niggemann O, Deuter A. Task-fidelity Assessment for
Programming Tasks Using Semantic Code Analysis. IEEE ASEE Frontiers in Education
Conference, Institute of Electrical and Electronics Engineers, American Society
for Engineering Education, editors. FIE 2023 : College Station, TX, USA, October
18-21, 2023 : conference proceedings / 2023 IEEE Frontiers in Education Conference
(FIE). [Piscataway, NJ]: IEEE; 2024.'
conference:
end_date: 2023-10-21
location: Texas
name: 2023 IEEE Frontiers in Education Conference (FIE)
start_date: 2023-10-18
corporate_editor:
- IEEE ASEE Frontiers in Education Conference
- Institute of Electrical and Electronics Engineers
- American Society for Engineering Education
date_created: 2025-06-18T13:05:11Z
date_updated: 2025-06-18T13:23:56Z
department:
- _id: DEP7022
- _id: DEP1306
- _id: DEP7001
doi: 10.1109/fie58773.2023.10342916
keyword:
- Codes
- Electronic learning
- Soft sensors
- Semantics
- Education
- Syntactics
- Task analysis
language:
- iso: eng
place: '[Piscataway, NJ]'
publication: 'FIE 2023 : College Station, TX, USA, October 18-21, 2023 : conference
proceedings / 2023 IEEE Frontiers in Education Conference (FIE)'
publication_identifier:
eisbn:
- 979-8-3503-3642-9
isbn:
- 979-8-3503-3643-6
publication_status: published
publisher: IEEE
status: public
title: Task-fidelity Assessment for Programming Tasks Using Semantic Code Analysis
type: conference_editor_article
user_id: '83781'
year: '2024'
...