---
_id: '12817'
abstract:
- lang: eng
text: Sub-optimal control policies in intersection traffic signal controllers (TSC)
contribute to congestion and lead to negative effects on human health and the
environment. Reinforcement learning (RL) for traffic signal control is a promising
approach to design better control policies and has attracted considerable research
interest in recent years. However, most work done in this area used simplified
simulation environments of traffic scenarios to train RL-based TSC. To deploy
RL in real-world traffic systems, the gap between simplified simulation environments
and real-world applications has to be closed. Therefore, we propose LemgoRL, a
benchmark tool to train RL agents as TSC in a realistic simulation environment
of Lemgo, a medium-sized town in Germany. In addition to the realistic simulation
model, LemgoRL encompasses a traffic signal logic unit that ensures compliance
with all regulatory and safety requirements. LemgoRL offers the same interface
as the well-known OpenAI gym toolkit to enable easy deployment in existing research
work. To demonstrate the functionality and applicability of LemgoRL, we train
a state-of-the-art Deep RL algorithm on a CPU cluster utilizing a framework for
distributed and parallel RL and compare its performance with other methods. Our
benchmark tool drives the development of RL algorithms towards real-world applications.
author:
- first_name: Arthur
full_name: Müller, Arthur
last_name: Müller
- first_name: Vishal
full_name: Rangras, Vishal
id: '76044'
last_name: Rangras
- first_name: Tobias
full_name: Ferfers, Tobias
last_name: Ferfers
- first_name: Florian
full_name: Hufen, Florian
last_name: Hufen
- first_name: Lukas
full_name: Schreckenberg, Lukas
last_name: Schreckenberg
- first_name: Jürgen
full_name: Jasperneite, Jürgen
id: '1899'
last_name: Jasperneite
- first_name: Georg
full_name: Schnittker, Georg
last_name: Schnittker
- first_name: Michael
full_name: Waldmann, Michael
last_name: Waldmann
- first_name: Maxim
full_name: Friesen, Maxim
id: '61517'
last_name: Friesen
- first_name: Marco
full_name: Wiering, Marco
last_name: Wiering
citation:
ama: Müller A, Rangras V, Ferfers T, et al. Towards Real-World Deployment of
Reinforcement Learning for Traffic Signal Control. (Wani MA, Sethi I, Shi
W, et al., eds.). IEEE; 2022:507-514. doi:10.1109/icmla52953.2021.00085
apa: Müller, A., Rangras, V., Ferfers, T., Hufen, F., Schreckenberg, L., Jasperneite,
J., Schnittker, G., Waldmann, M., Friesen, M., & Wiering, M. (2022). Towards
Real-World Deployment of Reinforcement Learning for Traffic Signal Control. In
M. A. Wani, I. Sethi, W. Shi, G. Qu, D. Stan Raicu, R. Jin, IEEE ICMLA , &
Institute of Electrical and Electronics Engineers (Eds.), 20th IEEE International
Conference on Machine Learning and Applications (ICMLA) (pp. 507–514). IEEE.
https://doi.org/10.1109/icmla52953.2021.00085
bjps: 'Müller A et al. (2022) Towards Real-World Deployment of
Reinforcement Learning for Traffic Signal Control, Wani MA et al. (eds). [Piscataway,
NJ]: IEEE.'
chicago: 'Müller, Arthur, Vishal Rangras, Tobias Ferfers, Florian Hufen, Lukas Schreckenberg,
Jürgen Jasperneite, Georg Schnittker, Michael Waldmann, Maxim Friesen, and Marco
Wiering. Towards Real-World Deployment of Reinforcement Learning for Traffic
Signal Control. Edited by M. Arif Wani, Ishwar Sethi, Weisong Shi, Guangzhi Qu,
Daniela Stan Raicu, Ruoming Jin, IEEE ICMLA , and Institute of Electrical and
Electronics Engineers. 20th IEEE International Conference on Machine Learning
and Applications (ICMLA). [Piscataway, NJ]: IEEE, 2022. https://doi.org/10.1109/icmla52953.2021.00085.'
chicago-de: 'Müller, Arthur, Vishal Rangras, Tobias Ferfers, Florian Hufen, Lukas
Schreckenberg, Jürgen Jasperneite, Georg Schnittker, Michael Waldmann, Maxim Friesen
und Marco Wiering. 2022. Towards Real-World Deployment of Reinforcement Learning
for Traffic Signal Control. Hg. von M. Arif Wani, Ishwar Sethi, Weisong Shi,
Guangzhi Qu, Daniela Stan Raicu, Ruoming Jin, IEEE ICMLA , und Institute of
Electrical and Electronics Engineers. 20th IEEE International Conference on
Machine Learning and Applications (ICMLA). [Piscataway, NJ]: IEEE. doi:10.1109/icmla52953.2021.00085,
.'
din1505-2-1: 'Müller,
Arthur ; Rangras, Vishal
; Ferfers, Tobias ; Hufen,
Florian ; Schreckenberg, Lukas
; Jasperneite, Jürgen ; Schnittker,
Georg ; Waldmann, Michael
; u. a. ; Wani, M. A. ; Sethi, I. ;
Shi, W. ; Qu, G. ; Stan Raicu, D. ; Jin,
R. ; IEEE ICMLA ; Institute of Electrical and Electronics Engineers
(Hrsg.): Towards Real-World Deployment of Reinforcement Learning for Traffic
Signal Control. [Piscataway, NJ] : IEEE, 2022'
havard: A. Müller, V. Rangras, T. Ferfers, F. Hufen, L. Schreckenberg, J. Jasperneite,
G. Schnittker, M. Waldmann, M. Friesen, M. Wiering, Towards Real-World Deployment
of Reinforcement Learning for Traffic Signal Control, IEEE, [Piscataway, NJ],
2022.
ieee: 'A. Müller et al., Towards Real-World Deployment of Reinforcement
Learning for Traffic Signal Control. [Piscataway, NJ]: IEEE, 2022, pp. 507–514.
doi: 10.1109/icmla52953.2021.00085.'
mla: Müller, Arthur, et al. “Towards Real-World Deployment of Reinforcement Learning
for Traffic Signal Control.” 20th IEEE International Conference on Machine
Learning and Applications (ICMLA), edited by M. Arif Wani et al., IEEE, 2022,
pp. 507–14, https://doi.org/10.1109/icmla52953.2021.00085.
short: A. Müller, V. Rangras, T. Ferfers, F. Hufen, L. Schreckenberg, J. Jasperneite,
G. Schnittker, M. Waldmann, M. Friesen, M. Wiering, Towards Real-World Deployment
of Reinforcement Learning for Traffic Signal Control, IEEE, [Piscataway, NJ],
2022.
ufg: 'Müller, Arthur u. a.: Towards Real-World Deployment of Reinforcement
Learning for Traffic Signal Control, hg. von Wani, M. Arif u. a., [Piscataway,
NJ] 2022.'
van: 'Müller A, Rangras V, Ferfers T, Hufen F, Schreckenberg L, Jasperneite J, et
al. Towards Real-World Deployment of Reinforcement Learning for Traffic Signal
Control. Wani MA, Sethi I, Shi W, Qu G, Stan Raicu D, Jin R, et al., editors.
20th IEEE International Conference on Machine Learning and Applications (ICMLA).
[Piscataway, NJ]: IEEE; 2022.'
conference:
end_date: 2021-12-16
location: Online
name: 20th IEEE International Conference on Machine Learning and Applications (ICMLA)
start_date: 2021-12-13
corporate_editor:
- ' IEEE ICMLA '
- Institute of Electrical and Electronics Engineers
date_created: 2025-04-17T08:45:40Z
date_updated: 2025-06-26T13:28:21Z
department:
- _id: DEP5023
doi: 10.1109/icmla52953.2021.00085
editor:
- first_name: 'M. Arif '
full_name: 'Wani, M. Arif '
last_name: Wani
- first_name: 'Ishwar '
full_name: 'Sethi, Ishwar '
last_name: Sethi
- first_name: Weisong
full_name: ' Shi, Weisong'
last_name: ' Shi'
- first_name: 'Guangzhi '
full_name: 'Qu, Guangzhi '
last_name: Qu
- first_name: 'Daniela '
full_name: 'Stan Raicu, Daniela '
last_name: Stan Raicu
- first_name: 'Ruoming '
full_name: 'Jin, Ruoming '
last_name: Jin
keyword:
- deep reinforcement learning
- traffic signal control
- intelligent transportation system
- traffic simulation
language:
- iso: eng
page: 507-514
place: '[Piscataway, NJ]'
publication: 20th IEEE International Conference on Machine Learning and Applications
(ICMLA)
publication_identifier:
isbn:
- 978-1-6654-4337-1
publication_status: published
publisher: IEEE
status: public
title: Towards Real-World Deployment of Reinforcement Learning for Traffic Signal
Control
type: conference_editor_article
user_id: '83781'
year: '2022'
...
---
_id: '11803'
abstract:
- lang: eng
text: Sub-optimal control policies in intersection traffic signal controllers (TSC)
contribute to congestion and lead to negative effects on human health and the
environment. Reinforcement learning (RL) for traffic signal control is a promising
approach to design better control policies and has attracted considerable research
interest in recent years. However, most work done in this area used simplified
simulation environments of traffic scenarios to train RL-based TSC. To deploy
RL in real-world traffic systems, the gap between simplified simulation environments
and real-world applications has to be closed. Therefore, we propose LemgoRL, a
benchmark tool to train RL agents as TSC in a realistic simulation environment
of Lemgo, a medium-sized town in Germany. In addition to the realistic simulation
model, LemgoRL encompasses a traffic signal logic unit that ensures compliance
with all regulatory and safety requirements. LemgoRL offers the same interface
as the well-known OpenAI gym toolkit to enable easy deployment in existing research
work. To demonstrate the functionality and applicability of LemgoRL, we train
a state-of-the-art Deep RL algorithm on a CPU cluster utilizing a framework for
distributed and parallel RL and compare its performance with other methods. Our
benchmark tool drives the development of RL algorithms towards real-world applications.
author:
- first_name: Arthur
full_name: Müller, Arthur
last_name: Müller
- first_name: Vishal
full_name: Rangras, Vishal
id: '76044'
last_name: Rangras
- first_name: Georg
full_name: Schnittker, Georg
last_name: Schnittker
- first_name: Michael
full_name: Waldmann, Michael
last_name: Waldmann
- first_name: Maxim
full_name: Friesen, Maxim
id: '61517'
last_name: Friesen
- first_name: Tobias
full_name: Ferfers, Tobias
last_name: Ferfers
- first_name: Lukas
full_name: Schreckenberg, Lukas
last_name: Schreckenberg
- first_name: Florian
full_name: Hufen, Florian
last_name: Hufen
- first_name: Jürgen
full_name: Jasperneite, Jürgen
id: '1899'
last_name: Jasperneite
- first_name: Marco
full_name: Wiering, Marco
last_name: Wiering
citation:
ama: Müller A, Rangras V, Schnittker G, et al. Towards Real-World Deployment
of Reinforcement Learning for Traffic Signal Control. (Wani MA, IEEE ICMLA, Institute
of Electrical and Electronics Engineers, eds.). IEEE; 2021. doi:10.1109/ICMLA52953.2021.00085
apa: Müller, A., Rangras, V., Schnittker, G., Waldmann, M., Friesen, M., Ferfers,
T., Schreckenberg, L., Hufen, F., Jasperneite, J., & Wiering, M. (2021). Towards
Real-World Deployment of Reinforcement Learning for Traffic Signal Control. In
M. A. Wani, IEEE ICMLA, & Institute of Electrical and Electronics Engineers
(Eds.), 20th IEEE International Conference on Machine Learning and Applications
(ICMLA). IEEE. https://doi.org/10.1109/ICMLA52953.2021.00085
bjps: 'Müller A et al. (2021) Towards Real-World Deployment of
Reinforcement Learning for Traffic Signal Control, Wani MA, IEEE ICMLA,
and Institute of Electrical and Electronics Engineers (eds). Piscataway, NJ:
IEEE.'
chicago: 'Müller, Arthur, Vishal Rangras, Georg Schnittker, Michael Waldmann, Maxim
Friesen, Tobias Ferfers, Lukas Schreckenberg, Florian Hufen, Jürgen Jasperneite,
and Marco Wiering. Towards Real-World Deployment of Reinforcement Learning
for Traffic Signal Control. Edited by M. Arif Wani, IEEE ICMLA, and Institute
of Electrical and Electronics Engineers. 20th IEEE International Conference
on Machine Learning and Applications (ICMLA). Piscataway, NJ: IEEE, 2021.
https://doi.org/10.1109/ICMLA52953.2021.00085.'
chicago-de: 'Müller, Arthur, Vishal Rangras, Georg Schnittker, Michael Waldmann,
Maxim Friesen, Tobias Ferfers, Lukas Schreckenberg, Florian Hufen, Jürgen Jasperneite
und Marco Wiering. 2021. Towards Real-World Deployment of Reinforcement Learning
for Traffic Signal Control. Hg. von M. Arif Wani, IEEE ICMLA, und Institute
of Electrical and Electronics Engineers. 20th IEEE International Conference
on Machine Learning and Applications (ICMLA). Piscataway, NJ: IEEE. doi:10.1109/ICMLA52953.2021.00085,
.'
din1505-2-1: 'Müller,
Arthur ; Rangras, Vishal
; Schnittker, Georg ; Waldmann,
Michael ; Friesen, Maxim
; Ferfers, Tobias ; Schreckenberg,
Lukas ; Hufen, Florian ;
u. a. ; Wani, M. A. ; IEEE ICMLA ;
Institute of Electrical and Electronics Engineers (Hrsg.): Towards Real-World
Deployment of Reinforcement Learning for Traffic Signal Control. Piscataway,
NJ : IEEE, 2021'
havard: A. Müller, V. Rangras, G. Schnittker, M. Waldmann, M. Friesen, T. Ferfers,
L. Schreckenberg, F. Hufen, J. Jasperneite, M. Wiering, Towards Real-World Deployment
of Reinforcement Learning for Traffic Signal Control, IEEE, Piscataway, NJ, 2021.
ieee: 'A. Müller et al., Towards Real-World Deployment of Reinforcement
Learning for Traffic Signal Control. Piscataway, NJ: IEEE, 2021. doi: 10.1109/ICMLA52953.2021.00085.'
mla: Müller, Arthur, et al. “Towards Real-World Deployment of Reinforcement Learning
for Traffic Signal Control.” 20th IEEE International Conference on Machine
Learning and Applications (ICMLA), edited by M. Arif Wani et al., IEEE, 2021,
https://doi.org/10.1109/ICMLA52953.2021.00085.
short: A. Müller, V. Rangras, G. Schnittker, M. Waldmann, M. Friesen, T. Ferfers,
L. Schreckenberg, F. Hufen, J. Jasperneite, M. Wiering, Towards Real-World Deployment
of Reinforcement Learning for Traffic Signal Control, IEEE, Piscataway, NJ, 2021.
ufg: 'Müller, Arthur u. a.: Towards Real-World Deployment of Reinforcement
Learning for Traffic Signal Control, hg. von Wani, M. Arif/ IEEE ICMLA, Institute
of Electrical and Electronics Engineers, Piscataway, NJ 2021.'
van: 'Müller A, Rangras V, Schnittker G, Waldmann M, Friesen M, Ferfers T, et al.
Towards Real-World Deployment of Reinforcement Learning for Traffic Signal Control.
Wani MA, IEEE ICMLA, Institute of Electrical and Electronics Engineers, editors.
20th IEEE International Conference on Machine Learning and Applications (ICMLA).
Piscataway, NJ: IEEE; 2021.'
conference:
end_date: 2021-12-16
location: 'Pasadena, CA, USA '
name: 20th IEEE International Conference on Machine Learning and Applications (ICMLA)
start_date: 2021-12-13
corporate_editor:
- ' IEEE ICMLA'
- ' Institute of Electrical and Electronics Engineers'
date_created: 2024-07-30T05:54:40Z
date_updated: 2024-07-30T07:45:47Z
department:
- _id: DEP5000
- _id: DEP5019
- _id: DEP5020
- _id: DEP6020
doi: 10.1109/ICMLA52953.2021.00085
editor:
- first_name: M. Arif
full_name: Wani, M. Arif
last_name: Wani
external_id:
arxiv:
- arXiv:2103.16223
keyword:
- deep reinforcement learning
- traffic signal control
- intelligent transportation system
- traffic simulation
language:
- iso: eng
place: Piscataway, NJ
publication: 20th IEEE International Conference on Machine Learning and Applications
(ICMLA)
publication_identifier:
eisbn:
- '9781665443371'
publication_status: published
publisher: IEEE
status: public
title: Towards Real-World Deployment of Reinforcement Learning for Traffic Signal
Control
type: conference_editor_article
user_id: '83781'
year: '2021'
...