master thesis autonomous driving

3D-LIDAR Multi Object Tracking for Autonomous Driving

Multi-target detection and tracking under urban road uncertainties.

Contributors

To reference this document use:

Published Date

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

The recent advancement of the autonomous vehicle has raised the need for reliable environmental perception. This is evident, as an autonomous vehicle has to perceive and interpret its local environment in order to execute reactive and predictive control action. Object Tracking is an integral part of vehicle perception, as it enables the vehicle to estimate surrounding objects trajectories to achieve dynamic motion planning. The 3D LIDAR has been widely used in object tracking research since the mechanically compact sensor provides rich, far-reaching and real-time data of spatial information around the vehicle. On the other hand, the development of autonomous driving is heading toward its use in the urban-driving situation. In an urban situation, a robust detection and tracking algorithm is required due to increasing number of Vulnerable Road User (e.g. pedestrian and cyclist), heterogeneous terrain, inherent measurement uncertainties and limited sensor reach. This thesis presents an integrated framework of multi-target object detection and tracking using 3D LIDAR geared toward urban use. The framework combines occlusion-aware detection methods, probabilistic adaptive filtering and computationally efficient heuristics logic-based filtering to handle uncertainties arising from sensing limitation of 3D LIDAR and complexity of the target object movement. The implemented framework takes a raw 3D LIDAR data as input to perform multi-target object detection while simultaneously maintaining track of the detected objects' kinematic states and dimension in robust, causal, and real-time manner. Robust detection is enabled by slope-based ground removal and L-shape fitting to reliably enclose bounding box to objects of interest in the presence of sensor occlusion. The tracker utilises three combined Bayesian filters (IMM-UK-JPDAF) which simultaneously tackle association uncertainties, motion uncertainties and estimate non-linear stochastic motion model in real-time. Logic-based rule filters are also designed to augment the rest of detection and tracking based on the understanding of LIDAR sensor limitation and occlusion characteristic. The evaluation results using real-world pre-recorded 3D LIDAR data show the proposed framework can achieve promising real-time tracking performance in urban situations. Diverse datasets are deliberately chosen to evaluate if the MOT system is capable of working in a varying driving scenario. The benchmark results highlight that the designed and implemented MOT system is performing on par with the state-of-art works and yield satisfying accuracy and precision in most given urban settings.

• Deutsch (Deutschland)
• English (United States)
• Magyar (Magyarország)

Master Thesis World-Model based Sensor Fusion in the Autonomous Driving Domain (f/m/x)

THE FUTURE WILL NEVER SIMPLY WORK IN THEORY.

SHARE YOUR PASSION.

World-leading technologies don’t make it into a BMW until they’ve undergone one of the most challenging journeys imaginable. It takes dynamic teams with outstanding technical skills to take them from the drawing board to the road. That’s why our experts will treat you as part of the team from day one, encourage you to bring your own ideas to the table – and give you the opportunity to really show what you can do.

We, the BMW Group, offer you an interesting and varied master's thesis within the area of autonomous driving sensor fusion. 

In autonomous driving, state-of-the-art deep neural networks performing sensor fusion are trained as backbones for complex tasks (e.g., perception, prediction, planning). While end-to-end training on those tasks allows for specialization of those backbones, it also is restricted to labeled data. A pre-training based on world model understanding (i.e., sensor reconstruction) allows for using larger amounts of data and might lead to a better generalization that can be fine-tuned to specific tasks with supervised training. 

What awaits you?     

• Close collaboration in a team within the BMW research department.      
• Explore different sensor fusion approaches for autonomous driving applications. 
• Evaluate approaches of world model (unsupervised) pre-trained sensor fusion for fine-tuned (supervised) perception tasks in autonomous driving.  
• You are working on state-of-the-art techniques and develop novel approaches on a cluster infrastructure.   

Please note that your thesis must be supervised by a university on your part.   

What should you bring along?   

• Strong knowledge in machine learning.    
• Knowledge in computer vision.    
• Strong programming skills.    
• Proficiency with Python and deep learning frameworks (PyTorch or TensorFlow).    
• Good English skills.   

What do we offer?

• Comprehensive mentoring & onboarding.
• Personal & professional development.
• Flexible working hours.
• Digital offers & mobile working.
• Attractive remuneration.
• Apartment offers for students (subject to availability & only Munich).
• And many other benefits - see  bmw.jobs/benefits

You are enthused by new technologies and an innovative environment? Apply now!

At the BMW Group, we see diversity and inclusion in all its dimensions as a strength for our teams. Equal opportunities are a particular concern for us, and the equal treatment of applicants and employees is a fundamental principle of our corporate policy. That is why our recruiting decisions are also based on personality, experience and skills.

Find out more about diversity at the BMW Group at  bmwgroup.jobs/diversity  

Earliest starting date: from 07/01/2024

Duration: 6 months

Working hours: Full-time

Contact: BMW Group HR Team +49 89 382-17001

• Legal Disclaimer
• Data protection information

• DSpace@MIT Home
• MIT Libraries
• Doctoral Theses

Affordable autonomous lightweight personal mobility

Other Contributors

Terms of use, description, date issued, collections.

Show Statistical Information

Deep Learning-Based Semantic Segmentation in Autonomous Driving

Perception is a fundamental task of autonomous driving systems, which gathers all the necessary information about the surrounding environment of the moving vehicle. Then a decision-making system takes the perception data as input and provides the optimum decision given a scenario, which maximizes the safety of the passengers. In this project, we have developed variants of the U-Net model to perform semantic segmentation on urban scene images to understand the surroundings of an autonomous vehicle. The U-Net model and its variants are adopted for semantic segmentation in this project to account for the power of the U-Net in handling large and small datasets. We have also compared the best-performing variant with other commonly used semantic segmentation models. The comparative analysis was performed using three well-known models, including FCN-16, FCN-8, and SegNet. After conducting sensitivity and comparative analysis, it is concluded that the U-Net variants performed the best in terms of the Intersection over Union (IoU) evaluation metric and other quality metrics.

• Master of Engineering
• Electrical and Computer Engineering

Granting Institution

Lac thesis type, thesis advisor, usage metrics.

• Autonomous vehicles (except systems infrastructures)

• Svenska Close Den här sidan är ej översatt Startsida på svenska

Planning and Motion Control in Autonomous Heavy-Duty Vehicles

Autonomous driving has been an important topic of research in recent years. In this thesis we study its application to Heavy Duty Vehicles, more specifically, vehicles consisting of a truck and a trailer. An overview study is done on three fundamental steps of an autonomous driving system, planning, trajectory tracking and obstacle avoidance.

In the planning part, we use RRT, and two other variants of the algorithm to find trajectories in an unstructured environment, e.g., a mining site. A novel path optimization post-processing technique well suited for use with RRT solutions was also developed. For the trajectory tracking task several well-known controllers were tested, and their performance compared. An extension is proposed to one of the controllers in order to take into account the trailer. The performance evaluation was done on scaled truck systems in the Smart Mobility Lab at KTH. The obstacle avoidance is done with the aid of a simple, yet functional Model Predictive Controller. For this purpose, we developed di erent formulations of the optimization problem, corresponding to distinct optimization goals and vehicle models, in order to assess both the quality of the MPC, and of the assumed truck model. The outcome of this thesis is a fully autonomous system, able to plan and move in constrained environments, while avoiding unpredicted obstacles. It was implemented using a 1:32 scale remote controlled truck, commanded by a desktop computer.

Rui Oliveira

Rui Oliveira's Master's Thesis (pdf 3.7 MB)

• Current Members
• Off-Campus Students
• Robot Videos
• Funded Projects
• Publications by Year
• Publications by Type
• Robot Learning Lecture
• Robot Learning IP
• Humanoid Robotics Seminar
• Research Oberseminar
• New, Open Topics
• Ongoing Theses
• Completed Theses
• External Theses
• Advice for Thesis Students
• Thesis Checklist and Template
• Jobs and Open Positions
• Current Openings
• Information for Applicants
• Apply Here!
• TU Darmstadt Student Hiwi Jobs
• Contact Information

Currently Available Theses Topics

We offer these current topics directly for Bachelor and Master students at TU Darmstadt who can feel free to DIRECTLY contact the thesis advisor if you are interested in one of these topics. Excellent external students from another university may be accepted but are required to first email Jan Peters before contacting any other lab member for a thesis topic. Note that we cannot provide funding for any of these theses projects.

We highly recommend that you do either our robotics and machine learning lectures ( Robot Learning , Statistical Machine Learning ) or our colleagues ( Grundlagen der Robotik , Probabilistic Graphical Models and/or Deep Learning). Even more important to us is that you take both Robot Learning: Integrated Project, Part 1 (Literature Review and Simulation Studies) and Part 2 (Evaluation and Submission to a Conference) before doing a thesis with us.

In addition, we are usually happy to devise new topics on request to suit the abilities of excellent students. Please DIRECTLY contact the thesis advisor if you are interested in one of these topics. When you contact the advisor, it would be nice if you could mention (1) WHY you are interested in the topic (dreams, parts of the problem, etc), and (2) WHAT makes you special for the projects (e.g., class work, project experience, special programming or math skills, prior work, etc.). Supplementary materials (CV, grades, etc) are highly appreciated. Of course, such materials are not mandatory but they help the advisor to see whether the topic is too easy, just about right or too hard for you.

Only contact *ONE* potential advisor at the same time! If you contact a second one without first concluding discussions with the first advisor (i.e., decide for or against the thesis with her or him), we may not consider you at all. Only if you are super excited for at most two topics send an email to both supervisors, so that the supervisors are aware of the additional interest.

FOR FB16+FB18 STUDENTS: Students from other depts at TU Darmstadt (e.g., ME, EE, IST), you need an additional formal supervisor who officially issues the topic. Please do not try to arrange your home dept advisor by yourself but let the supervising IAS member get in touch with that person instead. Multiple professors from other depts have complained that they were asked to co-supervise before getting contacted by our advising lab member.

NEW THESES START HERE

Data-Driven Bimanual Robotic Grasping

Scope: Bachelor/Master thesis Advisor: Vignesh Prasad and Alap Kshirsagar Added: 2024-04-25 Start: ASAP Topic: Topic:

Grasping is one of the most fundamental and challenging tasks in the robotic manipulation of objects. Most of the prior work on robotic grasping has focused on grasping with a single gripper and several large-scale datasets have been developed in recent years to tackle the problem of single-arm grasping in 3D by utilizing deep-learning techniques [1,2]. But many tasks in industrial and domestic environments require bimanual grasps. Bimanual grasps are required for manipulation of large, deformable or fragile objects. This project seeks to develop a data-driven technique for bimanual robotic grasp generation from visual input. We will utilize a large-scale dataset of simulated bimanual grasps [3] to train a bimanual grasp pose generation model. The method will be evaluated in simulation as well as on a real robot.

Requirements

• Strong Python programming skills
• Knowledge in Machine Learning / Supervised Learning
• Experience with deep learning libraries is a plus

Interested students can apply by sending an e-mail to [email protected] and attaching the documents mentioned below:

• Curriculum Vitae
• Motivation letter explaining why you would like to work on this topic and why you are the perfect candidate

References [1] C. Eppner, A. Mousavian, and D. Fox, “ACRONYM: A Large-Scale Grasp Dataset Based on Simulation,” in Proceedings - IEEE International Conference on Robotics and Automation, 2021, vol. 2021-May, pp. 6222–6227, doi: 10.1109/ICRA48506.2021.9560844. [2] A. Mousavian, C. Eppner, and Di. Fox, “6-DOF GraspNet: Variational grasp generation for object manipulation,” in Proceedings of the IEEE International Conference on Computer Vision, 2019, vol. 2019-Octob, pp. 2901–2910, doi: 10.1109/ICCV.2019.00299. [3] G. Zhai et al., “{DA2} Dataset: Toward Dexterity-Aware Dual-Arm Grasping,” IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 8941–8948, 2022.

Imitation Learning for High-Speed Robot Air Hockey

Scope: Master thesis Advisor: Puze Liu and Julen Urain De Jesus Start: ASAP Topic:

High-speed reactive motion is one of the fundamental capabilities of robots to achieve human-level behavior. Optimization-based methods suffer from real-time requirement when the problem is non-convex and contains constraints. Reinforcement learning requires extensive reward engineering to achieve the desired performance. Imitation learning, on the other hand, gathers human knowledge directly from data collection and enables robots to learn natural movements efficiently. In this paper, we explore how imitation learning can be performed in a complex robot Air Hockey Task. The robot needs to learn not only low-level skills, but also high-level tactics from human demonstrations.

• Good Knowledge in Robotics

References * Chi, Cheng, et al. "Diffusion policy: Visuomotor policy learning via action diffusion." arXiv preprint arXiv:2303.04137 (2023). * Liu, Puze, et al. "Robot reinforcement learning on the constraint manifold." Conference on Robot Learning. PMLR (2022). * Pan, Yunpeng, et al. "Imitation learning for agile autonomous driving." The International Journal of Robotics Research 39.2-3 (2020). Interested students can apply by sending an e-mail to [email protected] and attaching the required documents mentioned above.

Walk your network: investigating neural network’s location in Q-learning methods.

Scope: Master thesis Advisor: Theo Vincent Start: Flexible Topic:

Q-learning methods are at the heart of Reinforcement Learning. They have been shown to outperform humans on some complex tasks such as playing video games [1]. In robotics, where the action space is in most cases continuous, actor-critic methods are relying on Q-learning methods to learn the critic [2]. Although Q-learning methods have been extensively studied in the past, little focus has been placed on the way the online neural network is exploring the space of Q functions. Most approaches focus on crafting a loss that would make the agent learn better policies [3]. Here, we offer a thesis that focuses on the position of the online Q neural network in the space of Q functions. The student will first investigate this idea on simple problems before comparing the performance to strong baselines such as DQN or REM [1, 4] on Atari games. Depending on the result, the student might as well get into MuJoCo and compare the results with SAC [2]. The student will be welcome to propose some ideas as well.

Highly motivated students can apply by sending an email to [email protected] . Please attach your CV, a grade sheet and clearly state why you are interested in this topic. Students who have followed the Reinforcement Learning or Robot Learning course will be prioritized.

• Knowledge in Reinforcement Learning

References [1] Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." nature 518.7540 (2015): 529-533. [2] Haarnoja, Tuomas, et al. "Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor." International conference on machine learning. PMLR, 2018. [3] Hessel, Matteo, et al. "Rainbow: Combining improvements in deep reinforcement learning." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018. [4] Agarwal, R., Schuurmans, D. & Norouzi, M.. (2020). An Optimistic Perspective on Offline Reinforcement Learning International Conference on Machine Learning (ICML).

Co-optimizing Hand and Action for Robotic Grasping of Deformable objects

This project aims to advance deformable object manipulation by co-optimizing robot gripper morphology and control policies. The project will involve utilizing existing simulation environments for deformable object manipulation [2] and implementing a method to jointly optimize gripper morphology and grasp policies within the simulation.

Required Qualification:

• Familiarity with deep learning libraries such as PyTorch or Tensorflow

Preferred Qualification:

• Attendance of the lectures "Statistical Machine Learning", "Computational Engineering and Robotics" and "Robot Learning"

Application Requirements:

Interested students can apply by sending an e-mail to [email protected] and attaching the required documents mentioned above.

References: [1] Xu, Jie, et al. "An End-to-End Differentiable Framework for Contact-Aware Robot Design." Robotics: Science & Systems. 2021. [2] Huang, Isabella, et al. "DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets." arXiv preprint arXiv:2303.16138 (2023).

Geometry-Aware Diffusion Models for Robotics

In this thesis, you will work on developing an imitation learning algorithm using diffusion models for robotic manipulation tasks, such as the ones in [2, 3, 4], but taking into account the geometry of the task space.

If this sounds interesting, please send an email to [email protected] and [email protected] , and possibly attach your CV, highlighting the relevant courses you took in robotics and machine learning.

What's in it for you:

• You get to work on an exciting topic at the intersection of deep-learning and robotics
• We will supervise you closely throughout your thesis
• Depending on the results, we will aim for an international conference publication

Requirements:

• Be motivated -- we will support you a lot, but we expect you to contribute a lot too
• Robotics knowledge
• Experience setting up deep learning pipelines -- from data collection, architecture design, training, and evaluation
• PyTorch -- especially experience writing good parallelizable code (i.e., runs fast in the GPU)

References: [1] https://arxiv.org/abs/2112.10752 [2] https://arxiv.org/abs/2308.01557 [3] https://arxiv.org/abs/2209.03855 [4] https://arxiv.org/abs/2303.04137 [5] https://arxiv.org/abs/2205.09991

Learning Latent Representations for Embodied Agents

Interested students can apply by sending an E-Mail to [email protected] and attaching the required documents mentioned below.

• Experience with TensorFlow/PyTorch
• Familiarity with core Machine Learning topics
• Experience programming/controlling robots (either simulated or real world)
• Knowledgeable about different robot platforms (quadrupeds and bipedal robots)
• Resume / CV
• Cover letter explaining why this topic fits you well and why you are an ideal candidate

References: [1] Ho and Ermon. "Generative adversarial imitation learning" [2] Arenz, et al. "Efficient Gradient-Free Variational Inference using Policy Search"

Characterizing Fear-induced Adaptation of Balance by Inverse Reinforcement Learning

Interested students can apply by sending an E-Mail to [email protected] and attaching the required documents mentioned below.

• Basic knowledge of reinforcement learning
• Hand-on experience with reinforcement learning or inverse reinforcement learning
• Cognitive science background

References: [1] Maki, et al. "Fear of Falling and Postural Performance in the Elderly" [2] Davis et al. "The relationship between fear of falling and human postural control" [3] Ho and Ermon. "Generative adversarial imitation learning"

Timing is Key: CPGs for regularizing Quadruped Gaits learned with DRL

To tackle this problem we want to utilize Central Pattern Generators (CPGs), which can generate timings for ground contacts for the four feet. The policy gets rewarded for complying with the contact patterns of the CPGs. This leads to a straightforward way of regularizing and steering the policy to a natural gait without posing too strong restrictions on it. We first want to manually find fitting CPG parameters for different gait velocities and later move to learning those parameters in an end-to-end fashion.

Highly motivated students can apply by sending an E-Mail to [email protected] and attaching the required documents mentioned below.

Minimum Qualification:

• Good Python programming skills
• Basic knowledge of the PyTorch library
• Basic knowledge of Reinforcement Learning
• Good knowledge of the PyTorch library
• Basic knowledge of the MuJoCo simulator

References: [1] Cheng, Xuxin, et al. "Extreme Parkour with Legged Robots."

Damage-aware Reinforcement Learning for Deformable and Fragile Objects

Goal of this thesis will be the development and application of a model-based reinforcement learning method on real robots. Your tasks will include: 1. Setting up a simulation environment for deformable object manipulation 2. Utilizing existing models for stress and deformability prediction[1] 3. Implementing a reinforcement learning method to work in simulation and, if possible, on the real robot methods.

If you are interested in this thesis topic and believe you possess the necessary skills and qualifications, please submit your application, including a resume and a brief motivation letter explaining your interest and relevant experience. Please send your application to [email protected].

Required Qualification :

• Enthusiasm for and experience in robotics, machine learning, and simulation
• Strong programming skills in Python

Desired Qualification :

• Attendance of the lectures "Statistical Machine Learning", "Computational Engineering and Robotics" and (optionally) "Robot Learning"

References: [1] Huang, I., Narang, Y., Bajcsy, R., Ramos, F., Hermans, T., & Fox, D. (2023). DefGraspNets: Grasp Planning on 3D Fields with Graph Neural Nets. arXiv preprint arXiv:2303.16138.

Imitation Learning meets Diffusion Models for Robotics

The objective of this thesis is to build upon prior research [2, 3] to establish a connection between Diffusion Models and Imitation Learning. We aim to explore how to exploit Diffusion Models and improve the performance of Imitation learning algorithms that interact with the world.

We welcome highly motivated students to apply for this opportunity by sending an email expressing their interest to Firas Al-Hafez ( [email protected] ) Julen Urain ( [email protected] ). Please attach your letter of motivation and CV, and clearly state why you are interested in this topic and why you are the ideal candidate for this position.

Required Qualification : 1. Strong Python programming skills 2. Basic Knowledge in Imitation Learning 3. Interest in Diffusion models, Reinforcement Learning

Desired Qualification : 1. Attendance of the lectures "Statistical Machine Learning", "Computational Engineering and Robotics" and/or "Reinforcement Learning: From Fundamentals to the Deep Approaches"

References: [1] Song, Yang, and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution." Advances in neural information processing systems 32 (2019). [2] Ho, Jonathan, and Stefano Ermon. "Generative adversarial imitation learning." Advances in neural information processing systems 29 (2016). [3] Garg, D., Chakraborty, S., Cundy, C., Song, J., & Ermon, S. (2021). Iq-learn: Inverse soft-q learning for imitation. Advances in Neural Information Processing Systems, 34, 4028-4039. [4] Chen, R. T., & Lipman, Y. (2023). Riemannian flow matching on general geometries. arXiv preprint arXiv:2302.03660.

• Be extremely motivated -- we will support you a lot, but we expect you to contribute a lot too

Scaling Behavior Cloning to Humanoid Locomotion

Scope: Bachelor / Master thesis Advisor: Joe Watson Added: 2023-10-07 Start: ASAP Topic: In a previous project [1], I found that behavior cloning (BC) was a surprisingly poor baseline for imitating humanoid locomotion. I suspect the issue may lie in the challenges of regularizing high-dimensional regression.

The goal of this project is to investigate BC for humanoid imitation, understand the scaling issues present, and evaluate possible solutions, e.g. regularization strategies from the regression literature.

The project will be building off Google Deepmind's Acme library [2], which has BC algorithms and humanoid demonstration datasets [3] already implemented, and will serve as the foundation of the project.

To apply, email [email protected] , ideally with a CV and transcript so I can assess your suitability.

• Experience, interest and enthusiasm for the intersection of robot learning and machine learning
• Experience with Acme and JAX would be a benefit, but not necessary

References: [1] https://arxiv.org/abs/2305.16498 [2] https://github.com/google-deepmind/acme [3] https://arxiv.org/abs/2106.00672

Robot Gaze for Communicating Collision Avoidance Intent in Shared Workspaces

Scope: Bachelor/Master thesis Advisor: Alap Kshirsagar , Dorothea Koert Added: 2023-09-27 Start: ASAP

Topic: In order to operate close to non-experts, future robots require both an intuitive form of instruction accessible to lay users and the ability to react appropriately to a human co-worker. Instruction by imitation learning with probabilistic movement primitives (ProMPs) [1] allows capturing tasks by learning robot trajectories from demonstrations including the motion variability. However, appropriate responses to human co-workers during the execution of the learned movements are crucial for fluent task execution, perceived safety, and subjective comfort. To facilitate such appropriate responsive behaviors in human-robot interaction, the robot needs to be able to react to its human workspace co-inhabitant online during the execution. Also, the robot needs to communicate its motion intent to the human through non-verbal gestures such as eye and head gazes [2][3]. In particular for humanoid robots, combining motions of arms with expressive head and gaze directions is a promising approach that has not yet been extensively studied in related work.

Goals of the thesis:

• Develop a method to combine robot head/gaze motion with ProMPs for online collision avoidance
• Implement the method on a Franka-Emika Panda Robot
• Evaluate and compare the implemented behaviors in a study with human participants

Highly motivated students can apply by sending an email to [email protected]. Please attach your CV and transcript, and clearly state your prior experiences and why you are interested in this topic.

• Strong Programming Skills in python
• Prior experience with Robot Operating System (ROS) and user studies would be beneficial
• Strong motivation for human-centered robotics including design and implementation of a user study

References : [1] Koert, Dorothea, et al. "Learning intention aware online adaptation of movement primitives." IEEE Robotics and Automation Letters 4.4 (2019): 3719-3726. [2] Admoni, Henny, and Brian Scassellati. "Social eye gaze in human-robot interaction: a review." Journal of Human-Robot Interaction 6.1 (2017): 25-63. [3] Lemasurier, Gregory, et al. "Methods for expressing robot intent for human–robot collaboration in shared workspaces." ACM Transactions on Human-Robot Interaction (THRI) 10.4 (2021): 1-27.

Tactile Sensing for the Real World

Topic: Tactile sensing is a crucial sensing modality that allows humans to perform dexterous manipulation[1]. In recent years, the development of artificial tactile sensors has made substantial progress, with current models relying on cameras inside the fingertips to extract information about the points of contact [2]. However, robotic tactile sensing is still a largely unsolved topic despite these developments. A central challenge of tactile sensing is the extraction of usable representations of sensor readings, especially since these generally contain an incomplete view of the environment.

Recent model-based reinforcement learning methods like Dreamer [3] leverage latent state-space models to reason about the environment from partial and noisy observations. However, more work has yet to be done to apply such methods to real-world manipulation tasks. Hence, this thesis will explore whether Dreamer can solve challenging real-world manipulation tasks by leveraging tactile information. Initial results suggest that tasks like peg-in-a-hole can indeed be solved with Dreamer in simulation (see figure above), but the applicability of this method in the real world has yet to be shown.

In this work, you will work with state-of-the-art hardware and compute resources on a hot research topic with the option of publishing your work at a scientific conference.

Highly motivated students can apply by sending an email to [email protected]. Please attach a transcript of records and clearly state your prior experiences and why you are interested in this topic.

• Ideally experience with deep learning libraries like JAX or PyTorch
• Experience with reinforcement learning is a plus
• Experience with Linux

References [1] 2S Match Anest2, Roland Johansson Lab (2005), https://www.youtube.com/watch?v=HH6QD0MgqDQ [2] Gelsight Inc., Gelsight Mini, https://www.gelsight.com/gelsightmini/ [3] Hafner, D., Lillicrap, T., Ba, J., & Norouzi, M. (2019). Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603.

Large Vision-Language Neural Networks for Open-Vocabulary Robotic Manipulation

Robots are expected to soon leave their factory/laboratory enclosures and operate autonomously in everyday unstructured environments such as households. Semantic information is especially important when considering real-world robotic applications where the robot needs to re-arrange objects as per a set of language instructions or human inputs (as shown in the figure). Many sophisticated semantic segmentation networks exist [1]. However, a challenge when using such methods in the real world is that the semantic classes rarely align perfectly with the language input received by the robot. For instance, a human language instruction might request a ‘glass’ or ‘water’, but the semantic classes detected might be ‘cup’ or ‘drink’.

Nevertheless, with the rise of large language and vision-language models, we now have capable segmentation models that do not directly predict semantic classes but use learned associations between language queries and classes to give us ’open-vocabulary’ segmentation [2]. Some models are especially powerful since they can be used with arbitrary language queries.

In this thesis, we aim to build on advances in 3D vision-based robot manipulation and large open-vocabulary vision models [2] to build a full pick-and-place pipeline for real-world manipulation. We also aim to find synergies between scene reconstruction and semantic segmentation to determine if knowing the object semantics can aid the reconstruction of the objects and, in turn, aid manipulation.

Highly motivated students can apply by sending an e-mail expressing their interest to Snehal Jauhri (email: [email protected]) or Ali Younes (email: [email protected]), attaching your letter of motivation and possibly your CV.

Topic in detail : Thesis_Doc.pdf

Requirements: Enthusiasm, ambition, and a curious mind go a long way. There will be ample supervision provided to help the student understand basic as well as advanced concepts. However, prior knowledge of computer vision, robotics, and Python programming would be a plus.

References: [1] Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2”, https://github.com/facebookresearch/detectron2 , 2019. [2] F. Liang, B. Wu, X. Dai, K. Li, Y. Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” in CVPR, 2023, pp. 7061–7070, https://github.com/facebookresearch/ov-seg

Dynamic Tiles for Deep Reinforcement Learning

Linear approximators in Reinforcement Learning are well-studied and come with an in-depth theoretical analysis. However, linear methods require defining a set of features of the state to be used by the linear approximation. Unfortunately, the feature construction process is a particularly problematic and challenging task. Deep Reinforcement learning methods have been introduced to mitigate the feature construction problem: these methods do not require handcrafted features, as features are extracted automatically by the network during learning, using gradient descent techniques.

In simple reinforcement learning tasks, however, it is possible to use tile coding as features: Tiles are simply a convenient discretization of the state space that allows us to easily control the generalization capabilities of the linear approximator. The objective of this thesis is to design a novel algorithm for automatic feature extraction that generates a set of features similar to tile coding, but that can arbitrarily partition the state space and deal with arbitrary complex state space, such as images. The idea is to combine the feature extraction problem directly with Linear Reinforcement Learning methods, defining an algorithm that is able both to have the theoretical guarantees and good convergence properties of these methods and the flexibility of Deep Learning approaches.

• Curriculum Vitae (CV);
• A motivation letter explaining the reason for applying for this thesis and academic/career objectives.

Minimum knowledge

• Good Python programming skills;
• Basic knowledge of Reinforcement Learning.

Preferred knowledge

• Knowledge of the PyTorch library;
• Knowledge of the Atari environments (ale-py library).
• Knowledge of the MushroomRL library.

Accepted candidate will

• Define a generalization of tile coding working with an arbitrary input set (including images);
• Design a learning algorithm to adapt the tiles using data of interaction with the environment;
• Combine feature learning with standard linear methods for Reinforcement Learning;
• Verify the novel methodology in simple continuous state and discrete actions environments;
• (Optionally) Extend the experimental analysis to the Atari environment setting.

Deep Learning Meets Teleoperation: Constructing Learnable and Stable Inductive Guidance for Shared Control

This work considers policies as learnable inductive guidance for shared control. In particular, we use the class of Riemannian motion policies [3] and consider them as differentiable optimization layers [4]. We analyze (i) if RMPs can be pre-trained by learning from demonstrations [5] or reinforcement learning [6] given a specific context; (ii) and subsequently employed seamlessly for human-guided teleoperation thanks to their physically consistent properties, such as stability [3]. We believe this step eliminates the laborious process of constructing complex policies and leads to improved and generalizable shared control architectures.

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] and [email protected] , attaching your letter of motivation and possibly your CV.

• Experience with deep learning libraries (in particular Pytorch)
• Knowledge in reinforcement learning and/or machine learning

References: [1] Niemeyer, Günter, et al. "Telerobotics." Springer handbook of robotics (2016); [2] Selvaggio, Mario, et al. "Autonomy in physical human-robot interaction: A brief survey." IEEE RAL (2021); [3] Cheng, Ching-An, et al. "RMP flow: A Computational Graph for Automatic Motion Policy Generation." Springer (2020); [4] Jaquier, Noémie, et al. "Learning to sequence and blend robot skills via differentiable optimization." IEEE RAL (2022); [5] Mukadam, Mustafa, et al. "Riemannian motion policy fusion through learnable lyapunov function reshaping." CoRL (2020); [6] Xie, Mandy, et al. "Neural geometric fabrics: Efficiently learning high-dimensional policies from demonstration." CoRL (2023).

Dynamic symphony: Seamless human-robot collaboration through hierarchical policy blending

This work focuses on arbitration between the user and assistive policy, i.e., shared autonomy. Various works allow the user to influence the dynamic behavior explicitly and, therefore, could not satisfy stability guarantees [3]. We pursue the idea of formulating arbitration as a trajectory-tracking problem that implicitly considers the user's desired behavior as an objective [4]. Therefore, we extend the work of Hansel et al. [5], who employed probabilistic inference for policy blending in robot motion control. The proposed method corresponds to a sampling-based online planner that superposes reactive policies given a predefined objective. This method enables the user to implicitly influence the behavior without injecting energy into the system, thus satisfying stability properties. We believe this step leads to an alternative view of shared autonomy with an improved and generalizable framework.

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] or [email protected] , attaching your letter of motivation and possibly your CV.

References: [1] Niemeyer, Günter, et al. "Telerobotics." Springer handbook of robotics (2016); [2] Selvaggio, Mario, et al. "Autonomy in physical human-robot interaction: A brief survey." IEEE RAL (2021); [3] Dragan, Anca D., and Siddhartha S. Srinivasa. "A policy-blending formalism for shared control." IJRR (2013); [4] Javdani, Shervin, et al. "Shared autonomy via hindsight optimization for teleoperation and teaming." IJRR (2018); [5] Hansel, Kay, et al. "Hierarchical Policy Blending as Inference for Reactive Robot Control." IEEE ICRA (2023).

Feeling the Heat: Igniting Matches via Tactile Sensing and Human Demonstrations

In this thesis, we want to investigate the effectiveness of vision-based tactile sensors for solving dynamic tasks (igniting matches). Since the whole task is difficult to simulate, we directly collect real-world data to learn policies from the human demonstrations [2,3]. We believe that this work is an important step towards more advanced tactile skills.

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] and [email protected] , attaching your letter of motivation and possibly your CV.

• Good knowledge of Python
• Prior experience with real robots and Linux is a plus

References: [1] https://www.youtube.com/watch?v=HH6QD0MgqDQ [2] Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction; Klas Kronander and Aude Billard. [3] https://www.youtube.com/watch?v=jAtNvfPrKH8

Inverse Reinforcement Learning for Neuromuscular Control of Humanoids

Within this thesis, the problems of learning from observations and efficient exploration in overactued systems should be addressed. Regarding the former, novel methods incorporating inverse dynamics models into the inverse reinforcement learning problem [1] should be adapted and applied. To address the problem of efficient exploration in overactuted systems, two approaches should be implemented and compared. The first approach uses a handcrafted action space, which disables and modulates actions in different phases of the gait based on biomechanics knowledge [2]. The second approach uses a stateful policy to incorporate an inductive bias into the policy [3]. The thesis will be supervised in conjunction with Guoping Zhao ( [email protected] ) from the locomotion lab.

Highly motivated students can apply by sending an e-mail expressing their interest to Firas Al-Hafez ( [email protected] ), attaching your letter of motivation and possibly your CV. Try to make clear why you would like to work on this topic, and why you would be the perfect candidate for the latter.

Required Qualification : 1. Strong Python programming skills 2. Knowledge in Reinforcement Learning 3. Interest in understanding human locomotion

Desired Qualification : 1. Hands-on experience on robotics-related RL projects 2. Prior experience with different simulators 3. Attendance of the lectures "Statistical Machine Learning", "Computational Engineering and Robotics" and/or "Reinforcement Learning: From Fundamentals to the Deep Approaches"

References: [1] Al-Hafez, F.; Tateo, D.; Arenz, O.; Zhao, G.; Peters, J. (2023). LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learning, International Conference on Learning Representations (ICLR). [2] Ong CF; Geijtenbeek T.; Hicks JL; Delp SL (2019) Predicting gait adaptations due to ankle plantarflexor muscle weakness and contracture using physics-based musculoskeletal simulations. PLoS Computational Biology [3] Srouji, M.; Zhang, J:;Salakhutdinow, R. (2018) Structured Control Nets for Deep Reinforcement Learning, International Conference on Machine Learning (ICML)

Robotic Tactile Exploratory Procedures for Identifying Object Properties

Goals of the thesis

• Literature review of robotic EPs for identifying object properties [2,3,4]
• Develop and implement robotic EPs for a Digit tactile sensor
• Compare performance of robotic EPs with human EPs

Desired Qualifications

• Interested in working with real robotic systems
• Python programming skills

Literature [1] Lederman and Klatzky, “Haptic perception: a tutorial” [2] Seminara et al., “Active Haptic Perception in Robots: A Review” [3] Chu et al., “Using robotic exploratory procedures to learn the meaning of haptic adjectives” [4] Kerzel et al., “Neuro-Robotic Haptic Object Classification by Active Exploration on a Novel Dataset”

Scaling learned, graph-based assembly policies

• scaling our previous methods to incorporate mobile manipulators or the Kobo bi-manual manipulation platform. The increased workspace of both would allow for handling a wider range of objects
• [2] has shown more powerful, yet, it includes running a MILP for every desired structure. Thus another idea could be to investigate approaches aiming to approximate this solution
• adapting the methods to handle more irregular-shaped objects / investigate curriculum learning

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] , attaching your letter of motivation and possibly your CV.

• Experience with deep learning libraries (in particular Pytorch) is a plus
• Experience with reinforcement learning / having taken Robot Learning is also a plus

References: [1] Learn2Assemble with Structured Representations and Search for Robotic Architectural Construction; Niklas Funk et al. [2] Graph-based Reinforcement Learning meets Mixed Integer Programs: An application to 3D robot assembly discovery; Niklas Funk et al. [3] Structured agents for physical construction; Victor Bapst et al.

Long-Horizon Manipulation Tasks from Visual Imitation Learning (LHMT-VIL): Algorithm

The proposed architecture can be broken down into the following sub-tasks: 1. Multi-object 6D pose estimation from video: Identify the object 6D poses in each video frame to generate the object trajectories 2. Action segmentation from video: Classify the action being performed in each video frame 3. High-level task representation learning: Learn the sequence of robotic movement primitives with the associated object poses such that the robot completes the demonstrated task 4. Low-level movement primitives: Create a database of low-level robotic movement primitives which can be sequenced to solve the long-horizon task

Desired Qualification: 1. Strong Python programming skills 2. Prior experience in Computer Vision and/or Robotics is preferred

Long-Horizon Manipulation Tasks from Visual Imitation Learning (LHMT-VIL): Dataset

During the project, we will create a large-scale dataset of videos of humans demonstrating industrial assembly sequences. The dataset will contain information of the 6D poses of the objects, the hand and body poses of the human, the action sequences among numerous other features. The dataset will be open-sourced to encourage further research on VIL.

[1] F. Sener, et al. "Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities". CVPR 2022. [2] P. Sharma, et al. "Multiple Interactions Made Easy (MIME) : Large Scale Demonstrations Data for Imitation." CoRL, 2018.

Adaptive Human-Robot Interactions with Human Trust Maximization

• Good knowledge of Python and/or C++;
• Good knowledge in Robotics and Machine Learning;
• Good knowledge of Deep Learning frameworks, e.g, PyTorch;

References: [1] Xu, Anqi, and Gregory Dudek. "Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations." ACM/IEEE HRI, IEEE, 2015; [2] Kwon, Minae, et al. "When humans aren’t optimal: Robots that collaborate with risk-aware humans." ACM/IEEE HRI, IEEE, 2020; [3] Chen, Min, et al. "Planning with trust for human-robot collaboration." ACM/IEEE HRI, IEEE, 2018; [4] Poole, Ben et al. “On variational bounds of mutual information”. ICML, PMLR, 2019.

Causal inference of human behavior dynamics for physical Human-Robot Interactions

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] , attaching your a letter of motivation and possibly your CV.

• Good knowledge of Robotics;
• Good knowledge of Deep Learning frameworks, e.g, PyTorch
• Li, Q., Chalvatzaki, G., Peters, J., Wang, Y., Directed Acyclic Graph Neural Network for Human Motion Prediction, 2021 IEEE International Conference on Robotics and Automation (ICRA).
• Löwe, S., Madras, D., Zemel, R. and Welling, M., 2020. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006.10833.
• Yang, W., Paxton, C., Mousavian, A., Chao, Y.W., Cakmak, M. and Fox, D., 2020. Reactive human-to-robot handovers of arbitrary objects. arXiv preprint arXiv:2011.08961.

Incorporating First and Second Order Mental Models for Human-Robot Cooperative Manipulation Under Partial Observability

Scope: Master Thesis Advisor: Dorothea Koert , Joni Pajarinen Added: 2021-06-08 Start: ASAP

The ability to model the beliefs and goals of a partner is an essential part of cooperative tasks. While humans develop theory of mind models for this aim already at a very early age [1] it is still an open question how to implement and make use of such models for cooperative robots [2,3,4]. In particular, in shared workspaces human robot collaboration could potentially profit from the use of such models e.g. if the robot can detect and react to planned human goals or a human's false beliefs during task execution. To make such robots a reality, the goal of this thesis is to investigate the use of first and second order mental models in a cooperative manipulation task under partial observability. Partially observable Markov decision processes (POMDPs) and interactive POMDPs (I-POMDPs) [5] define an optimal solution to the mental modeling task and may provide a solid theoretical basis for modelling. The thesis may also compare related approaches from the literature and setup an experimental design for evaluation with the bi-manual robot platform Kobo.

Highly motivated students can apply by sending an e-mail expressing your interest to [email protected] attaching your CV and transcripts.

References:

• Wimmer, H., & Perner, J. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception (1983)
• Sandra Devin and Rachid Alami. An implemented theory of mind to improve human-robot shared plans execution (2016)
• Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, SM Ali Eslami,and Matthew Botvinick. Machine theory of mind (2018)
• Connor Brooks and Daniel Szafir. Building second-order mental models for human-robot interaction. (2019)
• Prashant Doshi, Xia Qu, Adam Goodie, and Diana Young. Modeling recursive reasoning by humans using empirically informed interactive pomdps. (2010)

Google Custom Search

Wir verwenden Google für unsere Suche. Mit Klick auf „Suche aktivieren“ aktivieren Sie das Suchfeld und akzeptieren die Nutzungsbedingungen.

Hinweise zum Einsatz der Google Suche

• Professorship of Autonomous Vehicle Systems
• TUM School of Engineering and Design
• Technical University of Munich

Our Thesis Offers

You are a student at TUM and are interested in current topics in the field of autonomous driving, autonomous systems and robotics? Then the AVS Lab is the right place for you! We offer numerous bachelor thesis, semester thesis and master thesis projects, you can find the current topics and calls here. If you are interested in a student research project, you can apply directly to the contact person. Simply send an e-mail with a short motivation, CV and a current performance record.

If your desired topic is not listed, you are welcome to send us an unsolicited application. Just write us what you are interested in and what you would like to write your term paper about. Together we will surely find an exciting topic with which you can successfully complete your student research project.

The following student research projects are currently open:

BA SA MA HW exp the kon Eintrag Titel – – ✔ – - - - 2024-07-09 Entwicklung eines differenzierbaren Trajektorien-Planner für das autonome Fahren – ✔ ✔ – ● ● - 2024-07-09 Entwicklung einer 3D Simulationsumgebung für Autonome Fahrzeuge aus realen Fahrdaten – ✔ ✔ – ● ● - 2024-07-09 Evaluierung und Vergleich verschiedener Prädiktionsmethoden im autonomen Fahren – ✔ ✔ – ● ● - 2024-07-09 Autonome Fahrzeugsysteme - Motion Planning (SA, MA, IDP, ...) – – – ✔ ● ● ● 2024-07-08 HiWi im Autonomous Vehicle Systems (AVS) Lab – ✔ ✔ – ● ● - 2024-05-28 Motion Planning for Autonomous Racing: Handling Secondary Drivable Areas on a Race Track ✔ ✔ ✔ – ● ● - 2024-05-28 TUM Autonomous Motorsports: Motion Planning für die Indy Autonomous Challenge (BA / SA / MA / ...) ✔ ✔ ✔ – ● ● - 2024-05-28 TUM Autonomous Motorsports: High-Performance Motion Planning in Multiple Reference Frames ✔ ✔ ✔ – - - - 2024-02-27 Implementing a Modular Autonomous Vehicle Software Stack in the CARLA Simulator – ✔ ✔ – - - - 2024-02-27 Improving Semantic Anomaly Detection through Prompt Tuning of LLMs – ✔ ✔ – - - - 2024-02-27 Enhancing Semantic Anomaly Detection using Low-Rank Adaptation (LoRA) of LLMs – ✔ ✔ – - - - 2024-02-27 Advancing Semantic Anomaly Detection through Fine-Tuning of Large Language Models (LLMs) – – ✔ – ● ● - 2024-02-27 First Comprehensive Dataset for Blind-Specific Outdoor Navigation – – ✔ – ● ● - 2024-02-27 Efficient Holistic Segmentation Techniques for Obstacle Avoidance in Assistive Navigation for the Visually Impaired – – ✔ – ● ● - 2024-02-27 Obstacle Avoidance and Path Planning for Blind People on Edge Devices ✔ ✔ ✔ – - ● - 2024-02-07 Motion Planning für das autonome Fahren (BA/SA/MA/IDP)

BA SA MA HW exp the kon Eintrag Titel – ✔ ✔ – ● ● - 2024-05-14 Oktoberfest 2024: Motion Control for EDGAR – ✔ ✔ – ● ● - 2024-03-04 Safe Deep Reinforcement Learning based on Bayesian Optimization for Weights-varying Model Predictive Control ✔ ✔ ✔ – ● ● - 2024-03-04 Machine Learning & Model Predictive Control for Autonomous Vehicle Motion Control – ✔ ✔ – ● ● - 2024-03-04 Multimodal Large Language Models for Autonomous Vehicles Awareness ✔ ✔ ✔ – ● ● - 2024-03-04 Deep Reinforcement Learning for Model Predictive Control based Autonomous Vehicle Motion Control – ✔ ✔ – ● ● - 2024-01-03 Bayesian Optimization of Model Predictive Control Parameters for Autonomous Vehicle Guidance – ✔ ✔ – ● ● - 2023-10-18 Motion control of autonomous vehicles with Stochastic Model Predictive Control – ✔ ✔ – ● ● - 2023-10-18 Estimating Parameters for Autonomous Vehicle Dynamics using Unscented Kalman Filter and Heuristics of Differential Evolution Algorithms – ✔ ✔ – ● ● - 2023-08-09 Learning-Based Model Predictive Control for Autonomous Vehicle Trajectory Following using Gaussian Process Regression – ✔ ✔ – ● ● - 2023-01-30 Motion control of autonomous vehicles with Tube Nonlinear Model Predictive Control ✔ – ✔ – ● ● - 2022-05-20 Sensitivity Analysis for Tractor-Trailer Dynamics – Prediction Model Quality Assessment

BA SA MA HW exp the kon Eintrag Titel – ✔ ✔ – ● ● ● 2024-02-12 Autonomous Driving Projekt: Entwicklung von Motion Planning Algorithmen für ein Level-5 Forschungsfahrzeug – ✔ ✔ – ● ● ● 2022-12-11 Autonomous Driving Motion Planning Parallel Computing & Acceleration

BA SA MA HW exp the kon Eintrag Titel – ✔ ✔ – ● ● - 2023-10-23 Prediktive Exploration der fahrdynamischen Grenzen für autonome Fahrzeuge auf Rennstrecken – ✔ ✔ – ● ● - 2023-10-23 Autonomous Driving Drift Control - Vehicle Stabilization für leistungsstarke Seriensportwagen ✔ ✔ ✔ – - - - 2023-08-30 Autonomous Driving - Rennlinienplanung im Grenzbereich ✔ ✔ ✔ – - ● ● 2023-08-10 TrackTraction Pro: Design und Entwicklung eines Reifen-Fahrbahn Reibungsmessgeräts ✔ ✔ ✔ – ● ● - 2023-08-10 Autonomous Racing: Antiblockierregelung für open-wheel Rennfahrzeuge ✔ – ✔ – ● ● - 2022-05-03 Indy Autonomous Challenge: Werde für deine Abschlussarbeit Teil von TUM Autonomous Motorsport. Alle Bereiche möglich, inkl. Reise in die USA. – ✔ – – ● ● - 2022-04-29 Schätzung der Fahrbahnneigungswinkel für Autonome Rennfahrzeuge auf Hochgeschwindigkeitsovalen

We believe that self-driving cars can’t come soon enough. If you feel the same way and want to be smart enough to do something about it, then rest assured: you’ve come to the right place.

Thesis topics

So you’re thinking of making the culmination of your studies about self-driving good, because we need all the help we can get. see all the available thesis topics and don’t hesitate to contact us if your idea for a self-driving topic is not listed, flow sensing for applications in autonomous driving, bachelor or masters, naveed muhammad.

The usual sensing modalities that are available to autonomous vehicles i.e. vision, radar and lidar have their limitations. For example, none of them is very suitable for estimating the length of a leading vehicle. Additional sensing modalities, such as wind flow sensing, might have the potential to meaningfully complement the usual sensing modalities in autonomous driving.You will build upon the innovative work done on flow-sensing applications in autonomous driving two previous theses at ADL There are multiple directions that can be investigated. For example, (i) working in simulations and proposing new applications of flow sensing in autonomous driving, (ii) investigating new feature extraction and classification/regression methods etc. or (iii) expanding out of simulations with physical validation of flow sensing in autonomous driving.

Localization

Map-based localization in autonomous driving using lidar, behavior prediction, rethinking the vehicle autonomy with full infrastructural support and connectivity to road agents.

Autonomous vehicles research has progressed rapidly during the last decade, but challenges remain in their wide-spread deployment. Modular as well as end-to-end approaches to vehicle autonomy, alike, suffer from the limitation of not being able to cover edge cases that arise in everyday traffic. At the same time, it is also believed that more and more infrastructural support and interconnectivity between vehicles will be available to autonomous vehicles of the future over time. In this project, you will have the possibility to propose an autonomous-driving system which can get as much information from infrastructure and other road agents (including, but not limited to, other vehicles, pedestrians, bicycles etc.) as needed to be fail safe in all usual or edge situations. After a thorough literature review of the open challenges in vehicle autonomy you will investigate methods for meeting those challenges using connectivity with infrastructure and road agents. You will then validate the proposed system in simulation.

Crossview geolocalization in non-urban environments for autonomous driving

Crossview geolocalization for autonomous driving, syeda zillay nain zukhraf, educational technology, create materials to teach data science via self-driving, ardi tampuu.

Developing a course (8 academic hours one-day seminar or a 3 ECTS course) teaching key aspects of data science in a practical way with self-driving toy cars. The particular emphasis is on aspects that become relevant when deploying a solution to the real world. The toy cars are equipped with a camera and a Raspberry Pi. The used machine learning solution imitates human driving, learning the function between the camera image and the command a human would give when seeing this image. As the task is fun and understandable, the failures are evident (car crashes or behaves weirdly), it is a very good medium to teach data science and machine learning. The course should cover key concepts that get mentioned in every data science, but rarely practically experienced by students, such as the following. (i) Garbage in, garbage out. If your training data is low quality, you get a bad model. Can be exemplified by using too low camera resolution for the task (garbage input) or bad driving examples (garbage training labels). (ii) Overfitting and finding non-causal relationships in the data, which later lead to bad generalization ability. Exemplified by visualizing the image areas that influence the decision the strongest (saliency maps). If top half of the image is not cut away, the model will learn to rely on lamp locations in the ceiling, not track walls for deciding when to turn.  (iii) Models struggle to extrapolate, generalize. Exemplified by models failing to drive if light conditions change. (iv) Test set metrics are not the final product, the model will be deployed in the real world and will find examples it can not deal with. Or will experience a shift in input distribution. Monitoring performance, characterizing failures and reacting to them is important. (v) Dataset management, iteratively building a better dataset (to fix the failures observed), and data cleaning are important. Exemplified by results before and after. (vi) Computational efficiency can be important in certain tasks. If the vehicle takes too few decisions per second it will crash. In other domains - if every query costs too much (on compute bill) or takes too long to compute, the product is not viable. The thesis should make use of known theories of developing educational materials, e.g. define learning objectives (what should the student know) and work backwards from there. In particular for a MSc thesis, the educational theory background is needed for a good mark. MSc thesis would also benefit from running the developed course once with test students. BSc thesis can get away without actual experimentation due to time constraints.

Localizing minicar within Delta 3rd floor hallways

The Autonomous Driving lab owns a dozen DonkeyCar S1 minicars. These vehicles come equipped with a camera and inertial measurement unit, as well as Raspberry Pi. Your task is to localize the vehicle with the best possible accuracy within the third floor of Delta building (starting with only one wing or hallway, for example) using the available sensors. This means the high-level localization (which hallway I’m in) can be based on the proximity of wifi access points, while more precise localization inside a certain hallway can be based on lines detected in the camera image. The supervisors see particle filters as a possible approach to the task, as it allows combining multiple sources of information (north direction from IMU, hallway-precision from wifi, location within hallway by camera). However, the student should complete a thorough literature review on camera-based localization and another method may prove applicable. If the vehicle is capable of localizing itself within the third floor of Delta with sufficient precision, it can follow paths and complete any routes given to it, opening possibilities for further applications, such as delivery of items, security patrols etc.

Autonomous Driving Lab

Discover ADL

• more THI pages
• semester dates
• PRIMUSS Application portal
• PRIMUSS student's portal
• Student Council website

Automated Driving and Vehicle Safety (M. Eng.)

Automated driving has the potential to drastically change the mobility of the future and thus the automotive industry. Due to the broad spectrum of challenges, the development of automated driving is a particularly interdisciplinary field and requires in-depth knowledge of both engineering and computer science. The Master's degree programme, mostly held in German, addresses this need in order to qualify students for the development of systems for safe automated driving. In addition, the theoretical-scientific basis of the students is to be broadened to enable them to pursue a doctorate or work in the scientific field.

The Master's programme "Automated Driving and Vehicle Safety" comprises two full-time semesters. The third semester is used to write the Master's thesis, which can also be written in a company as part of the practical work. The modules of the study programme are mainly held in German and are supplemented by some courses in English.

The modules of the Master's programme were defined with the aim of further qualifying students in three areas important to the development of automated and safe driving systems:

• components: Automated driving has an impact on all areas of the vehicle: drive, brake, steering, display instruments, navigation and sensors as well as networking inside and outside the vehicle. What is needed is a deep understanding of the components of these areas.
• concepts: Automated vehicles must recognize patterns, make smart decisions, and process a variety of data based on image and sensor data to navigate the street. The basis of every development and further development of such systems are knowledge of the concepts of machine learning in connection with those of integrated vehicle safety.
• methods: Of crucial importance for the successful realization of automated driving systems are methods of creating a system architecture or a functional safety concept, methods of test concepts and their management as well as the performance of tests and simulations.

[Translate to English:]

1. semester.

Vehicle Actuators and Environmental Sensors

Car2x-Communication

Knowledge Modeling and Machine Learning

Science and Ethic

2. Semester

System Architectures and their Development

Simulation and Test of Functions of Automated Driving

Software Development for Functions of Autom. Driving

Integral Safety

3. Semester

Master's Thesis

Applying for a degree course The programme starts in summer term (March 15th) as well as in winter term (October 1st). Please find all information on our master application pages

Application periods November 15th – Feb. 15th in case of admission for the summer term; May 2nd – July 31 in case of admission for the winter term

Admission The prerequisite for admission to the Master's programme is a degree in engineering science, computer science or a related field with at least 210 ECTS credits.

Qualified applicants are subject to a suitability procedure in which an overall grade is determined for the ranking of the admission procedure, consisting of

• the final grade of the Bachelor's programme with weight 0.6
• a grade with a weight of 0.4, with which a subject-specific suitability, independent scientific work and experience in the field of the design of mechatronic systems is evaluated and determined on the basis of the following criteria: - Knowledge in the areas of automotive engineering, control engineering, signal processing, sensors / actuators, modeling and simulation of mechatronic systems, electronics, microcomputer technology, programming, methods of artificial intelligence - Practical experience (comparable to a practical semester) in the field of designing or testing mechatronic systems including their software development - Project work in the Bachelor's programme with a thematic reference to the vehicle or methods of vehicle automation, which required knowledge of the areas design of mechatronic vehicle components, control engineering, electronics, modeling / simulation, control devices and their programming, methods of artificial intelligence.

The aptitude is considered to have been determined if the aptitude test is rated at least with the overall grade "good" (2.5).

Admission to the master's program requires a degree in an engineering discipline, computer science or a related field with at least 210 ECTS credits.

Since the main language of instruction is German, applicants from abroad must provide proof of German language skills at level B2. English language skills do not have to be proven separately.

Das duale Masterstudium an der THI bietet optimale Karrierechancen. Der duale Master umfasst neben dem regulären Studium eine intensive Berufspraxis im Unternehmen. Er ist konsekutiv und richtet sich sowohl an nicht duale als auch an duale Bachelorabsolvent/innen, die vorher ein Verbundstudium oder Studium mit vertiefter Praxis absolviert haben..

Während des dualen Masterstudiums erhalten Sie von Ihrem Unternehmen ein Gehalt. Die Höhe der Vergütung bei Masterstudiengängen sollte in angemessener Weise über der Vergütung in Bachelorstudiengängen liegen.

Besonderheit: Normale Studiendauer mit deutlich mehr Praxiserfahrung als im regulären Mastertudium

The job profile of the graduates is characterized by tasks in the areas of research and development of algorithms, safety concepts, sensor/actuator systems for safe, automated driving.

We look forward to receiving your application!

          To the online application                            

Contact for questions about the application or other topics         

Contact only for content-related questions:

Module Handbook

The module handbook (MHB) is the guide to all modules in the Master programme. It contains information on prerequisites, contents, learning outcomes, ECTS points, duration, work load, literature and references.

Module handbook (german language)

Study and Examination Regulations

Study and examination regulations (SPO) deal with judicial matters concerning your study course. Please turn to the pages of the legal department for Statutes of this course.

For any questions on requirements, exams regulations and more, please turn to the Service Center Study Affairs .

Semester dates

Accreditation

NameName