Open topics for theses and practical courses

Cancer organoids are a very useful in vitro model in cancer research because they resemble the actual morphologic and functional features of human or animal cancers. These cancer organoids may be developed in large quantities originating from the same patient cancer, therefore permitting in vitro drug screening for precision medicine. Optical coherence tomography (OCT) has been shown to provide high-throughput imaging of cancer organoids in a non-invasive and label-free fashion for longitudinal monitoring of these organoids. However, tracking each individual organoid over weeks and providing quantitative analysis is very labor intensive and time consuming. Algorithms that can automatically analyze the volumetric images acquired at different time points are urgently required to free up biological and biophotonics researchers. A master thesis project is possible to develop such algorithms for automatic tracking of individual organoid in 3D from OCT data. The scope and details of the project will be framed according to the background of the student in an initial meeting.

Prerequisites: The student needs solid programming skills in Python and background in machine learning and computer vision, e.g., by completing the courses "Foundations of data analysis" and "Image Processing and Image Analysis" or equivalent courses.

Supervisors: Claudia Plant, Kevin Sidak, Lukas Miklautz in collaboration with Mengyang Liu, Abigail Deloria, and Agnes Csiszar from Medical University of Vienna

Contact: Claudia Plant, Kevin Sidak, Lukas Miklautz

Cancer organoids can be cultivated in batches and undergo chemotherapy. The treatment efficacy can be then validated by imaging the treated organoids using fluorescence microscopy. For example, some cancer cells in the cancer organoids may be killed by the anti-cancer drug, and these dead cells can be labeled and show up in green color in fluorescence microscopy whereas the live cancer cells can be labeled otherwise and show up in red color in the fluorescence microscopy. However, fluorescence microscopy only provides 2D imaging and is time consuming in this live/dead classification task. By using optical coherence tomography (OCT), we can image these organoids that are treated by anti-cancer drugs in just a few seconds. However, the reconstructed OCT images only show structural information directly and lack the classification information. We assume that, using fluorescence microscopy as the ground truth, OCT images of the treated cancer organoids can also provide the classification of live and dead cancer cells. This master thesis work involves the development of classification algorithms to differentiate live cancer cells from dead cancer cells in cancer organoids imaged by OCT using the fluorescence image data as the ground truth.

Prerequisites: The student needs solid programming skills in Python and background in machine learning and computer vision, e.g., by completing the courses "Foundations of data analysis" and "Image Processing and Image Analysis" or equivalent courses.

Supervisors: Claudia Plant, Kevin Sidak, Lukas Miklautz in collaboration with Mengyang Liu, Abigail Deloria, and Agnes Csiszar from Medical University of Vienna

Contact: Claudia Plant, Kevin Sidak, Lukas Miklautz

The internal audit department produces yearly thousands of reports. The goal of each report is to highlight and communicate internal issues and criticalities, which are expressed through findings. Findings are presented as short text written in a (semi-)standardized format. Even though reports and therefore findings are already grouped by topic/area, additional and/or independent clustering could be helpful for risk assessment, audit planning, getting an overview. The goal of the project is to compare different cluster methodologies ranging from classical to hierarchical to non-redundant clustering. The student should explore different text representation approaches from classical LDA to modern LLM approaches. Clusters should contain texts describing similar or related issues and should be (at least partially) interpretable. The student is expected to use visualizations techniques, like word clouds, to inspect the clusters.

Important: The student will be onboarded as an intern in the internal audit department at Erste Group Bank AG.

Prerequisites: The student needs solid programming skills in Python, background in machine learning and data mining, e.g., by completing the courses "Foundations of data analysis", "Data Mining" and "Scientific Data Management" or equivalent courses.

Supervisors: Claudia Plant and Lukas Miklautz in collaboration with Stefano Melchionna from Erste Group Bank AG

Contact: Claudia Plant, Lukas Miklautz

Glass beads were among the most common grave goods in the early Middle Ages, and their number can be estimated in the millions. The color, size, shape, production technique and decoration of the beads are diverse and contain much information that is relevant for historians, e.g., regarding trade and production networks. This large scale makes the recording and digitalization of glass beads very labor intensive.

Within this project an image processing pipeline will be developed that consists of several tasks from which one should be implemented by the student. Some of these tasks are

• Image Segmentation
• Image Classification
• Image Clustering
• Image Generation
• Image Upscaling
• Automated Object Counting

The student is expected to implement one of the tasks above and design a simple and easy-to-use graphical interface, e.g., a web-based application. The scope of the project will be discussed in an initial meeting with the supervisors.

Prerequisites: The student needs solid programming skills in Python and background in machine learning and computer vision, e.g., by completing the courses "Foundations of data analysis" and "Image Processing and Image Analysis" or equivalent courses. Additionally, interested students should be eager to participate in an interdisciplinary project on the intersection of data mining, image processing and archeology.

Contact: Claudia Plant, Lukas Miklautz

Performative predictions are predictions supporting decisions that influence the outcome and are ubiquitous in many real-world applications of machine learning techniques. For instance, if a loan applicant is predicted to have an elevated risk of default and as a consequence will be charged higher interest rates, this likely further increase their default risk. Thus it is important to apprehend a prediction-based action's effect on the outcome and factor it in when deciding on the model used for making predictions.

In this project you will study performative prediction when domain knowledge about the taken action's effect is available, e.g., that the default risk increases with at least a certain probability. In this setting, you will develop techniques for selecting good predictive models with the goal of steering the outcome while accomodating constraints, e.g., a maximum loan sum for all customers.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Developing anonymized datasets based on proprietary data for session-based recommendations with Graph Neural Networks (GNNs) is crucial to adhere to GDPR requirements and protect business secrets. Using proprietary data from an industry partner, this thesis will focus on creating a publishable, anonymized dataset that retains the essential characteristics needed for effective session-based recommendation while ensuring no traceability to individual users or sensitive business information. Anonymization techniques such as data aggregation, noise addition, and generalization will be employed. Additionally, synthetic data generation methods will be explored to produce a dataset that mirrors the statistical properties of the original data without compromising confidentiality. Ensuring the preservation of graph structure and session patterns is paramount to maintaining the dataset's utility for GNN research. The project will involve a thorough evaluation of various anonymization and synthetic data generation techniques, with a focus on their effectiveness in maintaining data utility and anonymity. The expected outcome is a robustly anonymized benchmark dataset using state-of-the-art methodology for creating anonymized datasets suitable for research and publication, contributing to the broader field of machine learning on graphs in general and sequential recommendation in particular.

Requirements: Python, interest in data anonymization, interest in graph neural networks.

Contact: Lorenz Kummer, Nils Kriege

Since their introduction, the large language models (LLMs) have been rapidly used in many areas of science. This practical course/thesis will focus an exploration of the capabilities of LLMs in causal reasoning. The interested student is expected to have basic knowledge of statistics and good programming skills in python. Some background in causal inference or causal discovery is a plus point. More details to the interested student will be provided on request.

Contact: Katerina Schindlerova, Aditi Kathpalia

Causal learning methods make several assumptions which are necessary to make useful causal inferences based on given data that go beyond statistical correlations made by conventional learning methods. In real world situations, it is often the case that these assumptions either don’t hold or become too strict or constraining. In this project we would like to explore what kind of inferences are reached if certain assumptions that are not valid for given datasets are made. We would also be interested in exploring if there could be ways to get rid of certain assumptions and still reach useful conclusions.

The interested student is expected to have basic knowledge of statistics and good programming skills in python. Some background in causal inference or causal discovery is a plus point. More details to the interested student will be provided on request.

Contact: Katerina Schindlerova, Aditi Kathpalia

Time-series data is a collection of data points recorded over time, each associated with a specific timestamp. This form of data is prevalent in various fields such as finance, economics, meteorology, healthcare, energy, telecommunications, and transportation. Current algorithms assume that we only have time-series data of the same scaling, but in real-world data time-series often consists of different scalings, e.g. hourly, daily, or weekly weather forecasts.

This project will mainly focus on the development of a clustering algorithm that can handle time series with different scalings. Students working on this project need solid programming skills in Python. More details to the interested student will be provided on request.

Contact: Yllka Velaj, Pascal Weber

Time-series data is a collection of data points recorded over time, each associated with a specific timestamp. This form of data is prevalent in various fields such as finance, economics, meteorology, healthcare, energy, telecommunications, and transportation. Often we can gather additional information about these time series, which could improve cluster performance. This kind of clustering is also called "multi-view clustering". Unfortunately, currently, there are no algorithms for multi-view clustering of time series.

This project will mainly focus on the development of a clustering algorithm that can use additional information from an original time series to retrieve a better clustering. Students working on this project need solid programming skills in Python. More details to the interested student will be provided on request.

Contact: Yllka Velaj, Pascal Weber

The process of automatically learning features is termed representation learning and offers the advantage of eliminating the need to predetermine which features are crucial, often saving considerable effort in practical applications. Deep Clustering is a research domain that seeks to apply the successful principles of deep learning to the field of clustering. Deep clustering algorithms can autonomously extract crucial features. Additionally, deep clustering demonstrates scalability to large, high-dimensional datasets, resulting in more precise cluster assignments and predictions for new, previously unseen data points. Unfortunately, the usage of deep representation learning and deep clustering algorithms on time-series data is currently very limited.

This project will mainly focus on the development of a deep clustering algorithm for time-series data. Students working on this project need solid programming skills in Python. More details to the interested student will be provided on request.

Contact: Yllka Velaj, Pascal Weber

Clustering is a fundamental problem in data mining and allows us to discover a natural grouping of similar entities in a dataset. However, there are only a few studies that analyze the performance of clustering algorithms on time series data.

This project will mainly focus on the implementation of clustering algorithms, and on conducting an experimental study with real-world datasets and synthetic ones. Students working on this project need solid programming skills in Python. More details to the interested student will be provided on request.

Contact: Yllka Velaj, Pascal Weber

Time-series data is a collection of data points recorded over time, each associated with a specific timestamp. This form of data is prevalent in various fields such as finance, economics, meteorology, healthcare, energy, telecommunications, and transportation.

This work will mainly focus on developing interactive software for Exploratory Data Analysis (EDA) which will allow us to understand better the main patterns hidden in the data. Students working on this project need solid programming skills in Python. More details to the interested student will be provided on request.

Contact: Yllka Velaj, Pascal Weber

Graph Neural Networks (GNNs) have significantly enhanced the capability of recommendation systems in e-commerce by leveraging collaborative filtering and session-based methodologies. These advancements have demonstrated improved recommendation accuracy by effectively capturing complex user-item interactions within graph structures. However, the deployment of GNN-based systems at scale encounters substantial challenges, primarily due to their computational intensity and reliance on GPU hardware, which becomes a bottleneck in scenarios dealing with millions of requests per second. This thesis aims to address the scalability issues inherent in GNNs for e-commerce recommendation systems by exploring knowledge distillation techniques. The objective is to transfer knowledge from complex GNN models to simpler, more computationally efficient representations that can be easily managed and scaled on CPU servers and database environments. This research will involve a comprehensive evaluation of existing knowledge distillation methods applied to GNNs, focusing on their effectiveness in preserving the recommendation quality while significantly reducing computational requirements. A novel methodology for fair comparison of these techniques shall be developed, taking into account factors such as recommendation accuracy, computational efficiency, and scalability. The outcome of this thesis is expected to provide a systematic approach for making GNN-based recommendation systems more practical for large-scale e-commerce applications, thereby bridging the gap between advanced machine learning models and their real-world applicability in resource-constrained environments.

Requirements: Python, interest in recommendation systems, interest in knowledge distillation

Contact: Lorenz Kummer, Nils Kriege

Human cells need to precisely regulate the amount of proteins produced from each gene. Many of the regulatory mechanisms are governed by specific properties of the messenger RNA (mRNA). However, only a few of these properties have been studied in detail. This project employs deep learning to identify new and relevant RNA features that impact translation and stability. Deep learning models will be trained to predict the lifetime of an mRNA and its protein expression from mRNA sequence and structure information. The models will then be analyzed using techniques from the field of explainable AI to extract which mRNA features were crucial for the prediction.

Students working on this project should be knowledgeable in machine learning, have solid programming skills, have a desire to work on interpretable deep learning, and be keen to use machine learning to advance science and the discovery of biological knowledge.

Contact: Sebastian Tschiatschek

Inverse reinforcement learning is about identifying the reward function optimized by an agent (e.g., an AI system or a human) through its behavior. This is for instance important to enable the transfer of a reinforcement learning agent’s capabilities to new environments and value alignment. Most commonly, inverse reinforcement learning techniques assume that the agent demonstrating optimized behavior and the agent learning from these demonstrations about the reward function have the same capabilities in terms of actions and observations. This assumption is however unrealistic and you will challenge it in this project.

This project considers inverse reinforcement learning in settings in which there is some form of embodiment mismatch between the expert demonstrator and the learning agent. The scope of the project is to study how existing algorithms perform in this setting and propose modifications to existing algorithms to achieve better performance, e.g., in the form of demonstrations tailored to the learner, learning about the mismatch, or similar.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

While reinforcement learning has shown impressive performance in various applications, the learned agent’s behavior often lacks interpretability and planning is computationally demanding. One avenue to approach these challenges is learning abstractions, i.e., aggregating different states of an environment into a meta-state such that te size of the resulting meta-state space is reduced. Such abstractions are most useful if they satisfy certain properties, e.g., the resulting model is Markovian or allows for a causal interpretation.

In this project you will implement and compare existing abstraction techniques for reinforcement learning and extend a selected one to allow for hierarchical abstraction to enable interpretation and planning at different levels of granularity while ensuring the aforementioned properties.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Explainable Artificial Intelligence (XAI) focuses on providing explanations for domain experts and data scientists. However, 'lay people', i.e., people without or only with limited technical knowledge, are often supported by AI systems in public institutions in their work, e.g., to make decisions about loan applications. Thus it is important that these lay people can understand the AI systems to a certain extent and understand the impact of these systems on the decisions they make.

In this project, you will study how to best make AI systems comprehensible to lay people in sequential decision-making settings, in particular in settings motivated by the usage of AI systems in public institutions. It will involve developing AI models, interpretability techniques, and conducting user studies with lay people.

Contact: Sebastian Tschiatschek

Learning problems for molecular data are often imbalanced, i.e., there is a small subset of molecules having a desired property while most molecules do not have this property. Typically, the minority class of special interest in the considered application. However, most machine-learning algorithms do not work well in such settings. In this project, you will explore and compare several general-purpose methods for imbalanced data and develop specific data augmentation methods for molecular data.

Contact: Nils Kriege

The data regarded in this thesis is from the field of Molecular Dynamics (MD) and describes the positions of a protein's atoms over time. Clustering timesteps allows to detect different states of a protein, which is important for understanding the protein folding process. E.g., these states can serve for a Markov State Model of the protein's trajectories.

In this work, we compare results based on the usually applied molecular descriptors with 3d-Shape descriptors that were recently developed for general point clouds.

Contact: Anna Beer

Early graph neural networks (GNNs) were motivated by circular chemical fingerprints generalizing their concept by introducing learnable parameters. However, recent studies show that current state-of-the-art GNN architectures are still outperformed in molecular property prediction by concrete implementations of circular fingerprints combined with a multilayer perceptron. The task of this project is to explore the reasons for this through a detailed analysis of both methods. You will investigate a concrete implementation of a chemical fingerprint and develop a new neural version starting by reproducing the fingerprint and adding neural modules stepwise. Different stages should be investigated in an ablation study.

Contact: Nils Kriege

Explainable Artificial Intelligence (XAI) focuses on providing explanations for domain experts and data scientist. However, 'lay people', i.e., people without or only with limited technical knowledge, are often subject to the consequences of deploying algorithmic decision-making systems in public institutions, such as the COMPAS recidivism model. Finding new ways to render machine learning models comprehensible for lay people is therefore a current and highly relevant challenge. This project will focus on approaches for building machine learning models that are in themselves interpretable and explainable for lay people, soliciting slight or no external explanations. It will involve both programming and the conducting of user studies with lay people.

Contact: Timothée Schmude, Sebastian Tschiatschek

Auxiliary prediction tasks have been the source of many recent performance improvements for reinforcement learning agents, particularly when working with image inputs. These tasks are usually provided as expert knowledge by the algorithm designer and force the agent to pay attention to various aspects of the environment which might be helpful to solving the underlying task. In this project you will investigate whether such prediction tasks can also be learned by the agent itself. For this, you will first visualize what an RL agent actually "attends to" through investigating heatmaps of their first network layer. Then you will devise a method allowing the agent itself to decide which pixels of the state it wants to learn to reconstruct.

Required skills: Good programming skills, basic background in deep learning

Contact: Timo Klein, Sebastian Tschiatschek

The graph edit distance (GED) quantifies the dissimilarity of two graphs as the minimum cost of a sequence of edit operations turning one graph into the other. The problem complexity depends on the choice of the edit operation costs and the properties of the considered graphs. It has been shown that the classical NP-hard subgraph isomorphism problem (SI) and the maximum common subgraph problem (MCS) can be reduced to computing the GED. As a consequence, computing the GED also is NP-hard. However, for SI and MCS a fine distinction of the complexity depending on parameters such as treewidth, degree, and their combination is known. The goal of this project is to investigate the complexity of computing the graph edit distance in restricted graph classes identifying polynomial-time solvable and NP-hard cases.

Requirements: Strong interest in graph algorithms and theoretical computer science

Contact: Nils Kriege

In this project/thesis you will study the potential of using reinforcement learning (RL) for improving mental health treatments. In particular, you will investigate the potential of RL algorithms for optimizing exposure therapy protocols, which are usually used for the treatment of anxiety disorders, such as phobias.

Specifically, you will conduct simulation studies in which a computational associative learning model (such as the latent cause model) will be a stand-in for the patients, and where an RL algorithms will make modifications to the exposure protocol with the goal of robustly extinguishing negative associations, which play a crucial role in anxiety disorders. The investigation can start with standard model-free RL approaches, but is then planned to proceed with model-based augmentations, such as equipping the RL algorithm with "Theory-of-Mind"-type capabilities.

Students working on this project need basic background knowledge in machine learning, solid programming skills, a desire to work on reinforcement learning, and willingness to learn about models of human cognition.

Contact: Sebastian Tschiatschek

Uncertain graphs represent an important data model for real-world networks, where one can assign an independent probability of existence on every edge. Uncertainty, represented by the edge probability, may arise due to several reasons, e.g., errors in measurements, data integration from inconsistent and ambiguous sources, lack of precise information needs, inference and prediction models, and explicit manipulation for privacy purposes. Due to their rich expressiveness and utility in a wide range of applications, uncertain graphs have prompted a great deal of research by the data mining research community.

This work will mainly focus on designing efficient algorithms for selecting and modifying the probability of a given number of edges in an uncertain graphs. The developed algorithm will be evaluated against other existing methods. An experimental study will be conducted on real-world datasets. Students working on this project need basic knowledge in graph theory and algorithms, and solid programming skills in Python, C/C++ or Java.

Contact: Yllka Velaj, Claudia Plant

For two given graphs, G and H, the maximum common subgraph problem (MCS) asks for the largest graph contained in both G and H. An important application occurs in cheminformatics, where the similarity of molecular graphs needs to be quantified. Unfortunately, MCS is NP-hard unless additional constraints regarding the properties of G, H, and the subgraph are enforced. A polynomial-time solvable case requires that all these graphs are trees. Known negative complexity results for classes of tree-like graphs set narrow boundaries for generalization.

The project aims to investigate particular tree-like graph classes to design, implement, and experimentally evaluate new polynomial-time algorithms. New NP-hardness proofs for specific cases should refine the complexity status further.

Requirements: Strong interest in algorithms and graph theory

Contact: Nils Kriege

A graph isomorphism is a mapping between the nodes of two graphs that preserves the edges. An automorphism is an isomorphism of a graph to itself. The (non-trivial) automorphisms intuitively represent the symmetries of a graph. Two vertices are considered equivalent if there is an automorphism that maps one vertex to the other. The corresponding equivalence classes are referred to as orbit partition. The orbit partition is of interest for symmetry breaking techniques for various combinatorial problems. Identifying and pruning symmetric branches in the search tree based on the symmetries of the input instance can lead to drastic speed-ups. This project aims to develop dynamic algorithms for maintaining the orbit partition when the input graph changes. The focus of the project can be on the development and theoretical analysis of algorithms or their implementation and experimental evaluation.

Requirements: Strong interest in algorithms

Contact: Nils Kriege, Christian Schulz, Kathrin Hanauer

In this project you will study novel approaches for learning interpretable and explainable deep learning models with the goal of supporting users with different skill levels and knowledge. The models will be evaluated on applications with large societal impact, e.g., labour market data.

Contact: Sebastian Tschiatschek

In many areas of our life questionnaires/a collection of measurements are used to gather information for decision making, e.g., questionnaires are used to measure the success of marketing campaigns or customer satisfaction, or medical tests are conducted to decide on the best treatment for a patient. Common to those applications is that many questions need to be answered or many tests must be conducted. Often this is not because all the information is needed for deriving a decision but rather because the course of action is standardized (e.g., a survey always consists of the same questions).

In this project you will evaluate and develop methods for adaptive information acquisition that acquire the information necessary for decision making sequentially and adaptively, e.g., not all questions in a questionnaire are asked in all cases and the order of the questions can be changed. You will consider and evaluate recent machine learning models for this problem but also investigate how these methods can be extended to account for dependencies in the information that can be required, e.g., dependencies that only allow for a specific order of some questions of a questionnaire, the influence of how questions are asked on the answers, etc.

Contact: Sebastian Tschiatschek

Clustering is the task of finding groups in a data set. As an unsupervised learning approach it is important for exploratory data analysis, e.g., for finding commonalities between patients suffering from some disease. Often such data is high-dimensional and contains complicated non-linear relationships, which makes it difficult to apply standard clustering approaches. Deep clustering algorithms have been proposed recently to approach this kind of setting by combining clustering and deep learning in a common framework. An open problem that has not been sufficiently addressed in recent deep clustering research is incomplete and heterogeneous (e.g. survey data, blood tests, MRI images, etc.) data. Incomplete data is often an issue in practice, especially for health data where not everything about a patient’s history is known.

In this project you will study the clustering of incomplete data with heterogeneous data types with a particular focus on missingness and on determining which information is relevant for cluster assignments. You will develop an algorithm based on previous work and evaluate it on synthetic and real-world data. Some research questions of interest would be for instance: How do cluster assignments change when given new information? Or, given an uncertain cluster assignment (e.g. sick or healthy patient) which information would we need to query to make a decision?

Contact: Sebastian Tschiatschek

Enabling multiple interacting agents to convey their knowledge and skills through teaching.

Contact: Sebastian Tschiatschek

Learning abstractions of large state and action spaces in order to facilitate more efficient planning and exploration.

Contact: Sebastian Tschiatschek

Bivariate causal measures: Experiments with existing causal methods in python on synthetic and real data.

Contact: Katerina Schindlerova

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Reinforcement learning has shown remarkable successes in playing board games like Go but also in playing computer games like DOTA. While these results are impressive, the learned strategies (policies) are typically not directly interpretable by humans. However, interpretability is important for instance to enable better collaboration of humans with AI agents or to assess safety of the learned behavior.

In this project you will make such learned policies easier to understand by humans by incorporating constraints on their structure. You will develop and implement a novel policy training algorithm and evaluate it on simple computer games.

Students wanting to work on this topic are expected to have solid coding skills in Python, basic knowledge in PyTorch or TensorFlow, and be curious to learn about reinforcement learning and its applications.

Contact: Sebastian Tschiatschek

Reinforcement learning has shown remarkable successes in application domains like game play and robotics. These successes where achieved by performing large numbers of interactions of a learning agent with the environment. In these interactions, the agent constantly receives information about rewarding behavior. However, in many applications involving human users (e.g., personal assistants), the reward information is explicitly or implicitly provided by these human users. Therefore, this information must be treated as a scarce and valuable resource, and the (cognitive) cost of providing this information should be accounted for.

In this project you will study the cost of providing different types of feedback used for reinforcement learning in user studies. To this end, you will implement a simple reinforcement learning environment for which a human user can provide reward information in various forms (e.g., comparisons, sorted lists, etc.) and compare the cost of providing this feedback as well as the usefulness of the feedback for training the reinforcement learning agent. Depending on whether you are doing a P1/P2/Bachelor Thesis/Master Thesis, the scope of the project will be adjusted.

Students wanting to work on this topic are expected to have solid coding skills in Python, basic knowledge about the development of interactive web pages, and be curious to learn about reinforcement learning and its applications.

Contact: Sebastian Tschiatschek

Recent advances in reinforcement learning have enabled super-human performance of AI agents in many applications, e.g., game play. In this project you will work on building theory and/or models for reinforcement learning from implicit and explicit feedback. In particular, you will consider the setting in which no rewards are observed during execution of the agent but only cumulative reward information is obtained at the end of the execution. As a practical example, assume playing a computer game in which you cannot observe the score while you play the game but only see your achieved score at the very end. This setting has important applications in human-in-the-loop settings in which feedback can be expensive to obtain.

Students working on this project need basic background knowledge in machine learning, solid programming skills in Python, and the desire to work on reinforcement learning.

Contact: Sebastian Tschiatschek

In this project you will work on predicting students' responses to mathematical questions using machine learning models. This is important to assess a student's skills and provide personalized educational resources. We use real-world data provided by the "Diagnostic Questions: Predicting Student Responses and Measuring Question Quality" challenge at NeurIPS'20. The challenge consists of four different tasks from which a subset can be selected for the project, depending on background knowledge and programming skills.

Students working on this project need basic background knowledge in machine learning, programming skills in Python, and the desire to develop and test machine learning models.

Contact: Sebastian Tschiatschek

Automated sequential decision making is an important application of machine learning systems in which such a system needs to select a sequence of actions step by step to optimize a reward/utility function. For instance, in autonomous driving, such a system needs to execute a sequence of steering, breaking and acceleration actions, or in a medical intensive care setting, such a system needs to execute a sequence of measurement and treatment actions.

One challenge in realizing such automated sequential decision making systems is the definition of the reward/utility function. For example, in autonomous driving it is hard to specify all the factors which define good driving behavior. In such settings, automatically inferring the reward/utility function from users’ feedback can be beneficial.

This project investigates approaches for reward/utility inference from diverse and implicit feedback, building on ideas for inverse reinforcement learning, active learning, implicit feedback, etc.

Interested students are expected to have solid mathematical and machine learning skills, and have experience in Python and deep learning (using PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Reinforcement learning has been successfully used to solve certain challenging sequential decision making problems in recent years. The employed techniques commonly require (i) huge amounts of interactions with the envirnoment and (ii) clearly specified reward signals to work well. In many applications however, one or both of these requirements are not met. In such cases, imitation learning can be an efficient approach to sequential decision making problems: an expert demonstrates near-optimal behavior and a learning agent attempts to mimic this behavior.

This project considers imitation learning in settings in which there is some form of mismatch between the expert demonstrator and the learning agent. The scope of the project is to study how existing algorithms perform in this setting and proposes modifications to existing algorithms to achieve better performance.

Students wanting to work on this topic are expected to have a basic understanding of machine learning techniques, solid knowledge of Python and basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Variational Autoencoders (VAEs) are powerful deep generative models that have been successfully applied in a wide range of machine learning applications. Recently, the Partial VAE (PVAE), a variant of VAEs that can process partially observed inputs has been proposed and its effectiveness for data imputation has been demonstrated. Key to the fast training of VAEs and PVAEs is the amortized prediction of posterior distributions from observations. In PVAEs, these posterior distributions are predicted from partial observations.

This project aims at studying the consistency of these posterior distributions for different patterns of missing data. The insights are used to create/train better inference models and thereby improve the quality of PVAEs.

Students wanting to work on this topic are expected to have solid mathematical skills, a basic understanding of machine learning techniques and good programming skills in Python.

Contact: Sebastian Tschiatschek

This work will focus mainly on implementation of an algorithm for causal detection among processes as well as its testing on probability distributions with heavy tailed distributions and on climatological data provided by ZAMG (Die Zentralanstalt für Meteorologie und Geodynamik) following these distributions. More details to the interested student will be provided on request.

Contact: Katerina Schindlerova