Open topics for theses and practical courses

Deep Reinforcement Learning (RL) algorithms currently use widely used optimizers from deep learning, such as Adam or SGD, despite having notably different optimization characteristics. For example, DQN solves a series of related optimization tasks instead of a single task, which can lead to misaligned gradient updates when using the Adam optimizer. In this project, you will investigate whether recently proposed optimizers such as SOPHIA, Adam-atan2, AdaBelief, or Lion can improve deep RL performance. For this, you will implement these algorithms in Pytorch, train deep RL agents using them, and track various metrics such as gradient norms and the number of dead neurons.

Students working on this topic should have good programming skills and familiarity with the optimizers used in deep learning, such as SGD and Adam. Knowledge of deep RL algorithms such as DQN or Actor-Critic is an advantage.

Contact: Timo Klein, Sebastian Tschiatschek

Layer Normalization (LayerNorm) has recently been used with great benefits in deep Reinforcement Learning (RL). It is particularly effective at mitigating a deep RL optimization issue called plasticity loss, where the agent's network loses its ability to learn. In this project, you will examine which components of LayerNorm are responsible for boosting performance in deep RL. This means you will implement LayerNorm for different deep RL algorithms and perform ablation studies on its components. You will also investigate the effect of LayerNorm when using deep neural networks for regression tasks, which are used in DQN and other algorithms.

Students working on this topic should have good programming skills and knowledge of deep RL algorithms such as DQN or Actor-Critic.

Contact: Timo Klein, Sebastian Tschiatschek

Deep Reinforcement Learning (RL) algorithms using Q-Learning often have built-in exploration mechanisms based on multi-armed bandits, for example, epsilon-greedy. However, recent work has shown that optimization side-effects play a larger role in DQN exploration than epsilon-greedy. This is called policy churn. In this project, you will investigate how exploration is affected when a DQN agent has lost its ability to learn. In jargon, this amounts to drawing a connection between policy churn and plasticity loss.

Students working on this topic should have good programming skills and knowledge of value-based deep RL.

Contact: Timo Klein, Sebastian Tschiatschek

Graph Neural Networks (GNNs) are powerful tools for learning from graph-structured data. However, a largely unexplored area is how numerical errors from floating-point arithmetic affect GNN computations and expressivity—the model’s ability to distinguish between different graph structures. Since GNNs rely on operations like summation to aggregate node embeddings, their performance is impacted by the non-associativity of floating-point arithmetic, where the order of operations can lead to different results.

For example, using Float32 arithmetic, consider two nodes, v and u, with their neighbors' embeddings: v has {1e-8, 1, -1} and u has {1, -1}. Depending on how summation is grouped, the aggregation can differ: (1e-8 + (1 - 1)) equals 1e-8, while ((1e-8 + 1) - 1) equals 0. Such inconsistencies, influenced by computation order or node sequence (e.g., in edge_index from torch geometric [2]), may randomly determine whether the nodes are distinguishable.

The goal of this thesis is to explore the practical impact of these floating-point errors on GNN expressivity in real-world datasets. Specifically, the work will:

• Analyze how GNN expressivity is affected by numerical precision in operations like summation.
• Conduct experiments to quantify how often floating-point errors influence graph distinguishability.
• Explore mitigation strategies such as feature transformations, weight adjustments, and control over aggregation order.

This research will provide a better understanding of how floating-point precision affects GNN performance in practice, offering practical solutions to improve model reliability.

Requirements: Python proficiency, interest in numerical computing and floating-point arithmetic, familiarity with GNNs, and experience with graph processing libraries like torch geometric.

Contact: Lorenz Kummer, Nils Kriege

Chemical molecules can be abstracted as graphs where nodes represent atoms and edges represent bonds. Each node is assigned a feature vector containing information about the atom. For example the type of atom (carbon), the number of bonds (eg. 3), etc. However there exists no study that investigates the importance of these features to perform better property predictions. Often, all features provided from a chemical software package are taken without much thought about the potential of overfitting or the presence of redundant features.

During this project, different sets of node features and edge features will be investigated (based on chemical reasoning), ultimately leading to a justified explanation of which information is required to achieve optimal results.

Contact: Anatol Ehrlich

Transfer-learning is the process of using pre-trained weights from one-model to make better predictions with another model. In the chemical domain, labeled data is sparse and self-supervised methods using unlabeled data are used to obtain initial weights. While transferring these weights to a fine-tuning model, several questions need to be investigated: Should all layers be frozen? Or only some of them? Does this depend on the amount of layers or the network used? During this project you will investigate the factors that allow for the optimal transfer of weights from a pre-trained to a fine-tuning model.

Contact: Anatol Ehrlich

Understanding the effectiveness and quality of outputs from Large Language Models (LLMs) is crucial. This project focuses on qualitative research involving an LLM integrated with a Retrieval-Augmented Generation (RAG) system. The objective is to explore and identify meaningful metrics to evaluate the system's performance and effectiveness in real-world applications. This involves evaluating potential metrics, selecting the most relevant ones, and thoroughly investigating how these metrics can be measured and interpreted. The project includes designing a study, gathering and analyzing qualitative data through methods such as interviews, and developing insights that could shape the future development and deployment of the LLM-RAG system. This project offers a unique opportunity to explore practical applications of an LLM-RAG system. The aim of the model is to support experts in answering legal questions more quickly and effectively. Also relevant is the consideration of work, the change of work and the organization of work through the introduction of the model.

Contact: Sebastian Tschiatschek, Torsten Möller

In single-cell gene expression analysis, we measure the levels of genes in each cell within a biological sample. Among these genes, transcription factors play a critical role as they can regulate the expression of other genes.

This project involves several key tasks:

• Extracting patterns or rules within gene regulation
• Investigating combinations of regulators, specifically transcription factors, and their effects on gene expression
• Comparing gene targets across different species

Students participating in this project will need programming skills in Python. An interest in the interdisciplinary field of bioinformatics is advantageous. Further details on the project are available upon request and will be discussed during an initial meeting. A first overview of the topic can be found under this link (https://ucloud.univie.ac.at/index.php/s/4zLMbNQZ2anT5Cn).

Contact: Carolina Atria

In the interaction between computer science and molecular biology, a clear overview of provided data is crucial. Your task will be to build a website that enables interactive explanation and to help provide an overview of the different datasets, including:

• Connection to a glossary for definitions of technical terms
• Embedding SVG figures for easier explanation of underlying models

Familiarity with JavaScript, and especially the D3 library, is necessary for the implementation of this project. Further details on the project are available upon request and will be discussed during an initial meeting. A first overview of the topic can be found under this link (https://ucloud.univie.ac.at/index.php/s/4zLMbNQZ2anT5Cn).

Contact: Carolina Atria

The Kuramoto model for synchronisation can be used to calculate connectivity matrices from EEG recordings. They describe the functional interplay between brain regions. We derive these matrices from EEG recordings of each individual of a cohort of patients suffering from depression before and after 1 week of antidepressant therapy. We aim to predict the treatment outcome (responders vs. non-responders) with the connectivity matrices as our features. Possible analysis tools range from statistical tests to graph mining methods.

Details about the EEG data structure, the connectivity matrices and the goal of the analysis can be found in this document.

Contact: Lena Bauer, Yllka Velaj

With heat accounting for 50% of global energy demand, solar-thermal plants could play a crucial role in the transition to renewable energy. One of the most important questions for operators is whether the current heat production matches expectations. However, predicting the expected yield is challenging due to the constantly changing weather and demand, which significantly influence heat production. Additionally, seasonal changes in operation add to the complexity.

Do you want to contribute to a greener future? Leverage your knowledge of machine learning or physically-motivated algorithms to improve the current yield prediction. Join us in building on the successful collaboration between the University of Vienna and SOLID. Specifically, you would be embedded into the company through a (paid) internship.

Contact: Sebastian Tschiatschek

With heat accounting for 50% of global energy demand, solar-thermal plants could play a crucial role in the transition to renewable energy. Effective monitoring is essential to quickly identify issues and maintain optimal performance. However, existing fault detection algorithms lack precision. The constantly changing operating conditions (e.g., weather, demand) and time-dependent factors (e.g., fluid heat-up) complicate fault detection for both experts and algorithms.

Do you want to contribute to a greener future? Leverage your knowledge of machine learning or physically-motivated algorithms to improve fault detection. Join us in building on the successful collaboration between the University of Vienna and SOLID. Specifically, you would be embedded into the company through a (paid) internship.

Contact: Sebastian Tschiatschek

Cancer organoids are a very useful in vitro model in cancer research because they resemble the actual morphologic and functional features of human or animal cancers. These cancer organoids may be developed in large quantities originating from the same patient cancer, therefore permitting in vitro drug screening for precision medicine. Optical coherence tomography (OCT) has been shown to provide high-throughput imaging of cancer organoids in a non-invasive and label-free fashion for longitudinal monitoring of these organoids. However, tracking each individual organoid over weeks and providing quantitative analysis is very labor intensive and time consuming. Algorithms that can automatically analyze the volumetric images acquired at different time points are urgently required to free up biological and biophotonics researchers. A master thesis project is possible to develop such algorithms for automatic tracking of individual organoid in 3D from OCT data. The scope and details of the project will be framed according to the background of the student in an initial meeting.

Prerequisites: The student needs solid programming skills in Python and background in machine learning and computer vision, e.g., by completing the courses "Foundations of data analysis" and "Image Processing and Image Analysis" or equivalent courses.

Supervisors: Claudia Plant, Kevin Sidak, Lukas Miklautz in collaboration with Mengyang Liu, Abigail Deloria, and Agnes Csiszar from Medical University of Vienna

Contact: Claudia Plant, Kevin Sidak, Lukas Miklautz

Cancer organoids can be cultivated in batches and undergo chemotherapy. The treatment efficacy can be then validated by imaging the treated organoids using fluorescence microscopy. For example, some cancer cells in the cancer organoids may be killed by the anti-cancer drug, and these dead cells can be labeled and show up in green color in fluorescence microscopy whereas the live cancer cells can be labeled otherwise and show up in red color in the fluorescence microscopy. However, fluorescence microscopy only provides 2D imaging and is time consuming in this live/dead classification task. By using optical coherence tomography (OCT), we can image these organoids that are treated by anti-cancer drugs in just a few seconds. However, the reconstructed OCT images only show structural information directly and lack the classification information. We assume that, using fluorescence microscopy as the ground truth, OCT images of the treated cancer organoids can also provide the classification of live and dead cancer cells. This master thesis work involves the development of classification algorithms to differentiate live cancer cells from dead cancer cells in cancer organoids imaged by OCT using the fluorescence image data as the ground truth.

Prerequisites: The student needs solid programming skills in Python and background in machine learning and computer vision, e.g., by completing the courses "Foundations of data analysis" and "Image Processing and Image Analysis" or equivalent courses.

Supervisors: Claudia Plant, Kevin Sidak, Lukas Miklautz in collaboration with Mengyang Liu, Abigail Deloria, and Agnes Csiszar from Medical University of Vienna

Contact: Claudia Plant, Kevin Sidak, Lukas Miklautz

Performative predictions are predictions supporting decisions that influence the outcome and are ubiquitous in many real-world applications of machine learning techniques. For instance, if a loan applicant is predicted to have an elevated risk of default and as a consequence will be charged higher interest rates, this likely further increase their default risk. Thus it is important to apprehend a prediction-based action's effect on the outcome and factor it in when deciding on the model used for making predictions.

In this project you will study performative prediction when domain knowledge about the taken action's effect is available, e.g., that the default risk increases with at least a certain probability. In this setting, you will develop techniques for selecting good predictive models with the goal of steering the outcome while accomodating constraints, e.g., a maximum loan sum for all customers.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Developing anonymized datasets based on proprietary data for session-based recommendations with Graph Neural Networks (GNNs) is crucial to adhere to GDPR requirements and protect business secrets. Using proprietary data from an industry partner, this thesis will focus on creating a publishable, anonymized dataset that retains the essential characteristics needed for effective session-based recommendation while ensuring no traceability to individual users or sensitive business information. Anonymization techniques such as data aggregation, noise addition, and generalization will be employed. Additionally, synthetic data generation methods will be explored to produce a dataset that mirrors the statistical properties of the original data without compromising confidentiality. Ensuring the preservation of graph structure and session patterns is paramount to maintaining the dataset's utility for GNN research. The project will involve a thorough evaluation of various anonymization and synthetic data generation techniques, with a focus on their effectiveness in maintaining data utility and anonymity. The expected outcome is a robustly anonymized benchmark dataset using state-of-the-art methodology for creating anonymized datasets suitable for research and publication, contributing to the broader field of machine learning on graphs in general and sequential recommendation in particular.

Requirements: Python, interest in data anonymization, interest in graph neural networks.

Contact: Lorenz Kummer, Nils Kriege

CAUSEEFFECTPAIRS, introduced by Mooij et al. in 2016 has been an established benchmark data set used for evaluating bivariate causal discovery methods. The data set consists of 100 different cause - effect pairs selected from 37 data sets from various domains (e.g., meteorology, biology, medicine, etc.) This benchmark contains a heterogenous selection of cause-effect pairs with distinct underlying the data generation. The objective of the work will be to disentangle the source of errors of the state-of-the-art causal inference methods based on log-likelihood scoring and independence testing. The interested student is expected to have a good knowledge of statistics and good programming skills in python. Some background in causal inference or causal discovery is a plus point. More details to the interested student will be provided on request.

Contact: Katerina Schindlerova, Alexander Marx

Causal learning methods make several assumptions which are necessary to make useful causal inferences based on given data that go beyond statistical correlations made by conventional learning methods. In real world situations, it is often the case that these assumptions either don’t hold or become too strict or constraining. In this project we would like to explore what kind of inferences are reached if certain assumptions that are not valid for given datasets are made. We would also be interested in exploring if there could be ways to get rid of certain assumptions and still reach useful conclusions.

The interested student is expected to have basic knowledge of statistics and good programming skills in python. Some background in causal inference or causal discovery is a plus point. More details to the interested student will be provided on request.

Contact: Katerina Schindlerova, Aditi Kathpalia

Graph Neural Networks (GNNs) have significantly enhanced the capability of recommendation systems in e-commerce by leveraging collaborative filtering and session-based methodologies. These advancements have demonstrated improved recommendation accuracy by effectively capturing complex user-item interactions within graph structures. However, the deployment of GNN-based systems at scale encounters substantial challenges, primarily due to their computational intensity and reliance on GPU hardware, which becomes a bottleneck in scenarios dealing with millions of requests per second. This thesis aims to address the scalability issues inherent in GNNs for e-commerce recommendation systems by exploring knowledge distillation techniques. The objective is to transfer knowledge from complex GNN models to simpler, more computationally efficient representations that can be easily managed and scaled on CPU servers and database environments. This research will involve a comprehensive evaluation of existing knowledge distillation methods applied to GNNs, focusing on their effectiveness in preserving the recommendation quality while significantly reducing computational requirements. A novel methodology for fair comparison of these techniques shall be developed, taking into account factors such as recommendation accuracy, computational efficiency, and scalability. The outcome of this thesis is expected to provide a systematic approach for making GNN-based recommendation systems more practical for large-scale e-commerce applications, thereby bridging the gap between advanced machine learning models and their real-world applicability in resource-constrained environments.

Requirements: Python, interest in recommendation systems, interest in knowledge distillation

Contact: Lorenz Kummer, Nils Kriege

Within the Post Processing unit at GeoSphere Austria we aim to continually update and improve our methods with state-of-the-art techniques in AI and machine learning.  Coming mainly from a meteorological perspective and expertise, we offer - supported by the data mining group - to co-supervise student projects with a fairly open choice of topic within our working field. Our unit works on enhancing numerical forecasts by statistical approaches, within the recent years evolving more towards advanced machine learning. Primary use cases involve supporting the renewable energy sector. Our technology stack is mainly based on Python including backends written in C++ or Fortran tackling heavy tasks. Within a student's work Python (enabling the use of state-of-the-art libraries such as TensorFlow, PyTorch, etc.) is the preferred working environment. Within this frame, you are free to choose a topic fitting your background, research interest, course emphasis and extent (bachelor's thesis, PR, master's thesis). Depending on your background recent machine learning approaches or more classical statistic and data mining approaches from literature may be applied – if needed computing resources will be arranged by the University.   Also, we encourage you to split/follow up a topic over several courses, such as PR and thesis.  Bringing your own ideas or building on your individual background is very welcome will be supported, however, note that we cannot offer payment, office space, or resources.

If you like to work with us, please send us a message to arrange an online meeting to set your topic - at least two weeks before the end of course unenrolment period (u:space-Abmeldung). We will discuss possible topics and basic questions on your interest and previous experiences with statistics and machine learning (send ahead via mail before meeting, please read/prepare). After our discussion we will provide the most important key-words from our discussion (take your own notes, too), indicate literature, and data-sets suitable for the work. You shall afterwards write a 0.5 - 1 page abstract about your chosen working topic (including research objective, data-sets, models used) to agree with your official supervisor (Prof. Claudia Plant) one day before the end of course unenrolment period.

Apply until: 2 weeks before the end of course unenrolment period (u:space-Abmeldung)
Note: Due to external supervising finish your work at least 3 weeks before end of semester!

Contact: Petrina Papazek, www.geosphere.at

Human cells need to precisely regulate the amount of proteins produced from each gene. Many of the regulatory mechanisms are governed by specific properties of the messenger RNA (mRNA). However, only a few of these properties have been studied in detail. This project employs deep learning to identify new and relevant RNA features that impact translation and stability. Deep learning models will be trained to predict the lifetime of an mRNA and its protein expression from mRNA sequence and structure information. The models will then be analyzed using techniques from the field of explainable AI to extract which mRNA features were crucial for the prediction.

Students working on this project should be knowledgeable in machine learning, have solid programming skills, have a desire to work on interpretable deep learning, and be keen to use machine learning to advance science and the discovery of biological knowledge.

Contact: Sebastian Tschiatschek

Inverse reinforcement learning is about identifying the reward function optimized by an agent (e.g., an AI system or a human) through its behavior. This is for instance important to enable the transfer of a reinforcement learning agent’s capabilities to new environments and value alignment. Most commonly, inverse reinforcement learning techniques assume that the agent demonstrating optimized behavior and the agent learning from these demonstrations about the reward function have the same capabilities in terms of actions and observations. This assumption is however unrealistic and you will challenge it in this project.

This project considers inverse reinforcement learning in settings in which there is some form of embodiment mismatch between the expert demonstrator and the learning agent. The scope of the project is to study how existing algorithms perform in this setting and propose modifications to existing algorithms to achieve better performance, e.g., in the form of demonstrations tailored to the learner, learning about the mismatch, or similar.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

While reinforcement learning has shown impressive performance in various applications, the learned agent’s behavior often lacks interpretability and planning is computationally demanding. One avenue to approach these challenges is learning abstractions, i.e., aggregating different states of an environment into a meta-state such that te size of the resulting meta-state space is reduced. Such abstractions are most useful if they satisfy certain properties, e.g., the resulting model is Markovian or allows for a causal interpretation.

In this project you will implement and compare existing abstraction techniques for reinforcement learning and extend a selected one to allow for hierarchical abstraction to enable interpretation and planning at different levels of granularity while ensuring the aforementioned properties.

Students wanting to work on this topic are expected to have a solid basic understanding of machine learning techniques, a solid knowledge of Python, and a basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Explainable Artificial Intelligence (XAI) focuses on providing explanations for domain experts and data scientists. However, 'lay people', i.e., people without or only with limited technical knowledge, are often supported by AI systems in public institutions in their work, e.g., to make decisions about loan applications. Thus it is important that these lay people can understand the AI systems to a certain extent and understand the impact of these systems on the decisions they make.

In this project, you will study how to best make AI systems comprehensible to lay people in sequential decision-making settings, in particular in settings motivated by the usage of AI systems in public institutions. It will involve developing AI models, interpretability techniques, and conducting user studies with lay people.

Contact: Sebastian Tschiatschek

Explainable Artificial Intelligence (XAI) focuses on providing explanations for domain experts and data scientist. However, 'lay people', i.e., people without or only with limited technical knowledge, are often subject to the consequences of deploying algorithmic decision-making systems in public institutions, such as the COMPAS recidivism model. Finding new ways to render machine learning models comprehensible for lay people is therefore a current and highly relevant challenge. This project will focus on approaches for building machine learning models that are in themselves interpretable and explainable for lay people, soliciting slight or no external explanations. It will involve both programming and the conducting of user studies with lay people.

Contact: Timothée Schmude, Sebastian Tschiatschek

The graph edit distance (GED) quantifies the dissimilarity of two graphs as the minimum cost of a sequence of edit operations turning one graph into the other. The problem complexity depends on the choice of the edit operation costs and the properties of the considered graphs. It has been shown that the classical NP-hard subgraph isomorphism problem (SI) and the maximum common subgraph problem (MCS) can be reduced to computing the GED. As a consequence, computing the GED also is NP-hard. However, for SI and MCS a fine distinction of the complexity depending on parameters such as treewidth, degree, and their combination is known. The goal of this project is to investigate the complexity of computing the graph edit distance in restricted graph classes identifying polynomial-time solvable and NP-hard cases.

Requirements: Strong interest in graph algorithms and theoretical computer science

Contact: Nils Kriege

In this project/thesis you will study the potential of using reinforcement learning (RL) for improving mental health treatments. In particular, you will investigate the potential of RL algorithms for optimizing exposure therapy protocols, which are usually used for the treatment of anxiety disorders, such as phobias.

Specifically, you will conduct simulation studies in which a computational associative learning model (such as the latent cause model) will be a stand-in for the patients, and where an RL algorithms will make modifications to the exposure protocol with the goal of robustly extinguishing negative associations, which play a crucial role in anxiety disorders. The investigation can start with standard model-free RL approaches, but is then planned to proceed with model-based augmentations, such as equipping the RL algorithm with "Theory-of-Mind"-type capabilities.

Students working on this project need basic background knowledge in machine learning, solid programming skills, a desire to work on reinforcement learning, and willingness to learn about models of human cognition.

Contact: Sebastian Tschiatschek

Uncertain graphs represent an important data model for real-world networks, where one can assign an independent probability of existence on every edge. Uncertainty, represented by the edge probability, may arise due to several reasons, e.g., errors in measurements, data integration from inconsistent and ambiguous sources, lack of precise information needs, inference and prediction models, and explicit manipulation for privacy purposes. Due to their rich expressiveness and utility in a wide range of applications, uncertain graphs have prompted a great deal of research by the data mining research community.

This work will mainly focus on designing efficient algorithms for selecting and modifying the probability of a given number of edges in an uncertain graphs. The developed algorithm will be evaluated against other existing methods. An experimental study will be conducted on real-world datasets. Students working on this project need basic knowledge in graph theory and algorithms, and solid programming skills in Python, C/C++ or Java.

Contact: Yllka Velaj, Claudia Plant

For two given graphs, G and H, the maximum common subgraph problem (MCS) asks for the largest graph contained in both G and H. An important application occurs in cheminformatics, where the similarity of molecular graphs needs to be quantified. Unfortunately, MCS is NP-hard unless additional constraints regarding the properties of G, H, and the subgraph are enforced. A polynomial-time solvable case requires that all these graphs are trees. Known negative complexity results for classes of tree-like graphs set narrow boundaries for generalization.

The project aims to investigate particular tree-like graph classes to design, implement, and experimentally evaluate new polynomial-time algorithms. New NP-hardness proofs for specific cases should refine the complexity status further.

Requirements: Strong interest in algorithms and graph theory

Contact: Nils Kriege

In this project you will study novel approaches for learning interpretable and explainable deep learning models with the goal of supporting users with different skill levels and knowledge. The models will be evaluated on applications with large societal impact, e.g., labour market data.

Contact: Sebastian Tschiatschek

In many areas of our life questionnaires/a collection of measurements are used to gather information for decision making, e.g., questionnaires are used to measure the success of marketing campaigns or customer satisfaction, or medical tests are conducted to decide on the best treatment for a patient. Common to those applications is that many questions need to be answered or many tests must be conducted. Often this is not because all the information is needed for deriving a decision but rather because the course of action is standardized (e.g., a survey always consists of the same questions).

In this project you will evaluate and develop methods for adaptive information acquisition that acquire the information necessary for decision making sequentially and adaptively, e.g., not all questions in a questionnaire are asked in all cases and the order of the questions can be changed. You will consider and evaluate recent machine learning models for this problem but also investigate how these methods can be extended to account for dependencies in the information that can be required, e.g., dependencies that only allow for a specific order of some questions of a questionnaire, the influence of how questions are asked on the answers, etc.

Contact: Sebastian Tschiatschek

Clustering is the task of finding groups in a data set. As an unsupervised learning approach it is important for exploratory data analysis, e.g., for finding commonalities between patients suffering from some disease. Often such data is high-dimensional and contains complicated non-linear relationships, which makes it difficult to apply standard clustering approaches. Deep clustering algorithms have been proposed recently to approach this kind of setting by combining clustering and deep learning in a common framework. An open problem that has not been sufficiently addressed in recent deep clustering research is incomplete and heterogeneous (e.g. survey data, blood tests, MRI images, etc.) data. Incomplete data is often an issue in practice, especially for health data where not everything about a patient’s history is known.

In this project you will study the clustering of incomplete data with heterogeneous data types with a particular focus on missingness and on determining which information is relevant for cluster assignments. You will develop an algorithm based on previous work and evaluate it on synthetic and real-world data. Some research questions of interest would be for instance: How do cluster assignments change when given new information? Or, given an uncertain cluster assignment (e.g. sick or healthy patient) which information would we need to query to make a decision?

Contact: Sebastian Tschiatschek

Enabling multiple interacting agents to convey their knowledge and skills through teaching.

Contact: Sebastian Tschiatschek

Learning abstractions of large state and action spaces in order to facilitate more efficient planning and exploration.

Contact: Sebastian Tschiatschek

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Please see [Moodle] for a detailed description of this and other Natural Language Processing topics.

Contact: Benjamin Roth

Reinforcement learning has shown remarkable successes in playing board games like Go but also in playing computer games like DOTA. While these results are impressive, the learned strategies (policies) are typically not directly interpretable by humans. However, interpretability is important for instance to enable better collaboration of humans with AI agents or to assess safety of the learned behavior.

In this project you will make such learned policies easier to understand by humans by incorporating constraints on their structure. You will develop and implement a novel policy training algorithm and evaluate it on simple computer games.

Students wanting to work on this topic are expected to have solid coding skills in Python, basic knowledge in PyTorch or TensorFlow, and be curious to learn about reinforcement learning and its applications.

Contact: Sebastian Tschiatschek

Reinforcement learning has shown remarkable successes in application domains like game play and robotics. These successes where achieved by performing large numbers of interactions of a learning agent with the environment. In these interactions, the agent constantly receives information about rewarding behavior. However, in many applications involving human users (e.g., personal assistants), the reward information is explicitly or implicitly provided by these human users. Therefore, this information must be treated as a scarce and valuable resource, and the (cognitive) cost of providing this information should be accounted for.

In this project you will study the cost of providing different types of feedback used for reinforcement learning in user studies. To this end, you will implement a simple reinforcement learning environment for which a human user can provide reward information in various forms (e.g., comparisons, sorted lists, etc.) and compare the cost of providing this feedback as well as the usefulness of the feedback for training the reinforcement learning agent. Depending on whether you are doing a P1/P2/Bachelor Thesis/Master Thesis, the scope of the project will be adjusted.

Students wanting to work on this topic are expected to have solid coding skills in Python, basic knowledge about the development of interactive web pages, and be curious to learn about reinforcement learning and its applications.

Contact: Sebastian Tschiatschek

Recent advances in reinforcement learning have enabled super-human performance of AI agents in many applications, e.g., game play. In this project you will work on building theory and/or models for reinforcement learning from implicit and explicit feedback. In particular, you will consider the setting in which no rewards are observed during execution of the agent but only cumulative reward information is obtained at the end of the execution. As a practical example, assume playing a computer game in which you cannot observe the score while you play the game but only see your achieved score at the very end. This setting has important applications in human-in-the-loop settings in which feedback can be expensive to obtain.

Students working on this project need basic background knowledge in machine learning, solid programming skills in Python, and the desire to work on reinforcement learning.

Contact: Sebastian Tschiatschek

In this project you will work on predicting students' responses to mathematical questions using machine learning models. This is important to assess a student's skills and provide personalized educational resources. We use real-world data provided by the "Diagnostic Questions: Predicting Student Responses and Measuring Question Quality" challenge at NeurIPS'20. The challenge consists of four different tasks from which a subset can be selected for the project, depending on background knowledge and programming skills.

Students working on this project need basic background knowledge in machine learning, programming skills in Python, and the desire to develop and test machine learning models.

Contact: Sebastian Tschiatschek

Automated sequential decision making is an important application of machine learning systems in which such a system needs to select a sequence of actions step by step to optimize a reward/utility function. For instance, in autonomous driving, such a system needs to execute a sequence of steering, breaking and acceleration actions, or in a medical intensive care setting, such a system needs to execute a sequence of measurement and treatment actions.

One challenge in realizing such automated sequential decision making systems is the definition of the reward/utility function. For example, in autonomous driving it is hard to specify all the factors which define good driving behavior. In such settings, automatically inferring the reward/utility function from users’ feedback can be beneficial.

This project investigates approaches for reward/utility inference from diverse and implicit feedback, building on ideas for inverse reinforcement learning, active learning, implicit feedback, etc.

Interested students are expected to have solid mathematical and machine learning skills, and have experience in Python and deep learning (using PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Reinforcement learning has been successfully used to solve certain challenging sequential decision making problems in recent years. The employed techniques commonly require (i) huge amounts of interactions with the envirnoment and (ii) clearly specified reward signals to work well. In many applications however, one or both of these requirements are not met. In such cases, imitation learning can be an efficient approach to sequential decision making problems: an expert demonstrates near-optimal behavior and a learning agent attempts to mimic this behavior.

This project considers imitation learning in settings in which there is some form of mismatch between the expert demonstrator and the learning agent. The scope of the project is to study how existing algorithms perform in this setting and proposes modifications to existing algorithms to achieve better performance.

Students wanting to work on this topic are expected to have a basic understanding of machine learning techniques, solid knowledge of Python and basic knowledge of deep learning libraries (PyTorch or TensorFlow).

Contact: Sebastian Tschiatschek

Variational Autoencoders (VAEs) are powerful deep generative models that have been successfully applied in a wide range of machine learning applications. Recently, the Partial VAE (PVAE), a variant of VAEs that can process partially observed inputs has been proposed and its effectiveness for data imputation has been demonstrated. Key to the fast training of VAEs and PVAEs is the amortized prediction of posterior distributions from observations. In PVAEs, these posterior distributions are predicted from partial observations.

This project aims at studying the consistency of these posterior distributions for different patterns of missing data. The insights are used to create/train better inference models and thereby improve the quality of PVAEs.

Students wanting to work on this topic are expected to have solid mathematical skills, a basic understanding of machine learning techniques and good programming skills in Python.

Contact: Sebastian Tschiatschek