Dive into the intersection of AI and human practicality in this thesis. We’re crafting a metric that measures how achievable AI-generated advice is, for real people. Think of it as teaching AI common sense; for instance, you can’t just age a decade instantly. Your challenge: integrate advanced language models like GPT to gauge whether a “what if” scenario is useful or just a tech fantasy. And because this work contributes to smarter, more resilient national IT systems, your insights can ultimately help prevent or detect failures in the digital infrastructure we all rely on. Join us in shaping a more sensible, user-centric AI future.
This graduation assignment is part of the project ‘Mission: Impossible – Critical IT Protocol’. With this project, we are giving the Netherlands a digital boost by building IT solutions for critical societal challenges. From payment systems and public services to cybersecurity and crisis detection: together, we are strengthening the Netherlands’ digital infrastructure with impact that truly matters.
Counterfactual explanations are considered an intuitive and user-friendly form of explainable AI. A counterfactual explanation proposes minimal changes to the input data that would lead to a different model output. As an example, a bank could make use of a model to determine if a customer can get a loan or not. If a customer was denied a loan, a counterfactual explanation could tell them that, for instance, “if their savings were increased by €10.000, they could get the loan”.
However, counterfactual explanations are not aware of the semantics of the input features. It might be that the minimal change for a customer to get a loan is to “increase their age from 25 to 35 years old” which, from a human perspective, is evidently unfeasible in a short time span.
The Assignment
In this project, we would like you to design a new evaluation metric to automatically determine the practicality of counterfactuals, that is, how attainable a counterfactual explanation is (or if it is even humanly attainable). Since the determination of practicality involves semantic understanding of the input features, our idea would be to leverage GPT and/or other large language model-based technologies to accomplish this.
A user study could be conducted initially following the methodology proposed by Spreitzer et al (1) to evaluate the practicality of different state-of-the-art counterfactual techniques. The user study can then be extended to evaluate the accuracy of the proposed metric compared to the human evaluations.
(1) Spreitzer, H. Haned, & I. van der Linden, “Evaluating the Practicality of Counterfactual Explanations”
About Info Support
Info Support specializes in custom software, data/AI solutions, management, and training and is active in the Finance, Industry, Agriculture, Food & Retail, Mobility & Public, and Healthcare sectors. We provide solid and innovative solutions for complex and critical software issues. Our headquarters are located in Veenendaal (NL) and Mechelen (BE). At present, approximately 500 employees are employed by Info Support.
Info Support’s working method is characterized by a number of core values: solidity, integrity, craftsmanship, and passion. These core values are intertwined in our work and the way we interact with each other.
To ensure that all employees are always up to date with the latest developments, Info Support has an in-house IT Academy that eagerly satisfies the hunger for more or different knowledge and skills.
B2 language proficiency in Dutch is required.
