Master's thesis in Data & AI: Automated Practicality Evaluation of Counterfactual Explanations Using GPT

Counterfactual explanations are considered an intuitive and user-friendly form of explainable AI. A counterfactual explanation ­proposes minimal changes to the input data that would lead to a different model output. As an example, a bank could make use of a model to determine if a customer can get a loan or not. If a customer was denied a loan, a counterfactual explanation could tell them that, for instance, “if their savings were increased by 10.000€, they could get the loan”.

However, counterfactual explanations are not aware of the semantics of the input features. It might be that the minimal change for a customer to get a loan is to “increase their age from 25 to 35 years old” which, from a human perspective, is evidently unfeasible in a short time span.

Assignment

In this project, we would like you to design a new evaluation metric to automatically determine the practicality of counterfactuals, that is, how attainable a counterfactual explanation is (or if it is even humanly attainable). Since the determination of practicality involves semantic understanding of the input features, our idea would be to leverage GPT and/or other large language model-based technologies to accomplish this.

A user study could be conducted initially following the methodology proposed by Spreitzer et al (1) to evaluate the practicality of different state-of-the-art counterfactual techniques. The user study can then be extended to evaluate the accuracy of the proposed metric compared to the human evaluations.

(1) Spreitzer, H. Haned, & I. van der Linden, “Evaluating the Practicality of Counterfactual Explanations”

About Info Support Research Center

We anticipate on upcoming and future challenges and ensures our engineers develop cutting-edge solutions based on the latest scientific insights. Our research community proactively tackles emerging technologies. We do this in cooperation with renowned scientists, making sure that research teams are positioned and embedded throughout our organisation and our community, so that their insights are directly applied to our business. We truly believe in sharing knowledge, so we want to do this without any restrictions.

Read more about Info Support Research here.

What does Info Support offer you during your graduation period?

Of course, we offer you an excellent package of graduation conditions with various options. These include:

• Laptop, all necessary tools, and development environments to successfully carry out your assignment
• A graduation allowance, where you have the choice of:
  • € 1000 gross per month
  • € 500 gross per month + lease car with fuel card
  • € 600 gross per month + accommodation in Veenendaal

During your graduation internship, you will be included in one of our business units and in our Research Center. This will give you a good understanding of the developments within our projects and our research projects.

In addition, Info Support offers you:

• A challenging master’s thesis at a solid, growing, and financially healthy company
• Professional and experienced guidance, hybrid working
• A graduation program with, among other things, training sessions, knowledge evenings, project visits, activities, and team, unit, and company outings
• Experimentation with new products and releases that are in the pre-release phase
• Brainstorming with other graduates and IT colleagues who are collegial and passionate
• Potential for a permanent position in our IT Top Traineeship program!

About Info Support

Info Support specializes in custom software, data/AI solutions, management, and training and is active in the Finance, Industry, Agriculture, Food & Retail, Mobility & Public, and Healthcare sectors. We provide solid and innovative solutions for complex and critical software issues. Our headquarters are located in Veenendaal (NL) and Mechelen (BE). At present, approximately 500 employees are employed by Info Support.

Info Support’s working method is characterized by a number of core values: solidity, integrity, craftsmanship, and passion. These core values are intertwined in our work and the way we interact with each other. This year, we were also named a Top Employer in the Netherlands!

To ensure that all employees are always up to date with the latest developments, Info Support has an in-house knowledge center that eagerly satisfies the hunger for more or different knowledge and skills.

B2 language proficiency in Dutch is required.