EXACT 2026

Introduction

Large Language Models (LLMs) have demonstrated impressive abilities in question-answering systems, including those applied in educational contexts. However, these models often generate only brief, single-answer responses without providing any reasoning or explanatory detail. In educational environments, this lack of transparency poses significant challenges:

One promising direction within Explainable AI (XAI) is Symbolic Reasoning, where a Symbolic Engine, either standalone or integrated with an LLM, makes the reasoning process explicit. However, many other approaches can also achieve transparency. This challenge invites any method that enhances both the accuracy and interpretability of educational Question Answering (QA) systems, making them more suitable for verifiable use in learning environments.

Example Queries

The following examples illustrate the types of educational queries this challenge addresses, along with expected transparent responses:

Challenge Chairs

The quality and fairness of this challenge are ensured by an international committee of professors and experts:

• Prof. Quan Thanh Tho — Ho Chi Minh City University of Technology (HCMUT), Vietnam
• Prof. Emanuel Di Nardo — University of Naples Parthenope, Italy
• Prof. Nguyen Duc Anh — Department of IT and Economics, University of South Eastern Norway, Norway
• Prof. Fabien Baldacci — Université de Bordeaux, France
• Prof. Nguyen Le Minh — Japan Advanced Institute of Science and Technology (JAIST), Japan

Datasets

The official datasets will be released at the kick-off workshop. Two dataset types will be provided, covering logical reasoning in educational regulations and physics problem-solving. The input provided to each team's system depends on the dataset type (see details below). All other fields (FOL, CoT, explanations, etc.) shown in the samples are reference annotations provided in the training data only, which teams can use as templates for building their own reasoning pipelines.

Dataset Type 1: Logic-Based Educational Queries

This dataset contains 464 records with a total of 913 questions designed to evaluate logical reasoning in educational contexts. Topics cover university regulations such as grading policies, course enrollment rules, scholarship criteria, and academic requirements. Question types include Multiple Choice, Yes/No/Uncertain, and open-ended queries. Each record includes a set of premises in both natural language and FOL, along with derived questions, ground-truth answers, and human-written explanations. During evaluation, the system receives the question together with the natural language premises (premises-NL) as input. Teams are free to use the premises in any way (e.g., as prompt context, for FOL conversion, etc.).

{
  "premises-NL": [
    "If a curriculum is well-structured and has exercises, it enhances student engagement.",
    "If a curriculum enhances student engagement and provides access to advanced resources, it enhances critical thinking.",
    "If a faculty prioritizes pedagogical training and curriculum development, the curriculum is well-structured.",
    "The faculty prioritizes pedagogical training and curriculum development.",
    "The curriculum has practical exercises.",
    "The curriculum provides access to advanced resources."
  ],
  "premises-FOL": [
    "ForAll(c, (well_structured(c) ∧ has_exercises(c)) → enhances_engagement(c))",
    "ForAll(c, (enhances_engagement(c) ∧ advanced_resources(c)) → enhances_critical_thinking(c))",
    "..."
  ],
  "questions": [
    "Based on the premises, what can we conclude about the curriculum?\nA. It enhances student engagement but not critical thinking\nB. It enhances critical thinking\nC. It needs more resources to enhance critical thinking\nD. It is well-structured but lacks exercises",
    "Does the combination of faculty priorities and curriculum features lead to enhanced critical thinking?"
  ],
  "answers": ["B", "Yes"],
  "explanation": [
    "Premise 4 and premise 3 confirm the curriculum is well-structured. Premise 5 provides exercises, so premise 1 implies enhanced engagement. Premise 6 adds advanced resources, and premise 2 confirms enhanced critical thinking, supporting option B.",
    "Faculty priorities satisfy premise 3, making the curriculum well-structured. Exercises (premise 5) and premise 1 lead to enhanced engagement, and with advanced resources (premise 6), premise 2 confirms enhanced critical thinking."
  ]
}

Dataset Type 2: Physics Problems

This dataset contains 5,520 text-based physics problems focusing on electric circuits and electrostatics. Topics include resistance, voltage, current, power, capacitance, electric fields, and energy calculations. Questions are numerical, requiring multi-step computation. Each problem comes with step-by-step CoT reasoning and a final numerical answer with its unit. During evaluation, the system receives only the question as input. The source materials (textbooks, knowledge references) used to construct this dataset will be announced at the kick-off workshop.

{
  "id": "TD401",
  "question": "Calculate the energy stored in capacitor C when C = 100 μF and U = 30 V.",
  "cot": "Step 1: Identify the given values for capacitance (C) and voltage (U).\nStep 2: Recall the formula for energy: E = 0.5 * C * U^2.\nStep 3: Convert capacitance to Farads: C = 100 μF = 1 × 10^-4 F.\nStep 4: Substitute: E = 0.5 × (1 × 10^-4) × (30)^2.",
  "answer": "45",
  "unit": "J"
}

Submission Requirements

Each team must submit an API endpoint along with a brief solution description (1 page) detailing their approach, models used, and the dataset used for training. For each query, the API must return the required fields below. Teams are encouraged to include optional fields that demonstrate the depth of their system's reasoning. Richer evidence will have an advantage in the evaluation, particularly in the final round where the Challenge Chairs assess reasoning depth live.

{
  // Required (Mandatory)
  "answer": "B",
  "explanation": "The voltage across R2 is calculated using ...",

  // Optional (Encouraged)
  "fol": "∀x (Resistor(x) → HasVoltage(x, V))",
  "cot": [
    "Step 1: Identify the circuit topology ...",
    "Step 2: Apply Kirchhoff's voltage law ...",
    "Step 3: Solve for the unknown voltage ..."
  ],
  "premises": [
    "Ohm's law: V = IR",
    "KVL: sum of voltages in a loop = 0"
  ],
  "confidence": 0.92
}

Note: answer and explanation are mandatory for every submission. All other fields (fol, cot, premises, confidence) are optional but encouraged, as they contribute to higher scores in the reasoning depth evaluation. The final submission format will be finalized at the kick-off workshop.

Prizes and Awards

Outstanding teams will be recognized through a combination of cash prizes, publication opportunities, and certificates:

• Top 5 teams: Cash prize and invitation to present at CSoNet 2026 in Vietnam.
• Top 10 teams: Invited to submit a paper to our Special Session "Explainable AI for Educational Question-Answering" at The 15th International Conference on Computational Science and Network Intelligence (CSoNet 2026).
• All teams with valid submissions through Phase 1 and Phase 2: Official certificate of participation issued by the Challenge Chairs and the conference organizers.

Previous Edition

The first edition of this challenge was held at the International Workshop on Trustworthiness and Reliability in Neuro-Symbolic AI, co-located with IEEE IJCNN 2025. The event attracted 107 participants across 30 teams from multiple countries and produced the first public benchmark for explainable academic-regulation QA.

• Website: https://sites.google.com/view/trns-ai/challenge
• Our Findings: Long S. T. Nguyen*, Khang H. N. Vo*, Thu H. A. Nguyen*, Tuan C. Bui, Duc Q. Nguyen, Thanh-Tung Tran, Anh D. Nguyen, Minh L. Nguyen, Fabien Baldacci, Thang H. Bui, Emanuel Di Nardo, Angelo Ciaramella, Son H. Le, Ihsan Ullah, Lorenzo Di Rocco, and Tho T. Quan. Bridging LLMs and Symbolic Reasoning in Educational QA Systems: Insights from the XAI Challenge at IJCNN 2025. Italian Conference on Big Data and Data Science (ITADATA), 2025. [PDF]

Building on these results, EXACT 2026 broadens the scope from educational regulations into STEM reasoning and introduces structured evaluation criteria for explanations.