CLOSED: Special Issue on Metacognitive Prediction of AI Behavior

Important Dates

• Title and Abstracts Due: 1 October 2025 (to is3-26@computer.org)
• Full Manuscripts Due: 15 November 2025 (via submission site)
• Publication: May/June 2026

TOPIC SUMMARY:

As artificial intelligence (AI) becomes more prevalent in an increasing number of practical applications and systems, improved characterization of such systems will in turn become important to ensure system resiliency, safety, and reliability in the environments for which they are deployed, which often produces data that differs from data used in training.  However, while Intelligent systems, often using supervised machine learning or reinforcement learning, have provided excellent results for a variety of applications, the reasons behind their failure modes – or anomalous behavior they engage in – are generally not well understood.  The idea of metacognition, reasoning about an Intelligent system itself, is a key avenue to understanding the behavior and performance of machine learning systems.  Recently, a variety of methodologies have been explored in the literature, which include stress testing of robotic systems, model introspection, model certification, and performance prediction.  Moreover, researchers across multiple disciplines including Computer Science, Control Theory, Mechanical Engineering, Human Factors, and Business Schools have explored these problems from different angles.

This special issue seeks to collect cutting edge research associated with the 2nd Workshop on Metacognitive Prediction of AI Behavior (METACOG-25); the best papers presented at the workshop will be invited to submit an extended version to the special issue, and other relevant manuscripts will also be accepted. The main objectives of the special issue are aligned with those of the workshop:

• Survey the main approaches to metacognition in intelligent systems.
• Understand the requirements that metacognitive approaches have for successful deployment.
• Identify novel methods for metacognition that drive improved AI performance in an operational or cross-domain setting.
• Identify application areas suitable for the deployment of metacognitive methods.
• Understand the relationship between approaches to AI metacognition and the behavior of human operators.

Specific topics to be covered include, but are not limited to:

• Explainable performance prediction of black-box AI systems
• Stress testing of reinforcement learning systems
• Use of metacognition to increase trust in intelligent systems by their operators
• Applications of AI metacognition to vision and robotic systems
• New methods leveraging neuro-symbolic AI architectures for metacognition
• Techniques for AI systems to self-adapt (self-heal, self-repair) in new domains
• Hyperdimensional Computing (HDC) and Vector Symbolic Architectures (VSA) and their relationship to metacognition

Submission Guidelines

For author information and guidelines on submission criteria, visit the Author’s Information Page. Please submit papers through the IEEE Author Portal and be sure to select the special issue or special section name. Manuscripts should not be published or currently submitted for publication elsewhere. Please submit only full papers intended for review, not abstracts, to the ScholarOne portal. If requested, abstracts should be sent by email to the guest editors directly.

In addition to submitting your paper to [Conference Name], you are also encouraged to upload the data related to your paper to IEEE DataPort. IEEE DataPort is IEEE's data platform that supports the storage and publishing of datasets while also providing access to thousands of research datasets. Uploading your dataset to IEEE DataPort will strengthen your paper and will support research reproducibility. Your paper and the dataset can be linked, providing a good opportunity for you to increase the number of citations you receive. Data can be uploaded to IEEE DataPort prior to submitting your paper or concurrent with the paper submission. Thank you!

Questions? Contact the Guest Editors at is3-26@computer.org

Paulo Shakarian, Syracuse University (Lead Guest Editor)

Nathaniel D. Bastian, U.S. Military Academy

Gerardo I. Simari, Universidad Nacional del Sur

Andrea Pugliese, Università della Calabria