Understanding Agentic AI and Explainable AI

Agentic AI refers to autonomous AI systems capable of making decisions and acting independently based on external input and overarching goals, unlike current tools like ChatGPT or code assistants that rely on human prompts. While still emerging, Agentic AI is seen as a potential step toward Artificial General Intelligence (AGI), though current models still lack self learning abilities. 

Explainable AI (XAI) is crucial for understanding how these autonomous systems make decisions, ensuring transparency, accountability and trust especially in high-stakes fields like healthcare, law, and cybersecurity. 

Agentic AI brings both risks and opportunities. Unsupervised autonomy can lead to dangerous outcomes (e.g., financial crashes, safety failures), but properly managed, it can enhance cyber defense, automate complex monitoring, and increase efficiency. 

The safe and ethical development of Agentic AI depends on combining autonomy with explainability, strong oversight, and clear accountability.

Recently the term “Agentic AI” has surged up on the radar in the artificial intelligence industry. The term “Agentic AI” refers to AI systems that are aimed to act more autonomously than previous AI systems, without human supervision. The term refers to new AI systems that can act upon input from the outer world, make decisions based upon instructions or a range of overarching goals and then act upon the decision that the AI system makes, without needing continuos confirmation from a human. 

Such new Agentic AI systems therefore differs from current popular AI systems, like code assistants or general purpose assistants, like ChatGPT. Code assistants like Cursor and Lovable have made a huge headway in the last year, achieving results that recently were hard to imagine. With the input from a software developer, these systems can setup almost complete software projects, configurations and code based only on prompts from the software developer. 

ChatGPT is becoming more and more sophisticated, being able to provide an answer (although sometimes not satisfactory or correct) to a wide variety of questions or tasks, including technical tasks like chess. However these systems are not autonomous, despite their many benefits. Therefore they lack a crucial cornerstone that would make them AI agents.

While still being mostly theoretical, AI agents may have a big impact in the future. First of all, it is worth discussing how and if current AI systems are approaching a level of Artificial General Intelligence (AGI). Secondly, it is worth bringing up the caution one ought to take when making AI systems autonomous and able to act without human supervision. 

Artificial General Intelligence (AGI) is a type of intelligence reached by AI systems that match or surpass human intelligence. Some claim that certain AI language models have already reached early stages of AGI, but the consensus is that AGI is yet to be reached. With the development of AI agents that are supposed to act without human supervision, one could argue that AGI is getting closer and closer. It is worth noting that in order for AI systems to reach AGI, it would require them to have generalized intelligence, being able to perform a variety of complex tasks. 

With the development of Mixture of Experts (MoE) models like DeepSeek, the performance and reasoning ability of AI models are coming very close to AGI. For clarification, MoE models are deep learning models that consist of sub-networks (so called “experts”) that through their learning process have been specialized in different areas. Imagine it like different parts of the human brain specializing in different tasks, like speech, mathematics and chess. When MoE models are instructed to solve a task, these subnetworks work together in a joint effort to solve a task. 

However, one could also argue that AI systems would have to have the ability to learn new skills in order to have AGI. While this is possible through human intervention by retraining, they cannot self-learn or automatically acquire knowledge. As such, state of the art models with generalizable knowledge like DeepSeek are still still relying on on human intervention for their utility and for their acquiring of new knowledge. 

It is also worth noting the caution one ought to take for deploying AI agents to perform tasks without the supervision of a human being. It may seem that total automation would be a good thing, and it can be. Automation can alleviate the burden on humans to perform monotonous or even impossible tasks, like monitoring a high flow of financial transactions for fraudulent activities or monitoring for cybersecurity threats. However, one also takes a risk of AI systems to make rash decisions or take actions with disastrous consequences to reach an overarching goal. 

One practical example is the 2010 Flash Crash, which was caused by a chain of trades made by automated trading algorithms at high frequency trading firms. This chain reaction caused by automated systems caused the Dow Jones Industrial Average to plummet 1000 basis points in just minutes. Other more recent examples include Tesla’s automated driving systems that have documented cases of failing to recognize pedestrians or the concerning situation where a chatbot launched by New York city, intended to advice small business on the city’s bureaucracy, gave users advice to break the law. 

Needless to say, there are serious risks when processes are automated through AI to reach an overarching goal without human supervision.

Explainable AI (XAI) models are a type of AI models that serve to give insight and interpretability of how AI systems reason and make decisions. While modern deep learning models are very powerful, they are like black boxes. They act upon input and output results, while humans struggle to understand the reasoning behind the decisions due to the AI systems’ complex architecture. 

The goal of XAI is to bridge the gap between human understanding and machine intelligence. By making AI decision-making processes transparent, XAI allows users—whether they are developers, domain experts, or end-users—to trust and effectively interact with AI systems. This transparency is crucial in domains such as healthcare, finance, law, and autonomous systems, where the consequences of AI decisions can be significant or even life-altering.

Explainable AI is crucial for humans to understand why an AI system gives a certain recommendation or takes a certain decision. If AI models are to be used to give recommendations for individuals to act upon, it is important for humans to understand the causality behind the advice. This gives users a chance to look under the hood of AI systems and themselves judge the decisions of AI systems by looking at which factors have led to a certain recommendation. 

There is also the issue of accountability, especially in the context of discussing Agentic AI. If companies and organizations are to deploy autonomous AI agents that act on the behalf of the owner, who carries the responsibility of the AI agents’ decisions. 

• The engineer that deployed the AI agent? 
• The engineer’s manager? 
• The CEO of the company? 
• Or is it the engineer/manager/CEO of the tech company that originally developed the AI agent rather than the organization that deployed it for use? 

Responsibilities can quickly become blurred, especially when it comes to AI systems that are supposed to act autonomously. Nonetheless, or even because of this difficulty, it becomes very important for humans to understand the algorithmic reasoning behind individual decisions that autonomous AI systems make. Not because humans will be able to follow up every case or decision that the automated AI system make, but in the cases of sever mistakes by the AI agent that do require human follow-up, explainablity is crucial.

As Agentic AI systems evolve toward greater autonomy, they bring both significant risks and promising opportunities from a security perspective. On the one hand, autonomous agents that operate without human oversight raise red flags, especially if their decision-making processes are susceptible to manipulation. An autonomous AI agent could be exploited through prompt injection or adversarial attacks, manipulated to carry out tasks outside its intended scope, particularly if these agents are deployed in sensitive environments like finance, critical infrastructure, or defence. 

In geopolitical contexts, Agentic AI could be weaponized for information operations, deepfake propaganda, or cyber intrusion, amplifying the scale and speed of digital threats. However, when carefully developed and properly supervised, Agentic AI can also become a powerful ally in cyber defence. Autonomous systems can monitor large-scale networks, detect anomalies in real time, and respond to threats faster than any human analyst. In domains like threat hunting or even fraud detection, Agentic AI has the potential to increase precision and reaction speed.  

By combining autonomy with explainability, organizations can build intelligent agents that are both effective and accountable, enhancing cyber resilience in an increasingly complex digital threat landscape. As such, integrating security principles and ensuring explainability is not just a technical preference but a necessity. Building resilient and transparent AI systems will be vital to ensure that autonomy does not come at the cost of safety or trust.

The emergence of Agentic AI marks a significant shift in the evolution of artificial intelligence, from systems that passively respond to prompts, to agents capable of autonomous decision making. While current AI tools like code assistants and general-purpose models offer impressive capabilities, they remain fundamentally non-agentic due to their reliance on human interaction. In contrast, Agentic AI aims to act independently, pursuing goals with minimal or no human oversight. This shift opens the door to great potential but also introduces new layers of complexity and risk. 

As we move closer to the possibility of Artificial General Intelligence (AGI), agentic behaviour may become a key milestone. Technologies like Mixture of Experts (MoE) models illustrate how specialization and collaboration among sub-models can push the boundaries of what AI can achieve. Yet, the inability of these systems to self-learn without human intervention remains a clear limitation on the path to AGI. 

Equally important is the need for transparency through Explainable AI (XAI). As AI systems gain autonomy, it becomes increasingly vital that humans understand their decisions, both for trust and for accountability. This is especially true when errors occur or when AI actions result in unintended or even harmful outcomes. 

From a cybersecurity standpoint, Agentic AI introduces both threats and opportunities. On one hand, if misused or poorly secured, autonomous agents can become tools for cyber attacks or misinformation. On the other hand, when properly designed and monitored, these systems offer transformative advantages in threat detection, response time, and operational resilience. 

Ultimately, the future of Agentic AI must be built on a foundation of accountability, explainability, and cautious innovation. As we navigate this transition, the goal should not only be to create more powerful AI agents, but also to ensure that they are aligned with human values, goals, and safety standards. In doing so, we can unlock the benefits of AI autonomy while minimizing its risks.