Leveraging Agentic AI in Games

Introduction

Studios have years of experience building sophisticated, dynamic systems that work within the constraints of game development. Even with that in mind players want more. Players want more dynamism, control and replayability. They want game worlds that are more dynamic, characters that feel alive, and experiences that are truly interactive and personalized. Studios hear this loud and clear, and so do we. At the end of the day, our shared goal is simple: to make amazing games for players. We achieve this goal by establishing a shared understanding that respects the expertise already in the industry and focuses on solutions that actually help studios deliver the experiences players want.

Agentic AI systems can help game developers: create highly dynamic game worlds, NPCs that can react to the player, QAgents that speed development and produce higher quality outcomes for player support requests. Agentic systems can also be applied to line-of-business problems like generating personalized marketing creative. Too often, conversations regarding new technologies and capabilities focus on buzzwords and big promises, without fully appreciating the technical artistry and practical realities that go into making great games. The opportunities that we’ll share in this blog will range from: something you can do today with relative ease to more advanced future opportunities.

Before delving into the content, we would be remiss if we didn’t discuss our nomenclature. The words Artificial Intelligence (AI) can mean so many things in Games. The industry has built AI’s in the form of NPCs and bots for quite a while. Procedural generation has also been leveraged to help create content since Games were a thing. When Machine Learning (ML) and Reinforcement Learning (RL) became more prevalent in the industry they were often referred to as AI as well. Now Generative AI (Transformer Based Models) is being discussed and referred to as AI. To clarify and simplify, this blog when we say AI we are referring to GenAI. If we are referring to any of the other terms, we’ll name them specifically.

What is Agentic AI?

Agentic AI refers to autonomous, goal-driven artificial intelligence systems that can act independently, adapt in real time, and make complex decisions based on context and objectives. Unlike traditional, rule-based AI, which follows scripted behaviors or static routines, agentic AI is designed to learn, reason, and evolve within dynamic environments.

To build performant and scalable Agentic AI workflows, Games studios need to put their agents where their data is. Databricks offers the only unified platform for developing, evaluating, and governing AI Agents that deliver reliable, data-driven results in Games environments. By leveraging existing Databricks solutions, like AI Playground and MLflow Model Signatures to define agents' input and output schema, you can prototype agents right where your data lives.

Here is a quick look at what works and what does not:

High-Level Goals for Agentic AI in Games

Player-Centric Experiences: Agentic AI makes it possible to deliver worlds and characters that feel truly responsive to each player. Living NPCs augment the games’ narrative by enabling them to remember, adapt, and evolve, turning every playthrough into a unique, personalized journey that respects the story the developer wanted to convey.

Player Engagement: By enabling dynamic interactions and emergent gameplay, agentic AI drives deeper player engagement. Players encounter new challenges, storylines, and behaviors that keep them coming back for more.

Building Better Games: Automated QA agents (QAgents) streamline testing and content creation. This reduces development cycles, improves quality, and supports existing QA resources to focus on things AI can’t test.

Supporting Live Games: Agentic AI helps studios manage live games more efficiently by automating community support, moderating player interactions, and personalizing live content updates. This lowers operational costs and ensures a safer, more welcoming environment for players.

Exploring Examples of Agentic AI in Games

To further the high-level proposed definition, we’ve selected a subset of player-centric use case examples that frame the aforementioned goals and capabilities.

Living NPCs

With Agentic AI, non-player characters can remember your choices, adapt their personalities, and even pursue their own goals. These characters can react in unique, surprising ways that make the world feel more immersive and alive. Imagine a companion who grows and changes based on your play style, a rival who holds a grudge from a previous encounter or a character whose demeanour dynamically changes as a result of the decisions you’ve made. Games are interactive entertainment, you’ve been doing these types of things for quite a while, these concepts are not fundamentally new. Agentic AI is another tool that you can leverage to evolve your approach to this dynamism. Instead of you having to pre-define all the behavior changes and the different responses of your characters the agent creates that for you. The result is being able to create even more immersive worlds that feel more personal that drive greater player engagement and replayability.

Let’s imagine you’re building an NPC that is supposed to be the equivalent of the town historian. Traditionally, they’d have XYZ inputs and responses. You might write half of them, probably outsource the other half. With an Agentic system you provide the player more agency. For a PC game, you might have a chat interface that they can write into, perhaps in addition to pre-written prompts. For a console game, where typing is less enjoyable, you could explore a speech-to-text solution or use an agent to suggest prompts for the player, dynamically, based on the state of the game. With the player’s statement or question entered, it is time for the agent to build a response. Their response could be a simple knowledge base lookup that scans all the lore of the town, or your game world. The agent could also query a series of tables that describe the current state of the game or the player, and then leverage that to generate the prompt that is ultimately used to create their response. As a compound AI system you can keep it simple, complex and even evolve your Agentic AI augmented NPCs over time with relative ease.”

An interesting subcategory of this use case shared with us by Andrei Muratov at AWS is that of disembodied NPCs. They share that latency causes challenges with the quality of interaction that players expect these days. One approach that they’re seeing studios explore is integrating Agentic AI to create NPCs that have no physical form (disembodied). This could come in the form of an anthropomorphic companion, a voice from the sky or perhaps from inside your head. By removing the physical form, you simplify the problem set quite a bit. 1) responses no longer require facial movement, 2) additional time is available to perform the compute that is required to create the responses, 3) you can limit the interaction of the player with the entity, enabling you to keep the costs of serving responses at a reasonable level.

Bots and QAgents

As we mentioned previously, the creation of bots is something that game developers are highly familiar with already. By leveraging Agentic AI and Reinforcement Learning, we are able to evolve and improve upon that work. In continuation, we explore a specific example of a bot, the QA bot (dubbed QAgent). The architecture and technical approach you would take for a QAgent is the same that you’d employ for any bot. We leverage this example, however, because QAgents often need to be developed more often, more quickly and adapt as gameplay mechanics evolve over time.

QAgents represent a behind-the-scenes use case that is all about building better games and supporting live operations. These AI-powered automated QA testers, expressed as bots, interact with the game just like a human would, playing through levels, performing specific actions, and looking for bugs or unexpected behavior. Unlike traditional scripted test automation, QAgents can adapt to changes in the game, explore new content, and respond to dynamic environments. This enables studios to test more efficiently, catch issues earlier, and maintain higher quality in both new releases and live games. The result is a smoother experience for players and a more agile development process for studios.

One way to go about developing these agents is by using tools like Unreal Engine’s experimental Learning Agents plugin. This plugin provides you with an efficient, game-ready implementation of popular machine learning algorithms along with interfaces that allow your designers and developers to specify the required interface, either by Blueprints or C++ code. As long as you can specify the observations your agent can make, the actions your agent can take, and define what “good” looks like for you in terms of a reward function, Learning Agents can facilitate collecting experience data to train the required models to power your bot or QAgent with machine learning. The plugin supports both reinforcement learning, where the model learns based on optimizing cumulative rewards (i.e., it learns by playing the game itself rather than recordings of others playing), and imitation learning, which leverages recorded demonstrations (e.g., human player actions) to train agents. Even if you don’t use Learning Agents directly, you can still consider adopting a similar approach by building out your own general-purpose machine learning implementation for your game engine and combine it with a training loop to build your agents.

In addition to some form of model to help automate your agents' behaviors in-game, the other aspect of QAgents and bots from an ML perspective is recognizing different aspects of interest. Some of these may be deterministic in nature, for example, checking for various constraint violations among objects, or aggregating statistics across a play session (e.g., the bot is simply no longer able to successfully complete the level). Other tests may require more sophisticated solutions, incorporating additional machine learning models. For instance, an object recognition model that detects player characters visually on the screen, combined with an image classifier that detects whether a person in an image is in a T-pose, could be run on sampled frames from the game loop to determine a particular flavor of visual defect that would traditionally require human detection. As your human play testers work with your game, capturing the data produced by defects they’ve identified can be used to train these models and further amplify and scale their work, leading to a virtuous cycle of data and AI amplification: the so-called data flywheel applied to your QA practice for your game leading to faster and more successful launches, more positive reviews, and happier players.

Regardless of the use cases for the QAgents or bots, what we hear from studios loud and clear is that they need the ability to train and retrain their AI models quickly and efficiently. Building an adaptive machine learning workflow that can keep models up-to-date by synchronizing MLops pipeline to developer, design, and creative department build cycles allows your studio to truly integrate AI to accelerate your game launches. Building this out on a scalable, cloud native data and AI platform enables it to scale up and down efficiently, keeping pace with your schedule, Combined with architectural best practices around feature engineering and model management, including fine-tuning where possible to take advantage of transfer learning, makes it efficient to run throughout your development cycle, augmenting and amplifying your teams heroic efforts. QA teams are often already working on expedited turnaround times, and adding model training lag to this loop is ultimately unhelpful. Instead, the models need to live close to the data.

Community Support

When you have a negative experience in a game, file a ticket and get a response that says something like: Thank you for your ticket, someday we’ll get back to you. Once you get the response, it often appears to be a canned response that doesn’t address your concern. Staffing these roles is quite expensive, maintaining knowledge bases to be used and keeping them up to date as new bugs are found, features are released and guidelines change is overwhelming. It is a result of all of these details that responses to players are often less than ideal.

Agentic AI provides us an opportunity to create a more player-centric experience for community support. This approach represents an evolution of your support function, not a whole new paradigm. Your heuristic chatbot is replaced with a more dynamic knowledge base-backed chatbot, effectively. This is step one. With that in place, you can immediately provide a better experience for your players. We continue from there, this is where Agentic AI comes into play, and build a compound AI system that takes the input from the player, extracts details about what is asked, and takes advantage of additional systems to augment the prompts that are used by the controlling AI system.

Let’s explore what the storyboard for an Agentic AI system for Community Support might look like:

• Prompt: User reports their server crashed right before winning the match, robbing them of a reward.
• Builds Understanding: Agentic system accepts the prompt, pulls out key details (server crash, close to winning, reward missing)
  • Query: Server crash log and the player’s matches to see if their match was on a failed server (it was)
  • Query: Last state of the game (were you actually about to win)
  • Query: Were there players on that server that have a statistically high level of server failures (cheaters?)
  • Query: Is this a high-value player?
• Takes Action
  • Combine the above understanding to evaluate: What’s the right course of action
  • Responds to player with proposed course of action with reasons that integrate details from its understanding
  • Player agrees or disagrees.
    • Agree: Agentic AI system either takes immediate action or, if action would need review, adds it to a queue (with the understanding it pulled) for further review and lets the player know next steps.
    • Disagree: If player is high value, move them to a live agent or a high priority queue. If low-value player responds in a respectful manner, that makes it clear there’s no other outcome.

With Agentic AI, the above is within the realm of today’s possibilities. It will require time, testing and effort, but you’ll have created a player-centric customer support experience that improves your retention over time. Your player and community support team is still critical, but their function will evolve as one to learn from the positive and negative experiences that occur through these agents to improve them over time. It will also free them up to work more closely with development and operations to improve the title as a whole. The goal isn’t to eliminate roles but to improve the outcomes that they drive.

Marketing Creative Generation

So far, all the examples we’ve provided lean heavily on the interactive side of things. Not all uses of Agentic AI have to be interactive in nature. Agentic AI systems are most useful when we consider multi-step, dynamic requirements. One such requirement within games is at scale generation of marketing creative. As we discussed in our recent UA Segmentation Blog, ad platforms are increasingly black boxes where the input you can provide has diminished. Developers also receive much less information about inbound leads from their marketing campaigns. One approach to help with the related cold start problem is to create marketing creative that aligns with different player segments and, in doing so, assumes preferences of inbound players based on the specific ad that they engaged with. In order to make this scalable, developers are looking to Agentic AI-enabled marketing creative generation.

Envision the following: you have advertising creative that you’ve used in the past, screenshots of your game, and other visuals that would be the basis of your future marketing creative. You have leveraged K-Means clustering to build a series of named player clusters e.g., socializer, completionist, killer and explorer. You have LTV models, campaign source, attributed ad network, and other metrics applied to your players to give you a holistic view of your players and their quality. You are now preparing for your next marketing campaign. You go into this system and ask “Generate 4 potential marketing creatives, each, for the top two LTV player segments and recommend a UA spend mix across ad networks based on the past performance of those networks for the players in these segments. Only consider players that joined 60-120 days ago.”

The agentic system will break the above into a series of steps, leverage the images that you have provided as a basis for the generation, query your segment tables, LTV details and campaign outcome tables, generate your potential images and suggest your UA spend mix. It infers all of this from your prompt. With this output, your marketing creative team may pick one of the examples and run with it or, more likely, use it as a basis for them to create their final product. You can see with this approach how much more quickly, and at scale, you could create highly targeted and customized marketing campaigns that speak not just to your audience as a whole but to different portions of it, enabling you to maximize your ROAS, eCPM and grow your playerbase.

Getting started

As an interactive medium, Games' dynamic experiences are a core skill within the industry. From decision trees to procedural generation and now Agentic AI systems, the industry will continue to integrate new methodologies to create engaging experiences. We have shared a small subset of potential use cases for Agentic AI within Games. The approaches described can be applied to other similar use cases and can be combined with each other. For example, we talk about Living NPCs and Bots as separate things but you might leverage both to create, for example, a player coach that you communicate with in a Co-Op game or to build a trainer that could be used during the FTUE of your title. It is important to note that Agentic AI opens the door to additional creative approaches but does not negate the need for highly skilled, creative and knowledgeable staff. It is a tool, not unlike any other that we’ve evolved and integrated into games over the years. As Straus Zelnick put it well, “Genius is the domain of human beings.”

Getting all of your data in one place, whether it be structured, unstructured or knowledge-based is a critical first step to make Agentic AI systems possible. With Databricks, you can build these systems more easily, make player-centric experience projects possible and cost-effective. If you’d like to learn more about how Databricks helps game companies with these and other use cases, check out databricks.com/games or reach out to your account executive. You can also learn more about Data, AI and Games in our eBook or our solution accelerators.

We can’t wait to take part in the new innovative experiences that you continue to build. Thank you for serving the players of the world.

Huntting Buckley, GTM Leader with Carly Taylor and Corey Abshire, Games Solutions

Games @ Databricks