Scaling Relational Databases in the Era of Artificial Intelligence—Q&A With Vinodkrishna Gopalan

Relational databases are undergoing a profound transformation. Artificial intelligence (AI) and machine learning workloads exhibit access patterns that diverge sharply from traditional online transaction processing (OLTP) approaches. Large scans, unpredictable queries, vector search, low-latency inference, and distributed applications are redefining how teams address schema design, indexing, scaling, and operational strategy.

Vinodkrishna Gopalan is a subject-matter expert with more than 20 years of experience building and scaling enterprise software, with a strong focus on database architecture and performance. At Amazon Web Services (AWS), he led engineering initiatives for relational databases, application networking, and middleware powering millions of applications. Drawing on his expertise in distributed systems, cloud-native architecture, and AI-driven data access, Gopalan offers insights about how relational databases are evolving for AI and machine learning (ML) workloads.

Q: How are AI and ML workloads changing traditional database traffic patterns and performance assumptions?

Gopalan: Traditional relational databases are architected for online transaction processing (OLTP) workloads, characterized by small, fast, and frequent read-and-write operations. AI and ML workloads operate differently. Training pipelines require reading vast quantities of historical data, often terabytes, and place significant pressure on storage, network, and compute resources, far exceeding the demands typically associated with transactional workloads.

Query behavior also changes. Traditional applications usually have predictable access patterns, making it easier to tune queries, configure indexes, and implement caching. ML workloads are often more exploratory and ad hoc, meaning they may not follow the patterns a typical optimizer expects.

There is also a distinction between training and inference. Training workloads can tolerate slightly higher latency per operation as long as the overall process remains fast. Real-time inference, on the other hand, often requires lower latency than traditional applications. Resources can become overloaded in unanticipated ways, and standard caching methods may not work when workloads scan large amounts of data rather than repeatedly accessing the same datasets.

Q: How can teams design relational schemas that scale without costly rewrites as data volumes grow?

Gopalan: The key is to think about partitioning from the beginning. This does not mean setting up complex partitioning right away. Instead, companies can make schema choices that allow for partitioning later, reducing the need for major migrations as data volumes grow. When schema changes become necessary, managed blue/green deployment capabilities enable switchovers in under a minute, significantly reducing the risk and downtime of structural evolution.

Partition keys are most effective when they match how data is accessed. Time-based partitioning works well for time-series data, while hash-based strategies help spread the load. Each approach has trade-offs. Time-based models make archiving easier but can create hot spots if most queries target recent data. Hash-based models spread writes more evenly but can make time-based queries or archiving more challenging.

Separating hot, often-accessed data from cold, archived data helps reduce input/output (I/O) pressure and supports tiered storage strategies. Normalization is still useful for reducing redundancy, but sometimes selective denormalization, using materialized views or precomputed tables, can help frequent queries avoid costly joins. Even small data type choices matter. Saving a few bytes per row adds up when there are billions of rows.

Q: When does horizontal scaling become more useful than vertical scaling?

Gopalan: Horizontal scaling makes sense when the workload is already distributed. Multi-tenant software as a service (SaaS) platforms, geographically distributed applications, and systems that use standard hardware often take this route. Vertical scaling can help temporarily, but larger servers do not always provide equal improvements in central processing unit (CPU), memory, storage, and network. Serverless database architectures increasingly blur this boundary by automatically scaling capacity based on demand, particularly for workloads with unpredictable burst patterns and long idle periods.

Reducing disruption depends on effective architecture and disciplined operations. Read replicas can help with migrations by allowing companies to promote a replica during a planned maintenance window. Database proxies and connection pooling protect applications from backend changes by handling connections on the client side.

Connection management is often overlooked, yet it is one of the most common causes of production outages at scale. Direct application connections can quickly exhaust database resources, especially with microservices or AI workloads that spin up ephemeral workers. A proxy layer consolidates and manages connections, preventing cascading failures and making scaling and failover significantly easier.

Q: What sharding approaches best balance query performance with operational simplicity?

Gopalan: Sharding adds more complexity. Vertical sharding by business domain, such as products, orders, or inventory, works well with microservices. Tenant-based sharding is best for multi-tenant SaaS applications because it creates clear boundaries. Geographic sharding keeps data close to users and helps meet data residency requirements.

Using a mix of strategies is usually more practical than sticking to just one. The key is to know which data should stay together, which can be spread out, and where cross-shard operations might get costly.

Q: How can database architecture evolve as AI-powered workloads bring new patterns of data access and throughput?

Gopalan: Leading relational database systems now integrate native vector data types and approximate nearest neighbor indexes, directly supporting retrieval-augmented generation (RAG) and semantic search workloads. Comparisons between vector and traditional databases illustrate the blurring lines between relational and AI-native platforms.

Change data capture and streaming architectures are now more essential than ever. AI inference workloads may require low-latency feature serving, and recovery benefits from replaying missed changes.

Another major trend is separating storage and compute. Training workloads might need large parallel scans, while inference requires fast, predictable lookups. Separating storage and compute allows each resource to scale independently, enabling teams to increase inference-read capacity without affecting training throughput. Lakehouse architectures, outlined in the Azure Databricks documentation, further support unified data access across analytical, transactional, and AI workloads.

Security is also evolving. Agentic workflows raise new questions about context-aware authorization. Traditional resource-based controls might not be detailed enough when an AI agent requests data for a user or workflow. Fine-grained authentication, authorization, and protection against prompt injection will become more important.

Q: Who is responsible for the effective implementation, maintenance, and scaling of database architecture?

Gopalan: Responsibility is distributed across several specialized roles. Database architects and engineers are responsible for modeling, partitioning, replication, and capacity planning. Site reliability engineers oversee monitoring, incident response, backup and recovery, automation, and uptime.

Platform and infrastructure teams administer deployment, resource management, and governance, while application developers critically influence performance through query design and connection usage. Successful database architecture demands disciplined coordination among all contributors.

The talent model is also becoming more mixed. Teams perform more effectively when database experts know cloud-native patterns, developers understand data access costs, and site reliability engineers are aware of workload changes. AI can help identify hotspots, summarize signals, and recommend actions, but critical decisions still require human judgment. Application architecture and database design remain tightly interconnected, particularly in distributed microservices environments that rely on atomicity, consistency, isolation, and durability (ACID) transactions.

Q: What trends stand out as relational databases adapt to AI-driven systems?

Gopalan: Several trends are converging. Databases will continue to incorporate AI-native capabilities, including vector support, adaptive indexing, query optimization, and self-tuning. Unified platforms will support transactional, analytical, and vector workloads with less data movement. Distributed systems will also continue to advance, with stronger consistency and global scalability models, such as those described in Google Cloud’s overview of Spanner, TrueTime, and external consistency across geographically distributed servers.

Design for Reality, Scale With Flexibility

The primary lesson surrounding relational databases is to design for real workloads, rather than assumptions. Start simple, monitor carefully, avoid optimizing too soon, and ensure engineering teams can partition, evolve their schema, and scale as needed. Relational databases are not going away in the AI era. Instead, they will become part of a broader, more flexible data architecture.

About the Author

Paul Chaney is a seasoned writer, editor, and content strategist who helps businesses craft compelling, ethical marketing narratives through his consultancy, Prescriptive Writing. With a focus on clarity, authenticity, and responsible communication, Paul empowers organizations to tell their stories with purpose and precision. Connect with him on LinkedIn.

Disclaimer: The authors are completely responsible for the content of this article. The opinions expressed are their own and do not represent IEEE’s position nor that of the Computer Society nor its Leadership.