The AI Application Layer

December 9th, 2024

by David Spuler, Ph.D.

The AI Application Layer

The application layer for AI has not been much of a focus in the past. Most of the funding and attention has gone to the makers of GPUs (i.e., NVIDIA), and the companies that train frontier LLMs (e.g., OpenAI and Anthropic). Literally, billions of dollars in funding.

For the application layer, not so much.

What is the Application Layer?

The overall layers of the AI stack look something like this:

User Interface
Application Layer
Middleware
LLMs and AI Engine (Transformers)
Hardware layer (GPUs)

Applications are either consumer-specific, enterprise-specific (horizontal AI apps for HR, marketing, sales, etc.), or vertical-based (e.g., medical, finance). Generally speaking, these haven't been setting the world on fire. The big, billion-dollar funding rounds have been going to foundation model providers like OpenAI or Anthropic.

Why the Application Layer?

Proponents of investments in the application layer draw a parallel between AI advances and the cloud computing trend. The benefits from the trend of moving business applications from on-premises to the cloud did not benefit the incumbents (e.g., IBM), but instead created a whole new layer of cloud-specific companies providing AI infrastructure:

Amazon AWS
Microsoft Azure
Google GCP
And more more...

Not only this, but a whole set of new companies arose that provided "applications" on top of the cloud infrastructure. Some examples include:

Nutanix
Datadog
Salesforce
ServiceNow
Databricks
Snowflake
GoDaddy
Network Solutions
CrowdStrike
Odoo
NetSuite
Workday
Hubspot
Splunk
Okta

And the internet generally created some big new consumer-focused companies:

Google
Meta
Yahoo
Amazon
Ebay
PayPal
Zoom
Slack

Will we see this pattern again with AI?

History Repeats for AI Applications?

Will history repeat with AI, whereby the incumbents will not capitalize on the opportunity, and a whole new swathe of enterprises will create AI infrastructure? Seems doubtful at this point, since the "hyperscalers" (AWS, Azure and GCP) have captured much of the AI hosting market. But there are a number of AI-specific GPU hosting companies (e.g., Lambda Labs).

What about AI applications? Will there be a whole new set of AI-first application companies? This seems more likely, as the whole market for "AI applications" seems rather fragmented at this point. Not the least of which is the fact that there doesn't seem to be an "AI killer app" as yet, except perhaps the ChatGPT consumer app itself.

Related Research

Free AI and C++ Books

Generative AI programming books:

The Sweetest Lesson: Your Brain Versus AI, November 2025: full text online, free PDF available
RAG Optimization: Accurate and Efficient LLM Applications, June 2025: full text online, free PDF available
Generative AI Applications: Planning, Design and Implementation, November 2024: full text online, free PDF available
Generative AI in C++ (Spuler, March 2024): full text online, free PDF available, table of contents, bonus materials, reference lists, source code

CUDA C++ GPU Programming Books:

CUDA C++ Optimization: Coding Faster GPU Kernels, July 2024: full text online, bonus materials, free PDF available
CUDA C++ Debugging: Safer GPU Kernel Programming, July 2024: full text online, free PDF available

Modern C++ Programming Books

C++ AVX Optimization: CPU SIMD Vectorization, 2025: full text online, free PDF available
C++ Ultra-Low Latency: Multithreading and Low-Level Optimizations, 2025: full text online, free PDF available
Advanced C++ Memory Techniques: Efficiency and Safety, 2025: full text online, free PDF available
Efficient C++ Multithreading: Modern Concurrency Optimization, 2025: free PDF available
Efficient Modern C++ Data Structures: Container and Algorithm Optimizations, 2025: free PDF available
C++ Low Latency: Multithreading and Hotpath Optimizations, 2025: free PDF available
Safe C++: Fixing Memory Safety Issues, Oct 2024: full text online, free PDF available