Aussie AI

Chapter 16. Building an AI Project

  • Book Excerpt from "Generative AI Applications: Planning, Design and Implementation"
  • by David Spuler

Chapter 16. Building an AI Project

I promised myself that this book would be code-free. But here I am about to write about software development. I’ll have to smack my wrist every time I start typing an assignment statement.

At a high level, given the problem of how to build an AI project, you should just put out an RFP. Then you will get about 3,793 responses from “AI expert companies”, all offering to code it fast, usually from a small corner of the planet with low taxes.

One way to reduce that is to type “build an AI app” into Google, and just click on the top 27 ads, which all appear above any useful content. Alternatively, you could ask ChatGPT for suggestions, and you’ll get some overly general advice, which is kind of like what I’m going to write anyway.

Failing that, you can ask your internal R&D staff whether they can build it. Yes, of course we can!, will be the shrill reply. Get ready for the budget requests for coding copilots and offsite training in Hawaii.

The only question is whether you should believe any of them. Because everyone is confused about building AI apps, and I don’t think anybody really knows how to do it well.

Alas, that also includes myself. I don’t presume to know the one true way of AI software development. Oh well, at least it’ll be a short chapter.

AI Application Development

There are many ways to build an AI app, and many companies will happily help you. Your main choices are basically:

  • Consulting companies
  • Hyperscaler cloud platforms
  • LLM-specific platforms
  • Specific LLM components
  • Startups in all of the above

Earlier chapters already discussed the various project requirements and design issues, but a reminder of the main questions you need to ask yourself:

  • Commercial versus open source?
  • Cloud-hosted versus on-premises?
  • Turnkey versus component-wise architectures?

How much of this do you want to build yourself? Let’s choose your own adventure:

    1. None — buy a turnkey app and configure it.

    2. All — build your whole architecture, including LLMs and Transformer engines.

    3. Something in-between — use an existing platform, but extend.

You’ve probably already figured what I’m going to recommend: focus on business-specific areas. Don’t rebuild everything from the ground up, because it’s unbelievably difficult (and costly). But also don’t just throw money at someone else to build everything. You’re just paying for them to become AI experts with your money. As I’ve said earlier, your goal should be to build your own core competency in the use of AI for business-aligned purposes.

The middle ground is to carefully choose between using third-party “AI infrastructure” (whether commercial or open source), and then take control of the “business layer.” Hence, one of the main choices is the “platform” or “backbone” on which you can build multiple AI applications for your business.

AI Application Platforms

There are numerous notable AI platforms with hosted LLMs and various higher-level functionality built on top including the hyperscalers and various heavily-funded AI companies. There are also about 65,000 AI startups to choose from, ranging from AI-specific hosters to add-on component products, and I will have finished reviewing them by about the 37th edition of this book. Let’s look at some of the options.

AI Model Startups. The various AI companies that are experts in models are now offering full platforms to build applications on top:

  • OpenAI (ChatGPT)
  • Cohere
  • Anthropic (Claude)
  • Mistral
  • Hugging Face
  • H2O.ai
  • Together AI
  • X AI (Grok)
  • Fireworks AI

Meta’s Llama. There’s one massive company that’s really in its own category: Meta. It’s a public company that’s not a hyperscaler, but has forged ahead with a massive amount of LLM research work, and now has not only very capable “Llama” models, but also its own platform for building on top. However, it has embraced the open source ethos, and therefore has a distinct offering.

Hyperscalers. The hyperscalers and hosting providers now offer AI capabilities:

  • Azure
  • AWS
  • GCP

Cloud hosters. Below the “top three” hyperscalers, there are numerous other incumbent players in the pre-AI hosting market, with a smaller market share, but nevertheless strong capabilities:

  • Oracle
  • Alibaba
  • OVH

AI-Specific Hosters (GPU Hosting). Various startups have emerged that offer cheaper AI-specific hosting on GPUs, such as:

  • HuggingFace
  • Lambda Labs
  • RunPod
  • Vast AI
  • Linode
  • Ori.co
  • Northern Data
  • Salad

Data Companies. There are the “data warehouse” or database companies with AI capabilities now:

  • Oracle
  • Databricks (acquired MosaicML)
  • Snowflake

Business App Platforms. There are also various platforms that have been offering business application development platforms for years, which are now re-tooled with extra AI capabilities. Companies include:

  • SalesForce
  • ServiceNow
  • IBM

IBM has been in the AI space for a long time, even before LLMs were a thing. Watson was often the face of IBM AI, but they have been doing the ML side of “AI” for a long time

AI Orchestration Platforms. There are numerous other technology companies with AI platform capabilities:

  • OctoAI
  • Datadog
  • Datastax
  • DataRobot
  • Scale AI

AI Writing Integrations. If your requirements are in a limited area of AI use cases, such as writing or collaboration, various companies in this space offer API and integration capabilities that compete against the various general AI platforms. Some examples include:

  • Notion
  • Grammarly
  • Jasper
  • Writer.com

AI Image Models. There are various companies in the image generation or image editing space. Various integrations and usages are possible with companies such as:

  • Adobe (Firefly)
  • Dall-E-2 (OpenAI)
  • Stable Diffusion
  • Runway
  • Midjourney
  • Canva
  • Figma
  • Craiyon

Open Source Generative AI Platforms. Some of the notable open source platforms for general use cases include:

  • PyTorch
  • TensorFlow
  • LangChain
  • LlamaIndex
  • Llama.cpp
  • vLLM
  • Keras
  • Ollama
  • DeepSeek

So, you should choose your top 30 options, write them in a list, and then throw five dice. And I’m only half joking because, honestly, any of the above would be fine.

Vendor Lock-in versus Open Source Platforms

The biggest problem with all of the above commercial AI platform options is that you will have difficulty swapping it out later. As your use of AI grows, so too will the checks that you need to write to your platform vendor.

But it’s difficult to be platform agnostic, and still have all the advanced features. The main choice is the various open source AI platforms, but these are not as well-resourced as the commercial options. Many are probably not yet at the same level of maturity with regard to production-level scaling and manageability of the final applications.

References

  1. Aarushi Kansal, 2024, Building Generative AI-Powered Apps: A Hands-on Guide for Developers, Apress, https://www.amazon.com/Building-Generative-AI-Powered-Apps-Hands-ebook/dp/B0CTXXP1S4/
  2. Louis-François Bouchard, Louie Peters, May 2024, Building LLMs for Production: Enhancing LLM Abilities and Reliability with Prompting, Fine-Tuning, and RAG, https://www.amazon.com/Building-LLMs-Production-Reliability-Fine-Tuning/dp/B0D4FFPFW8/
  3. Chip Huyen, Jul 25, 2024, Building A Generative AI Platform, https://huyenchip.com/2024/07/25/genai-platform.html
  4. Juan Pablo Bottaro, April 25, 2024, Musings on building a Generative AI product, https://www.linkedin.com/blog/engineering/generative-ai/musings-on-building-a-generative-ai-product?_l=en_US
  5. Xiang Chen, Chaoyang Gao, Chunyang Chen, Guangbei Zhang, Yong Liu, 12 Aug 2024 (v2), An Empirical Study on Challenges for LLM Developers, https://arxiv.org/abs/2408.05002
  6. Chaojun Xiao, Zhengyan Zhang, Chenyang Song, Dazhi Jiang, Feng Yao, Xu Han, Xiaozhi Wang, Shuo Wang, Yufei Huang, Guanyu Lin, Yingfa Chen, Weilin Zhao, Yuge Tu, Zexuan Zhong, Ao Zhang, Chenglei Si, Khai Hao Moo, Chenyang Zhao, Huimin Chen, Yankai Lin, Zhiyuan Liu, Jingbo Shang, Maosong Sun, Sep 2024, Configurable Foundation Models: Building LLMs from a Modular Perspective, https://arxiv.org/pdf/2409.02877
  7. Lior Solomon, Sep 2024, Gen AI testing strategies and tools, https://medium.com/ai-in-grc/gen-ai-testing-strategies-and-tools-257383e5cbfb
  8. Matt Asay, Sep 23, 2024, Too much assembly required for AI, https://www.infoworld.com/article/3536292/too-much-assembly-required-for-ai.html
  9. Melissa Malec, June 5, 2024, AI Orchestration Explained: The What, Why & How for 2024, https://hatchworks.com/blog/gen-ai/ai-orchestration/
  10. Gary Grossman, September 8, 2024, AI orchestration: Crafting harmony or creating dependency? https://venturebeat.com/ai/ai-orchestration-crafting-harmony-or-creating-dependency/
  11. A. R. Ali, K. Kumar, M. A. Siddiqui and M. Zahid, 2024, An Open-source Cross-Industry and Cloud-agnostic Generative AI Platform, 2024 International Joint Conference on Neural Networks (IJCNN), Yokohama, Japan, 2024, pp. 1-10, doi: 10.1109/IJCNN60899.2024.10650688, https://ieeexplore.ieee.org/abstract/document/10650688
  12. Ben Auffarth, Dec 22, 2023 Generative AI with LangChain: Build large language model (LLM) apps with Python, ChatGPT and other LLMs, https://www.amazon.com/Generative-AI-LangChain-language-ChatGPT/dp/1835083463/
  13. Adva Nakash Peleg, May 30, 2024, An LLM Journey: From POC to Production, https://medium.com/cyberark-engineering/an-llm-journey-from-poc-to-production-6c5ec6a172fb
  14. Dr Kris Jamsa, Dec 2023, OpenAI and ChatGPT Programming: Using Python to Unlock OpenAI and ChatGPT, https://www.amazon.com/OpenAI-ChatGPT-Programming-Python-Unlock/dp/B0CQK41P6B/
  15. Raymond Lo, Jul 10, 2024, How to Build Faster GenAI Apps with Fewer Lines of Code using OpenVINO™ GenAI API, https://medium.com/openvino-toolkit/how-to-build-faster-genai-apps-with-fewer-lines-of-code-using-openvino-genai-api-5dd5fcabea17

 

• Online: Table of Contents

• PDF: Free PDF book download

• Buy: Generative AI Applications: Planning, Design and Implementation

Generative AI in C++ The new Generative AI Applications book by Aussie AI co-founders:
  • Deciding on your AI project
  • Planning for success and safety
  • Designs and LLM architectures
  • Expediting development
  • Implementation and deployment

Get your copy from Amazon: Generative AI Applications