Open Source C++ Transformer Engines

Open Source C++ Transformer Engines

You don't need to create a whole C++ Transformer engine from scratch. There are multiple fully coded AI engines, with C++ source code to download, that are available under permissive open source licenses.

Some of the useful C++ engines that are great to experiment with on your desktop PC, or even a laptop, include these options:

• GGML: https://github.com/ggerganov/ggml (MIT License)
• Llama.cpp: https://github.com/ggerganov/llama.cpp (MIT License)
• StarCoder from BigCode: https://github.com/bigcode-project/starcoder (Apache 2.0 License)
• Intel Extension for Transformers: https://github.com/intel/intel-extension-for-transformers (Apache 2.0 License)
• gemma.cpp: https://github.com/google/gemma.cpp (Apache 2.0 License and BSD-3-clause license)

You should check the current license status of these projects yourself, since many of these are not only free, but allow commercial usage. However, even permissive licenses have restrictions and obligations.