introduction

kael is a deep learning framework written from zero in modern C++. It has tensors with shapes and strides, views that share memory, a loop engine with broadcasting and type promotion, reverse-mode autograd, neural network modules and optimizers, a multithreaded CPU backend, a CUDA backend, Python bindings, and a GPT-2 that loads real weights and generates text.

None of it is a wrapper. Every layer is written by hand, tested against PyTorch, and explained in a book that rebuilds it step by step.

#include <iostream>

#include "kael/kael.h"

int main() {
    kael::Tensor x = kael::randn({64, 784});
    kael::Tensor w = kael::randn({784, 10}).requires_grad_();

    kael::Tensor loss = kael::relu(kael::matmul(x, w)).sum();
    loss.backward();

    std::cout << *w.grad() << "\n";   // dloss/dw, shape=[784, 10]
}

motivation

• Why does reshape sometimes copy and sometimes not?
• How does a + b work when a is [3, 1] and b is [4]?
• What does loss.backward() actually walk, and in which order?
• Why is a naive matrix multiply fifty times slower than a good one?
• What happens between a .safetensors file on disk and a sentence of generated text?

Frameworks such as PyTorch answer these questions in millions of lines of code. kael answers them in a codebase small enough to read in a few weeks, with every decision written down.

quick start

kael needs CMake 3.24 or newer and a C++20 compiler.

git clone https://github.com/JetBerri/kael.cpp
cd kael.cpp
cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

Then start with the overview.

the book

Fourteen chapters. Each one adds a layer, shows the real code and ends with steps to write that layer yourself.

1. 00 Overview The architecture, the repository map and the build
2. 01 Shapes and strides The index arithmetic that turns a flat buffer into an n-dimensional array, and broadcasting
3. 02 Memory model Data types, storage, TensorImpl, the Tensor handle, factories and random numbers
4. 03 Views Reshape, transpose, slicing and expand without copying data
5. 04 TensorIterator The loop engine behind every elementwise op: broadcasting, type promotion and dimension coalescing
6. 05 Operations Arithmetic, math functions, reductions, matmul, indexing and softmax, checked against PyTorch
7. 06 Autograd Reverse-mode automatic differentiation over a dynamic graph
8. 07 Neural networks and training Modules, losses, optimizers, data loading and an MNIST classifier
9. 08 CPU performance A thread pool, a fast matmul, a caching allocator and fused kernels
10. 09 CUDA backend Device memory, device kernels and a host emulation backend for testing
11. 10 Python bindings A Python package on top of the C++ library, with NumPy interop
12. 11 Transformer Attention, a GPT model, a KV cache and sampling
13. 12 Loading GPT-2 The safetensors format, the GPT-2 weight mapping and the BPE tokenizer
14. 13 Serving and tooling An HTTP server, debugging tools, fuzzing, sanitizers and CI

a growing knowledge base

The book explains how kael works. The concepts section explains the ideas around it: the math behind gradients, how memory hierarchies shape kernels, why attention works. It is written by the community and it is meant to keep growing.

Every page on this site is a Markdown file in a public repository. Fix a sentence, add an exercise, or write the article you wish you had read when you started. Read how to contribute, or request a topic you want explained.