View Proposal

---

Proposer

Mohammad Ammad-Uddin

Title

A Minimal Transformer Inference Engine: Design, Implementation, and Experimental Evaluation of a SIMD-Optimized Matrix Multiplication Kernel

Goal

To design and implement a minimal Transformer inference engine with a SIMD-optimized matrix multiplication kernel for a fixed target model and CPU architecture, and to determine, through controlled experimentation, the kernel's performance impact relative to a scalar baseline of verified numerical equivalence.

Description

This thesis presents the design and implementation of a minimal Transformer inference engine in Zig, targeting a single small pretrained language model (GPT-2-small class, ~117M parameters) on an x86_64 CPU with AVX2 support. The engine's matrix multiplication routine — the dominant computational cost of Transformer inference — is implemented behind a single shared interface with two interchangeable backends: a scalar baseline, used to establish correctness and serve as the point of comparison, and a SIMD-optimized kernel using Zig's @Vector type targeting AVX2, which constitutes the sole technical contribution of this work. Correctness is established prior to any performance claim. Because SIMD execution reorders floating-point summation, bit-exact equivalence with the scalar baseline cannot be assumed; correctness is instead defined as output equivalence within a stated numerical tolerance, verified at the level of model logits rather than generated text alone, since textual equivalence can mask numerical divergence at low temperature or short output length. Performance is evaluated using three metrics — throughput (tokens/sec), per-token latency, and speedup relative to the scalar baseline — measured under a fixed experimental protocol (fixed model, fixed input, repeated trials, reported with variance) to ensure the results are reproducible and not an artifact of a single favorable run. The evaluation further situates the measured speedup against the theoretical ceiling implied by AVX2's 8-wide single-precision throughput, so that the result speaks to why the achieved speedup falls where it does, not merely that it occurred. The scope is deliberately bounded to this single contribution. Memory management, model loading, and tokenization are implemented only to the extent required for a correct, working pipeline and are not presented as independent contributions. Networking, quantization, multi-model support, and non-x86 targets are explicitly out of scope and are not claimed, even as secondary results.

Resources

Background

Url

Difficulty Level

Easy

Ethical Approval

None

Number Of Students

1

Supervisor

Mohammad Ammad-Uddin

Keywords

Degrees

Bachelor of Science in Computational Sciences and Software Engineering