A normal add instruction processes one value. Load one float, add one float, store one float. SIMD(Single Instruction Multiple Data) loads a batch of values into one wide register and applies the same operation to the whole batch in a single instruction. Four floats added to four floats, or eight, or sixteen. Not a faster core, not a shortcut at the algorithm level. The instruction itself works on a batch instead of a single value.
What SIMD is at the Hardware Level
Every arithmetic instruction on a scalar CPU operates on one value per register. A 32-bit float lives in 32 bits of a register. The add instruction reads two registers, adds their contents, writes one result. That is one float per instruction.
SIMD repurposes the same register file with wider registers. Instead of one 32-bit float, a 256-bit register holds eight 32-bit floats packed side by side. The add instruction now reads two 256-bit registers and adds all eight corresponding pairs in parallel. One instruction, eight additions, same latency as a scalar add. The silicon for doing eight additions in parallel exists in the execution unit, and the wider register format is how the CPU knows to use it.
The packed layout is what makes this work. The eight floats in a 256-bit register are not eight separate values that happen to be stored nearby, they are one operand from the instruction’s point of view. The instruction encodes that all eight lanes get the same operation applied independently and simultaneously. There is no coordination between lanes, no carry from one to the next, just eight independent additions happening at the same time inside one execution unit.
Superscalar and SIMD: Two Different Things
Superscalar execution is a CPU issuing multiple separate instructions in a single clock cycle, each one going to a different execution unit. Three independent integer adds can all retire in the same cycle if the CPU has three integer Arithmetic Logic Units (ALUs) and no data dependency prevents it. That is more instructions at once.
SIMD is one instruction doing more work. The instruction count does not increase, the work per instruction does. A CPU runs both mechanisms simultaneously. Superscalar handles the breadth of independent instructions the compiler emits per cycle. SIMD handles the width of each individual instruction’s operands. They stack: a superscalar CPU running SIMD instructions gets both the Instruction-Level Parallelism (ILP) from superscalar and the data-level parallelism from the wide registers.
Confusing the two matters because they have different performance implications. Superscalar throughput is limited by the number and types of execution units. SIMD throughput is limited by register width and whether the operation is compute-bound at all, which the benchmark section covers.
SSE, AVX, and AVX-512: the x86 SIMD Extensions
SSE (Streaming SIMD Extensions), AVX (Advanced Vector Extensions), and AVX-512 are Instruction Set Architecture (ISA) extensions added to x86 over time, each generation widening the registers.
SSE introduced 128-bit XMM registers: four 32-bit floats, two 64-bit doubles, or various integer widths per register. AVX doubled the register width to 256 bits, adding YMM registers and renaming the 128-bit operations to use the lower half of the same registers. AVX-512 doubled again to 512-bit ZMM registers and added masking operations that scalar and earlier SIMD could not express efficiently. The batch size scales with register width: 128 bits gives four floats, 256 gives eight, 512 gives sixteen. The arithmetic itself does not change, just how many values each instruction covers.
The intrinsic naming convention encodes this. _mm_ prefix is SSE (128-bit), _mm256_ is AVX (256-bit), _mm512_ is AVX-512 (512-bit). The operation name follows: add, mul, load, store. The suffix encodes the element type: _ps is packed single-precision float, _pd is packed double, _epi32 is packed 32-bit integer. _mm256_add_ps therefore means: AVX, add, packed floats, eight of them. Once the naming pattern is clear, the intrinsics guide becomes readable rather than opaque.
Not all hardware supports all extensions. -march=native tells the compiler to target the CPU running the build and enables every extension it supports. -mavx2 enables AVX2 explicitly without requiring a native build. Compiling AVX intrinsics without one of these flags produces an error since the target would have no instruction to emit them to.
Auto-Vectorization: What the Compiler Needs to See
Auto-vectorization is the compiler recognizing a scalar loop and replacing it with SIMD instructions on its own. The loop over out[i] = a[i] + b[i] is exactly the kind of pattern it looks for. Whether it applies depends on what the compiler can prove.
The requirements are specific. No loop-carried dependency: the result of iteration i must not feed into iteration i+1. A loop that accumulates into a single variable total += a[i] has a dependency on every step and cannot be vectorized without reduction instructions. No aliasing: if out could point to the same memory as a or b, writing out[i] during iteration i might change what a[i+1] or b[i+1] reads, which would produce a different result from the scalar version. The compiler has to either prove no alias exists or generate both a vectorized and a scalar path with a runtime check to pick between them. The -O3 objdump of add_normal shows this directly: GCC emits a runtime aliasing check before the vectorized loop because the function takes two separate const references and cannot prove at compile time that they do not overlap. The -O2 build does not vectorize add_normal at all.
-fopt-info-vec-optimized makes the compiler report every loop it successfully auto-vectorized. Running this at -O2 produces no output for add_normal. Running it at -O3 reports the loop by name. That single flag is the fastest way to check whether a loop got the vectorized treatment without reading disassembly, though the disassembly confirms what instruction was actually emitted.
Loop unrolling is a separate transformation and is worth distinguishing explicitly. Unrolling takes one loop body and replicates it multiple times per iteration, reducing the loop overhead and exposing more independent work for the CPU’s superscalar backend to run in parallel. It is an ILP optimization, not a SIMD one. A compiler can unroll a scalar loop without vectorizing it, and it can vectorize a loop without unrolling it, and it can do both at the same time. They are independent passes that happen to interact. The benchmark’s compute_normal does something structurally similar to unrolling manually: four independent chains across four different elements, giving the CPU parallel work to fill the serial dependency stall on each chain. That is ILP exploitation, not vectorization.
Writing Intrinsics Directly
Intrinsics give direct access to SIMD instructions from C++. Each intrinsic is a compiler built-in that maps to exactly one instruction. What you write is what gets emitted, no auto-vectorizer discretion involved.
std::vector<float> add_simd(const std::vector<float>& a, const std::vector<float>& b) {
std::vector<float> out(a.size());
size_t i = 0;
for (; i + 8 <= a.size(); i += 8) {
__m256 va = _mm256_loadu_ps(&a[i]);
__m256 vb = _mm256_loadu_ps(&b[i]);
__m256 vsum = _mm256_add_ps(va, vb);
_mm256_storeu_ps(&out[i], vsum);
}
for (; i < a.size(); i++)
out[i] = a[i] + b[i];
return out;
}
__m256 is a 256-bit type that maps to a YMM register. It is a compiler extension, not a standard C++ type. Eight floats fit in it. _mm256_loadu_ps loads eight floats from the pointer into a YMM register. The u in loadu means unaligned: the pointer is not required to be 32-byte aligned. The aligned version, _mm256_load_ps, requires 32-byte alignment and faults or behaves incorrectly on a misaligned address. std::vector’s default allocator does not guarantee 32-byte alignment on its data, so the unaligned load is the correct choice here unless the allocator is replaced with one that provides it. On modern microarchitectures the penalty for an unaligned load that crosses a cache line boundary is small, and for loads that happen to be naturally aligned it is zero.
The tail loop handles the remaining elements when the array size is not a multiple of eight. AVX operates in chunks of eight floats, so any array whose length is not divisible by eight leaves a remainder that the SIMD loop cannot touch. The scalar fallback loop at the end covers those. This is standard practice for any SIMD loop: a main SIMD body covering as many full chunks as possible, followed by a scalar tail for the remainder.
The resulting assembly for add_simd contains a single vaddps ymm, ymm, ymm per loop iteration. add_normal compiled at -O2 contains a scalar vaddss per iteration. Same operation, eight times fewer instructions for the SIMD version.
Memory-Bound vs Compute-Bound: When SIMD Actually Helps
The add_normal and add_simd functions exist to show the codegen difference, not to produce a wall-clock speedup. Running them as a timed benchmark showed they land within noise of each other at every array size tested, and the reason is worth understanding because it applies to any SIMD workload.
A simple addition, out[i] = a[i] + b[i], does one floating-point operation per float loaded. Loading a float from memory takes many cycles relative to the time the arithmetic unit spends on the add. The CPU spends nearly all its time waiting on memory, not computing. SIMD does not change memory bandwidth. A 256-bit load brings in eight floats at once instead of one, but memory bandwidth is measured in bytes per second and eight floats is eight times more bytes. The bandwidth consumed per useful result does not change. The operation is memory-bandwidth-bound, and SIMD’s benefit is in compute throughput, not bandwidth.
The timed speedup becomes measurable when the arithmetic-to-memory ratio goes up. The benchmark adds a tight multiply-add loop per element, x = x * y + a[i] repeated ten thousand times per float. Now the CPU is doing ten thousand multiply-adds per float loaded. Arithmetic dominates, memory waits are a small fraction of total time, and the difference between eight-wide SIMD and scalar shows up clearly in the wall clock.
This distinction matters practically. Before reaching for intrinsics, confirming whether the bottleneck is compute or memory determines whether SIMD is the right tool. A memory-bound loop will not get faster from intrinsics. A compute-bound loop will.
Designing a Benchmark: ILP vs Width
The benchmark has three modes: normal, simd, and simd4. The distinction between them is more careful than it looks.
compute_normal runs four independent chains across four different elements per outer iteration:
x0 = x0 * y0 + a[i];
x1 = x1 * y1 + a[i+1];
x2 = x2 * y2 + a[i+2];
x3 = x3 * y3 + a[i+3];
Each chain is a serial dependency: step k+1 cannot start until step k resolves, since x0 at step k+1 depends on x0 at step k. A single chain would stall the CPU at every step waiting for the multiply-add result. Four independent chains give the CPU parallel work to fill that wait, exposing instruction-level parallelism. The chains read four different elements specifically so the compiler cannot prove they produce identical results and collapse them. An earlier attempt used four accumulators starting from the same value running the same formula, which the compiler correctly recognized as identical computations and reduced back to one chain. Confirmed in the disassembly before catching it.
compute_simd is a single eight-wide vector chain: the same serial dependency structure as one scalar chain, just operating on eight values at once per step. Comparing normal against simd measures what happens when normal has four-way ILP and simd has one chain. The result is roughly 2x in favor of simd, which is smaller than the register width would suggest because the ILP advantage in normal partially offsets the width disadvantage.
compute_simd4 gives the vector version the same four-chain treatment: four independent __m256 chains each covering eight elements, 32 elements per outer iteration total. Now both sides have equal ILP and the only remaining difference is the 8x register width. The result is roughly 8x in favor of simd4, matching AVX’s theoretical ceiling.
Neither number is wrong. Normal vs simd isolates the ILP comparison. Normal vs simd4 isolates the width comparison. Conflating them is how SIMD speedup claims end up overstated or understated relative to what the hardware actually provides.
Run: main.cpp
g++ -O2 -std=c++20 -march=native main.cpp -o main && ./main
-march=native targets the CPU running the build and is required since AVX intrinsics do not compile without an AVX-enabled target. Expect outputs match: yes.
$ ./main
outputs match: yes
Run: codegen at -O2
g++ -O2 -std=c++20 -march=native -c main.cpp -o main-o2.o
objdump -d -M intel --no-show-raw-insn main-o2.o | grep -A 15 "add_normal\|add_simd"
-c compiles to an object file without linking. grep -A 15 keeps each matched function and the 15 lines after it, cutting the startup and library glue code objdump otherwise prints. Look at add_normal first for the scalar path, then add_simd for the intrinsic path.
$ objdump -d -M intel --no-show-raw-insn main-o2.o | grep -A 15 "add_normal\|add_simd"
<add_normal(...)>:
...
c0: vmovss xmm0,DWORD PTR [rsi+rax*4]
c5: vaddss xmm0,xmm0,DWORD PTR [rdi+rax*4] <-- scalar, one float per instruction
ca: vmovss DWORD PTR [rcx+rax*4],xmm0
cf: inc rax
d2: cmp rax,rdx
d5: jb c0
<add_simd(...)>:
...
204: vmovups ymm0,YMMWORD PTR [rsi+rcx*1]
210: vaddps ymm0,ymm0,YMMWORD PTR [rax+rcx*1] <-- 8-wide AVX add, present at -O2
219: vmovups YMMWORD PTR [rax+rcx*1],ymm0
...
260: vmovss xmm0,DWORD PTR [rsi+rdx*4]
265: vaddss xmm0,xmm0,DWORD PTR [rdi+rdx*4] <-- scalar tail loop, unvectorized at -O2
26a: vmovss DWORD PTR [rcx+rdx*4],xmm0
add_normal has only vaddss in its loop body, scalar, no ymm register anywhere. add_simd shows vaddps ymm in its main loop already at -O2, confirming intrinsics do not need -O3: the instruction written is the instruction emitted. The tail loop inside add_simd handling leftover elements is plain vaddss here, matching what the vectorizer report below shows.
Run: codegen at -O3
g++ -O3 -std=c++20 -march=native -c main.cpp -o main-o3.o
objdump -d -M intel --no-show-raw-insn main-o3.o | grep -A 40 "add_normal"
-A 40 since the vectorized version is significantly longer than the scalar one. Look for three separate paths in add_normal: a main vectorized body, an epilogue for leftovers, and a scalar fallback for when the alias check fails.
$ objdump -d -M intel --no-show-raw-insn main-o3.o | grep -A 40 "add_normal"
<add_normal(...)>:
...
120: vmovups ymm0,YMMWORD PTR [rsi+rax*1]
125: vaddps ymm0,ymm0,YMMWORD PTR [rdx+rax*1] <-- main vectorized path, 8-wide
12a: vmovups YMMWORD PTR [rcx+rax*1],ymm0
...
15c: vmovups xmm0,XMMWORD PTR [rsi+rax*4]
164: vaddps xmm0,xmm0,XMMWORD PTR [rdx+rax*4] <-- epilogue, 4-wide, leftover after 8-wide chunks
171: vmovups XMMWORD PTR [rcx+rax*4],xmm0
...
220: vmovss xmm0,DWORD PTR [rdx+rax*4]
225: vaddss xmm0,xmm0,DWORD PTR [rsi+rax*4] <-- scalar fallback, taken if alias check fails
22a: vmovss DWORD PTR [rcx+rax*4],xmm0
Three separate code paths in one function. The main 8-wide ymm path is what the vectorizer report’s “loop vectorized using 32 byte vectors” line names. The 4-wide xmm epilogue handles whatever elements are left after full 8-wide chunks are exhausted, matching “epilogue loop vectorized using 16 byte vectors” exactly. The plain scalar block at the end is the fallback taken only if the runtime alias check between a and b fails, making the “loop versioned for vectorization because of possible aliasing” note concrete: that code path physically exists in the binary.
Run: vectorizer report
g++ -O2 -std=c++20 -march=native -fopt-info-vec-optimized -c main.cpp -o /dev/null
g++ -O3 -std=c++20 -march=native -fopt-info-vec-optimized -c main.cpp -o /dev/null
-fopt-info-vec-optimized makes the compiler report every loop it successfully auto-vectorized, in its own words rather than inferred from disassembly. Output goes to /dev/null since the object file is not needed here.
$ g++ -O2 -std=c++20 -march=native -fopt-info-vec-optimized -c main.cpp -o /dev/null
/nix/store/hkldzpgigap7kkqzdr5j4qqyy5ac4l6x-gcc-16.1.0/include/c++/16.1.0/bits/stl_vector.h:106:4: optimized: basic block part vectorized using 16 byte vectors
At -O2, the only vectorization reported is inside std::vector’s own internals, its zero-initializing constructor. Nothing mentions add_normal’s loop, confirming no auto-vectorization happened there. add_simd’s tail loop is also absent since intrinsics are explicit instructions, not auto-vectorization candidates the report tracks.
$ g++ -O3 -std=c++20 -march=native -fopt-info-vec-optimized -c main.cpp -o /dev/null
/nix/store/hkldzpgigap7kkqzdr5j4qqyy5ac4l6x-gcc-16.1.0/include/c++/16.1.0/bits/stl_vector.h:106:4: optimized: basic block part vectorized using 16 byte vectors
main.cpp:7:26: optimized: loop vectorized using 32 byte vectors and unroll factor 8
main.cpp:7:26: optimized: loop versioned for vectorization because of possible aliasing
main.cpp:7:26: optimized: epilogue loop vectorized using 16 byte vectors and unroll factor 4
main.cpp:21:14: optimized: loop vectorized using 16 byte vectors and unroll factor 4
main.cpp:21:14: optimized: loop versioned for vectorization because of possible aliasing
main.cpp:21:14: optimized: epilogue loop vectorized using 8 byte vectors and unroll factor 2
/nix/store/hkldzpgigap7kkqzdr5j4qqyy5ac4l6x-gcc-16.1.0/include/c++/16.1.0/bits/stl_algobase.h:1196:20: optimized: loop vectorized using 32 byte vectors and unroll factor 8
/nix/store/hkldzpgigap7kkqzdr5j4qqyy5ac4l6x-gcc-16.1.0/include/c++/16.1.0/bits/stl_algobase.h:1196:20: optimized: loop versioned for vectorization to enhance alignment
main.cpp:7:26 is add_normal’s loop, vectorized with 32-byte vectors and unrolled 8 times, with a runtime alias check inserted. The bonus finding is main.cpp:21:14: the scalar tail loop inside add_simd that handles leftover elements after full batches of 8 gets auto-vectorized at -O3 on its own, using 16-byte vectors. It handles at most 7 elements, so 16-byte (4-float) vectors are the widest that fit usefully. This loop was plain scalar at -O2 as the objdump confirms. The alias check appears here too since the tail loop also reads from two separate references with the same aliasing concern.
Run: benchmark.cpp
g++ -O2 -std=c++20 -march=native benchmark.cpp -o benchmark
./benchmark normal
./benchmark simd
./benchmark simd4
-march=native is required for AVX. Run each mode as a separate invocation so no result is biased by cache state left warm from another. The printed checksum should be identical across all three modes, confirming every version computed the same answer regardless of how the arithmetic was organized.
$ ./benchmark normal
normal: 3466099 us, checksum=1.17644e+09
$ ./benchmark simd
simd: 1738350 us, checksum=1.17644e+09
$ ./benchmark simd4
simd4: 433877 us, checksum=1.17644e+09
normal vs simd: 3466099 / 1738350 = 1.99x. normal vs simd4: 3466099 / 433877 = 7.99x, essentially the full theoretical ceiling for 256-bit AVX. Confirmed on a second machine with a different GCC version at 7.97x, same shape and same conclusion.
Quick Reference
Coming from other languages
Most languages with standard library math operations rely on the auto-vectorizer for SIMD, and the same requirements apply regardless of the language: no aliasing between operand arrays, no loop-carried dependencies, predictable iteration counts. Languages with array-native operations or explicit vector types expose the same hardware-level batch processing under a different syntax. The register width and batch size are hardware constants that no language layer changes, and whether an operation is memory-bound or compute-bound is equally hardware-determined. SIMD helps compute-bound operations in any language that can emit the right instructions.
The 90% mental model
SIMD is one instruction operating on a batch of values instead of one. The batch size is the register width: 128 bits for SSE (4 floats), 256 for AVX (8 floats), 512 for AVX-512 (16 floats). Auto-vectorization applies this silently when the compiler can prove no aliasing and no loop-carried dependencies, but it is never guaranteed and the level at which it kicks in varies by optimization flag. Writing intrinsics directly bypasses that uncertainty entirely. Whether SIMD improves runtime performance depends on whether the operation is compute-bound: a simple add is memory-bound and SIMD changes nothing at the clock level regardless of how many values each instruction handles.