File:Aslr stack smash.svg

English: Illustration of the target of a stack based buffer overflow. The orange triangle indicates the target address for the injected shellcode or ret2libc attack.

In the case of shellcode, a NOP buffer will defeat minor randomization; however, a non-executable stack prevents the use of shellcode. This gives the attack an X/R probability of success, where X is the width of injected NOPs divided by the width of a randomization period and R is the number of possible positions the stack could end at.

The ret2libc case is possible with a non-executable stack; however, moving the stack by even a single byte will mis-align the injected stack frames, causing the attack to fail. The injected stack frames can be repeated, which gives the attack a Y/R probability of success, where Y is the number of times the stack frames are repeated on relative alignment to the width of a randomization period and R is the number of possible positions the stack could end at.