Types of Buffer Overflows and Methods of Detection

Buffer overflows are exploits that hackers use against an operating system or application; like SQL injection attacks, they’re usually targeted at user input fields. A buffer overflow exploit causes a system to fail by overloading memory or executing a command shell or arbitrary code on the target system. A buffer overflow vulnerability is caused by a lack of bounds checking or a lack of input-validation sanitization in a variable field (such as on a web form). If the application doesn’t check or validate the size or format of a variable before sending it to be stored in memory, an overflow vulnerability exits.

The two types of buffer overflows are stack based and heap based.

The stack and the heap are storage locations for user-supplied variables within a run-ning program. Variables are stored in the stack or heap until the program needs them. Stacks are static locations of memory address space, whereas heaps are dynamic memory address spaces that occur while a program is running.

The picture represents a heap-based buffer overflow occurring in the lowest part of the memory and in that way it overwrites other dynamic variables.

Buffer Overflow Countermeasures

A hacker must know the exact memory address and the size of the stack in order to make the return pointer execute their code. A hacker can use a No Operation (NOP) instruction, which is just padding to move the instruction pointer and does not execute any code. The NOP instruction is added to a string before the malicious code to be executed. To bypass the IDS, the hacker can randomly replace some of the NOP instructions with equivalent pieces of code, such as: x++,x-;?NOPNOP
This example of a mutated buffer overflow attack can bypass detection by an IDS. Programmers should not use the built-in strcpy(), strcat(), and streadd() C/C++ functions because they are susceptible to buffer overflows. Alternatively, Java can be used as the programming language since Java is not susceptible to buffer overflows.