killchain-compendium/exploit/binaries/buffer_overflow/docs/amd64.md

amd64

• rax return value, caller saved.
• rbx base register (used for mem basepointer)
• rcx counter register
• r10, r11 are caller saved.
• rbx, r12, r13, r14 are callee saved
• rdx data register
• rbp is also callee saved(and can be optionally used as a frame pointer)
• rsp is callee saved
• rip next instruction pointer

Function argument registers

• rdi,rsi,rdx,rcx,r8 ,r9 , called saved.
• Further function args are stored inside its stack frame.

Overwriting Variables and Padding

• Overwrite an atomic variable behind a buffer

int main ( int argc, char ** argv ) {
    int var = 0 
    char buffer[12];
    
    gets(buffer);
    [...]
}

• Stack layout

Bottom 
+------------------+
| Saved registers  |
+------------------+
| int var          |
+------------------+
| char buffer [11] |
| ...              |
| ...              |
| ...              |
| char buffer [0]  |
+------------------+
| char ** argv     |
+------------------+
| char argc        |
+------------------+
Top

• Watch out! I.e., a 12 byte array is padded to system memory allocation size.

+-------------+----+
|12 byte array| 4b |
+-------------+----+
0            12   16 byte