[Question]
  [
I noticed that there's no such question, so here it is:
Do you have general tips for golfing in x86/x64 machine code? If the tip only applies to a certain environment or calling convention, please specify that in your answer.
> 
> Please only one tip per answer (see [here](https://codegolf.meta.stackexchange.com/a/9296/48198)).
> 
> 
> 
  
[Answer]
# Choose your calling convention to put args where you want them.
**The language of your answer is asm (actually machine code), so treat it as part of a program written in asm, not C-compiled-for-x86.** Your function doesn't have to be easily callable from C with any standard calling convention. That's a nice bonus if it doesn't cost you any extra bytes, though.
In a pure asm program, it's normal for some helper functions to use a calling convention that's convenient for them and for their caller. Such functions document their calling convention (inputs/outputs/clobbers) with comments.
In real life, even asm programs do (I think) tend to use consistent calling conventions for most functions (especially across different source files), but any given important function could do something special. In code-golf, you're optimizing the crap out of one single function, so obviously it's important/special.
---
**To test your function from a C program, you can write a wrapper** that puts args in the right places, saves/restores any extra registers you clobber, and puts the return value into `e/rax` if it wasn't there already.
---
### The limits of what's reasonable: anything that doesn't impose an unreasonable burden on the caller:
* `esp`/`rsp` must be call-preserved1; other integer regs are fair game for being call-clobbered. (`rbp` and `rbx` are usually call-preserved in normal conventions, but you *could* clobber both.)
* Any arg in any register (except `rsp`) is reasonable, but asking the caller to copy the same arg to multiple registers is not.
* Requiring `DF` (string direction flag for `lods`/`stos`/etc.) to be clear (upward) on call/ret is normal. Letting it be undefined on call/ret would be ok. Requiring it to be cleared or set on entry but then leaving it modified when you return would be weird.
* Returning FP values in x87 `st0` is reasonable, but returning in `st3` with garbage in other x87 registers isn't. The caller would have to clean up the x87 stack. Even returning in `st0` with non-empty higher stack registers would also be questionable (unless you're returning multiple values).
* Your function will be called with `call`, so `[rsp]` is your return address. You *can* avoid `call`/`ret` on x86 using a link register like `lea rbx, [ret_addr]`/`jmp function` and return with `jmp rbx`, but that's not "reasonable". That's not as efficient as `call`/`ret`, so it's not something you'd plausibly find in real code.
* Clobbering unlimited memory above `rsp` is not reasonable, but clobbering your function args on the stack is allowed in normal calling conventions. x64 Windows requires 32 bytes of shadow space above the return address, while x86-64 System V gives you a 128 byte red-zone below `rsp`, so either of those are reasonable. (Or even a much larger red-zone, especially in a stand-alone program rather than function.)
Note 1: or have some well-defined sensible rule for how RSP is modified: e.g. callee-pops stack args like with `ret 8`. (Although stack args and a larger `ret imm16` encoding are usually not what you want for code-golf). Or even returning an array by value on the stack is an unconventional but usable calling-convention. e.g. pop the return address into a register, then push in a loop, then `jmp reg` to return. Probably only justifiable with a size in a register, else the caller would have to save the original RSP somewhere. RBP or some other reg would have to be call-preserved so a caller could use it as a frame pointer to easily clean up the result. Also probably not smaller than using `stos` to write an output pointer passed in RDI.
Borderline cases: write a function that produces a sequence in an array, given the first 2 elements *as function args*. [I chose](https://codegolf.stackexchange.com/questions/101145/stewies-sequence/102741#102741) to have the caller store the start of the sequence into the array and just pass a pointer to the array. This is definitely bending the question's requirements. I considered taking the args packed into `xmm0` for `movlps [rdi], xmm0`, which would also be a weird calling convention and ever harder to justify / more of a stretch.
---
### Return a boolean in FLAGS (condition codes)
OS X system calls do this (`CF=0` means no error): [Is it considered bad practice to use the flags register as a boolean return value?](https://stackoverflow.com/questions/48381234/is-it-considered-bad-practice-to-use-the-flags-register-as-a-boolean-return-valu).
Any condition that can be checked with one `jcc` is perfectly reasonable, especially if you can pick one that has any semantic relevance to the problem. (e.g. a compare function might set flags so `jne` will be taken if they weren't equal).
---
### Require narrow args (like a `char`) to be sign or zero extended to 32 or 64 bits.
This is not unreasonable; using `movzx` or `movsx` [to avoid partial-register slowdowns](https://stackoverflow.com/questions/41573502/why-doesnt-gcc-use-partial-registers) is normal in modern x86 asm. In fact clang/LLVM already makes code that depends on an undocumented extension to the x86-64 System V calling convention: [args narrower than 32 bits are sign or zero extended to 32 bits by the caller](https://stackoverflow.com/questions/36706721/is-a-sign-or-zero-extension-required-when-adding-a-32bit-offset-to-a-pointer-for/36760539#36760539).
You can document/describe extension to 64 bits by writing `uint64_t` or `int64_t` in your prototype if you want, e.g. so you can use a `loop` instruction, which uses the whole 64 bits of `rcx` unless you use an address-size prefix to override the size down to 32 bit `ecx` (yes really, address-size not operand-size).
Note that `long` is only a 32-bit type in the Windows 64-bit ABI, and [the Linux x32 ABI](https://en.wikipedia.org/wiki/X32_ABI); `uint64_t` is unambiguous and shorter to type than `unsigned long long`.
---
### Existing calling conventions:
* Windows 32-bit `__fastcall`, [already suggested by another answer](https://codegolf.stackexchange.com/questions/132981/tips-for-golfing-in-x86-x64-machine-code/133048#133048): integer args in `ecx` and `edx`.
* **x86-64 System V**: passes lots of args in registers, and has lots of call-clobbered registers you can use without REX prefixes. More importantly, it was [actually chosen](https://stackoverflow.com/questions/63891991/whats-the-best-way-to-remember-the-x86-64-system-v-arg-register-order) to allow compilers to inline (or implement in libc) `memcpy` or `memset` as `rep movsb` easily: the first 6 integer/pointer args are passed in `rdi`, `rsi`, `rdx`, `rcx`, `r8`, and `r9`.
If your function uses `lodsd`/`stosd` inside a loop that runs `rcx` times (with the `loop` instruction), you can say "callable from C as `int foo(int *rdi, const int *rsi, int dummy, uint64_t len)` with the x86-64 System V calling convention". [example: chromakey](https://codegolf.stackexchange.com/questions/132719/the-chroma-key-to-success/132757#132757).
* 32-bit GCC `regparm`: **Integer args in `eax`**, `ecx`, and `edx`, return in EAX (or EDX:EAX). Having the first arg in the same register as the return value allows some optimizations, [like this case with an example caller and a prototype with a function attribute](https://codegolf.stackexchange.com/questions/160100/the-repetitive-byte-counter/160236#160236). And of course AL/EAX is special for some instructions.
* The Linux x32 ABI uses 32-bit pointers in long mode, so you can save a REX prefix when modifying a pointer ([example use-case](https://codegolf.stackexchange.com/questions/101145/stewies-sequence/102741#102741)). You can still use 64-bit address-size, unless you have a 32-bit negative integer zero-extended in a register (so it would be a large unsigned value if you did `[rdi + rdx]`, going outside the low 32 bits of address space.).
Note that `push rsp`/`pop rax` is 2 bytes, and equivalent to `mov rax, rsp`, so you can still *copy* full 64-bit registers in 2 bytes.
[Answer]
## `mov`-immediate is expensive for constants
This might be obvious, but I'll still put it here. In general it pays off to think about the bit-level representation of a number when you need to initialize a value.
### Initializing `eax` with `0`:
```
b8 00 00 00 00          mov    $0x0,%eax
```
should be shortened ([for performance as well as code-size](https://stackoverflow.com/questions/33666617/what-is-the-best-way-to-set-a-register-to-zero-in-x86-assembly-xor-mov-or-and/33668295#33668295)) to
```
31 c0                   xor    %eax,%eax
```
### Initializing `eax` with `-1`:
```
b8 ff ff ff ff          mov    $-1,%eax
```
can be shortened to
```
31 c0                   xor    %eax,%eax
48                      dec    %eax
```
or
```
83 c8 ff                or     $-1,%eax
```
---
Or more generally, **any 8-bit sign-extended value can be created in 3 bytes with `push -12` (2 bytes) / `pop %eax` (1 byte).** This even works for 64-bit registers with no extra REX prefix; `push`/`pop` default operand-size = 64.
```
6a f3                   pushq  $0xfffffffffffffff3
5d                      pop    %rbp
```
Or given a known constant in a register, you can create another nearby constant using `lea 123(%eax), %ecx` (3 bytes). This is handy if you need a zeroed register *and* a constant; xor-zero (2 bytes) + `lea-disp8` (3 bytes).
```
31 c0                   xor    %eax,%eax
8d 48 0c                lea    0xc(%eax),%ecx
```
See also [Set all bits in CPU register to 1 efficiently](https://stackoverflow.com/questions/45105164/set-all-bits-in-cpu-register-to-1-efficiently)
[Answer]
In a lot of cases, accumulator-based instructions (i.e. those that take `(R|E)AX` as the destination operand) are 1 byte shorter than general-case instructions; see [this question](https://stackoverflow.com/questions/38019386/what-is-the-significance-of-operations-on-the-register-eax-having-their-own-opco) on StackOverflow.
[Answer]
**Use special-case short-form encodings for AL/AX/EAX, and other short forms and single-byte instructions**
Examples assume 32 / 64-bit mode, where the default operand size is 32 bits. An operand-size prefix changes the instruction to AX instead of EAX (or the reverse in 16-bit mode).
* **`inc/dec` a register** (other than 8-bit): `inc eax` / `dec ebp`. (Not x86-64: the `0x4x` opcode bytes were repurposed as REX prefixes, so `inc r/m32` is the only encoding.)
8-bit `inc bl` is 2 bytes, using the [`inc r/m8` opcode + ModR/M operand encoding](http://felixcloutier.com/x86/INC.html). So **use `inc ebx` to increment `bl`, if it's safe.** (e.g. if you don't need the ZF result in cases where the upper bytes might be non-zero).
* `scasd`: `e/rdi+=4`, requires that the register points to readable memory. Sometimes useful even if you don't care about the FLAGS result (like `cmp eax,[rdi]` / `rdi+=4`). And **in 64-bit mode, `scasb` can work as a 1-byte `inc rdi`**, if lodsb or stosb aren't useful.
* **[`xchg eax, r32`](http://felixcloutier.com/x86/XCHG.html)**: this is where 0x90 NOP came from: `xchg eax,eax`. Example: re-arrange 3 registers with two `xchg` instructions in a `cdq` / `idiv` loop [for GCD in 8 bytes](https://codegolf.stackexchange.com/questions/77270/greatest-common-divisor/77364#77364) where most of the instructions are single-byte, including an abuse of `inc ecx`/`loop` instead of `test ecx,ecx`/`jnz`
* [`cdq`](http://felixcloutier.com/x86/CWD:CDQ:CQO.html): sign-extend EAX into EDX:EAX, i.e. copying the high bit of EAX to all bits of EDX. To create a zero with known non-negative, or to get a 0/-1 to add/sub or mask with. [x86 history lesson: `cltq` vs. `movslq`](https://stackoverflow.com/questions/37743476/assembly-cltq-and-movslq-difference/37746322#37746322), and also AT&T vs. Intel mnemonics for this and the related `cdqe`.
* [lodsb/d](http://felixcloutier.com/x86/LODS:LODSB:LODSW:LODSD:LODSQ.html): like `mov eax, [rsi]` / `rsi += 4` without clobbering flags. (Assuming DF is clear, which standard calling conventions require on function entry.) Also stosb/d, sometimes scas, and more rarely movs / cmps.
* [`push`](http://felixcloutier.com/x86/PUSH.html)/[`pop reg`](http://felixcloutier.com/x86/POP.html). e.g. in 64-bit mode, `push rsp` / `pop rdi` is 2 bytes, but `mov rdi, rsp` needs a REX prefix and is 3 bytes.
[`xlatb`](http://felixcloutier.com/x86/XLAT:XLATB.html) exists, but is rarely useful. A large lookup table is something to avoid. I've also never found a use for AAA / DAA or other packed-BCD or 2-ASCII-digit instructions, except for a [hacky use of DAS as part of converting a 4-bit integer to an ASCII hex digit](https://codegolf.stackexchange.com/questions/193793/little-endian-number-to-string-conversion/193842#193842), thanks to Peter Ferrie.
1-byte `lahf` / `sahf` are rarely useful. You *could* `lahf` / `and ah, 1` as an alternative to `setc ah`, but it's typically not useful.
And for CF specifically, there's `sbb eax,eax` to get a 0/-1, or even **un-documented but universally supported 1-byte `salc` (set AL from Carry)** which effectively does `sbb al,al` without affecting flags. (Removed in x86-64). I used SALC in [User Appreciation Challenge #1: Dennis ‚ô¶](https://codegolf.stackexchange.com/questions/123194/user-appreciation-challenge-1-dennis/123458#123458).
1-byte `cmc` / `clc` / `stc` (flip ("complement"), clear, or set CF) are rarely useful, although I did find [**a use for `cmc`** in extended-precision addition](https://codegolf.stackexchange.com/questions/133618/extreme-fibonacci/135618#135618) with base 10^9 chunks. To unconditionally set/clear CF, usually arrange for that to happen as part of another instruction, e.g. `xor eax,eax` clears CF as well as EAX. There are no equivalent instructions for other condition flags, just DF (string direction) and IF (interrupts). The carry flag is special for a lot of instructions; shifts set it, `adc al, 0` can add it to AL in 2 byte, and I mentioned earlier the undocumented SALC.
**`std` / `cld` rarely seem worth it**. Especially in 32-bit code, it's better to just use `dec` on a pointer and a `mov` or memory source operand to an ALU instruction instead of setting DF so `lodsb` / `stosb` go downward instead of up. Usually if you need downward at all, you still have another pointer going up, so you'd need more than one `std` and `cld` in the whole function to use `lods` / `stos` for both. Instead, just use the string instructions for the upward direction. (The standard calling conventions guarantee DF=0 on function entry, so you can assume that for free without using `cld`.)
---
### 8086 history: why these encodings exist
In original 8086, AX was very special: instructions like `lodsb` / `stosb`, `cbw`, `mul` / `div` and others use it implicitly. That's still
the case of course; current x86 hasn't dropped any of 8086's opcodes (at least not any of the officially documented ones, [except](https://stackoverflow.com/questions/32868293/x86-32-bit-opcodes-that-differ-in-x86-x64-or-entirely-removed) in [64-bit mode](https://stackoverflow.com/questions/44089163/is-x86-32-bit-assembly-code-valid-x86-64-bit-assembly-code)). But later CPUs added new instructions that gave better / more efficient ways to do things without copying or swapping them to AX first. (Or to EAX in 32-bit mode.)
e.g. 8086 lacked later additions like `movsx` / `movzx` to load or move + sign-extend, or 2 and 3-operand [`imul cx, bx, 1234`](http://felixcloutier.com/x86/IMUL.html) that don't produce a high-half result and don't have any implicit operands.
Also, **8086's main bottleneck was instruction-fetch, so optimizing for code-size was important for performance back then**. 8086'[s ISA designer (Stephen Morse)](https://retrocomputing.stackexchange.com/questions/5121/why-are-first-four-x86-gprs-named-in-such-unintuitive-order#comment11346_5128) spent a lot of opcode coding space on special cases for AX / AL, [including special (E)AX/AL-destination opcodes for all the basic immediate-src ALU- instructions](https://stackoverflow.com/questions/38019386/what-is-the-significance-of-operations-on-the-register-eax-having-their-own-opco), just opcode + immediate with no ModR/M byte. 2-byte `add/sub/and/or/xor/cmp/test/... AL,imm8` or `AX,imm16` or (in 32-bit mode) `EAX,imm32`.
But there's no special case for `EAX,imm8`, so the regular ModR/M encoding of `add eax,4` is shorter.
The assumption is that if you're going to work on some data, you'll want it in AX / AL, so swapping a register with AX was something you might want to do, maybe even more often than *copying* a register to AX with `mov`.
Everything about 8086 instruction encoding supports this paradigm, from instructions like `lodsb/w` to all the special-case encodings for immediates with EAX to its implicit use even for multiply/divide.
---
[Answer]
### Create 3 zeroes with `mul` (then `inc`/`dec` to get +1 / -1 as well as zero)
You can zero eax and edx by multiplying by zero in a third register.
```
xor   ebx, ebx      ; 2B  ebx = 0
mul   ebx           ; 2B  eax=edx = 0
inc   ebx           ; 1B  ebx=1
```
will result in EAX, EDX, and EBX all being zero in just four bytes.
You can zero EAX and EDX in three bytes:
```
xor eax, eax
cdq
```
But from that starting point you can't get a 3rd zeroed register in one more byte, or a +1 or -1 register in another 2 bytes. Instead, use the mul technique.
Example use-case: [concatenating the Fibonacci numbers in binary](https://codegolf.stackexchange.com/questions/160857/binary-fibonacci/160894#160894).
Note that after a `LOOP` loop finishes, ECX will be zero and can be used to zero EDX and EAX; you don't always have to create the first zero with `xor`.
[Answer]
# Subtract -128 instead of add 128
```
0100 81C38000      ADD     BX,0080
0104 83EB80        SUB     BX,-80
```
Samely, add -128 instead of subtract 128
[Answer]
# Skipping instructions
Skipping instructions are opcode fragments that combine with one or more subsequent opcodes. The subsequent opcodes can be used with a different entrypoint than the prepended skipping instruction. Using a skipping instruction instead of an unconditional short jump can save code space, be faster, and set up incidental state such as `NC` (No Carry).
My examples are all for 16-bit Real/Virtual 86 Mode, but a lot of these techniques can be used similarly in 16-bit Protected Mode, or 32- or 64-bit modes.
Quoting [from my ACEGALS guide](https://pushbx.org/ecm/doc/acegals.htm#skipping):
> 
> ## 11: Skipping instructions
> 
> 
> The constants \_\_TEST\_IMM8, \_\_TEST\_IMM16, and \_\_TEST\_OFS16\_IMM8 are defined to the respective byte strings for these instructions. They can be used to skip subsequent instructions that fit into the following 1, 2, or 3 bytes. However, note that they modify the flags register, including always setting NC. The 16-bit offset plus 16-bit immediate test instruction is not included for these purposes because it might access a word at offset 0FFFFh in a segment. Also, the \_\_TEST\_OFS16\_IMM8 as provided should only be used in 86M, to avoid accessing data beyond a segment limit. After the db instruction using one of these constants, a parenthetical remark should list which instructions are skipped.
> 
> 
> 
The 86 Mode defines [in lmacros1.mac 323cc150061e (2021-08-29 21:45:54 +0200)](https://hg.pushbx.org/ecm/lmacros/file/323cc150061e/lmacros1.mac#l246):
```
%define __TEST_IMM8 0A8h                        ; changes flags, NC
%define __TEST_IMM16 0A9h                       ; changes flags, NC
                                        ; Longer NOPs require two bytes, like a short jump does.
                                        ; However they execute faster than unconditional jumps.
                                        ; This one reads random data in the stack segment.
                                        ;  (Search for better ones.)
%define __TEST_OFS16_IMM8 0F6h,86h              ; changes flags, NC
```
The `0F6h,86h` opcode in 16-bit modes is a `test byte [bp + disp16], imm8` instruction. I believe I am not using this one anywhere actually. (A stack memory access might actually be slower than an unconditional short jump, in fact.)
`0A8h` is the opcode for `test al, imm8` in any mode. The `0A9h` opcode changes to an instruction of the form `test eax, imm32` in 32- and 64-bit modes.
Two use cases [in ldosboot boot32.asm 07f4ba0ef8cd (2021-09-10 22:45:32 +0200)](https://hg.pushbx.org/ecm/ldosboot/file/07f4ba0ef8cd/boot32.asm):
First, chain two different entrypoints for a common function which both need to initialise a byte-sized register. The `mov al, X` instructions take 2 bytes each, so `__TEST_IMM16` can be used to skip one such instruction. (This pattern can be repeated if there are more than two entrypoints.)
```
error_fsiboot:
        mov al,'I'
        db __TEST_IMM16 ; (skip mov)
read_sector.err:
        mov al, 'R'     ; Disk 'R'ead error
error:
```
Second, a certain entrypoint that needs two bytes worth of additional teardown but can otherwise be shared with the fallthrough case of a later code part.
```
                mov bx, [VAR(para_per_sector)]
                sub word [VAR(paras_left)], bx
                jbe @F          ; read enough -->
                loop @BB
                pop bx
                pop cx
                call clust_next
                jnc next_load_cluster
                inc ax
                inc ax
                test al, 8      ; set in 0FFF_FFF8h--0FFF_FFFFh,
                                ;  clear in 0, 1, and 0FFF_FFF7h
                jz fsiboot_error_badchain
                db __TEST_IMM16
@@:
                pop bx
                pop cx
                call check_enough
                jmp near word [VAR(fsiboot_table.success)]
```
Here's a use case [in inicomp lz4.asm 4d568330924c (2021-09-03 16:59:42 +0200)](https://hg.pushbx.org/ecm/inicomp/file/4d568330924c/lz4.asm#l491) where we depend on the `test al, X` instruction clearing the Carry Flag:
```
.success:
        db __TEST_IMM8                  ; (NC)
.error:
        stc
        retn
```
Finally, here's a very similar use of a skipping instruction in [DOSLFN Version 0.41c (11/2012)](http://adoxa.altervista.org/doslfn/). Instead of `test ax, imm16` they're using `mov cx, imm16` which has no effect on the status flags but clobbers the `cx` register instead. (Opcode `0B9h` is `mov ecx, imm32` in non-16-bit modes, and writes to the full `ecx` or `rcx` register.)
```
;THROW-Geschichten... [english: THROW stories...]
SetErr18:
        mov     al,18
        db      0B9h            ;mov cx,nnnn
SetErr5:
        mov     al,5
        db      0B9h            ;mov cx,nnnn
SetErr3:
        mov     al,3
        db      0B9h            ;mov cx,nnnn
SetErr2:
        mov     al,2
SetError:
```
[Answer]
# CPU registers and flags are in known startup states
For a full/standalone program, we can assume that the CPU is in a known and documented default state based on platform and OS.
For example:
DOS
<http://www.fysnet.net/yourhelp.htm>
Linux x86 ELF
<http://asm.sourceforge.net/articles/startup.html> - in `_start` in a static executable, most registers are zero other than the stack pointer, to avoid leaking info into a fresh process. `pop` will load `argc` which is a small non-negative integer, `1` if run normally from a shell with no args.
Same applies for x86-64 processes on Linux.
[Answer]
# `mov` small immediates into lower registers when applicable
If you already know the upper bits of a register are 0, you can use a shorter instruction to move an immediate into the lower registers.
```
b8 0a 00 00 00          mov    $0xa,%eax
```
versus
```
b0 0a                   mov    $0xa,%al
```
# Use `push`/`pop` for imm8 to zero upper bits
Credit to Peter Cordes. `xor`/`mov` is 4 bytes, but `push`/`pop` is only 3!
```
6a 0a                   push   $0xa
58                      pop    %eax
```
[Answer]
# Use do-while loops instead of while loops
This is not x86 specific but is a widely applicable beginner assembly tip. If you know a while loop will run at least once, rewriting the loop as a do-while loop, with loop condition checking at the end, often saves a 2 byte jump instruction. In a special case you might even be able to use [`loop`](https://codegolf.stackexchange.com/a/133095/17360). 
[Answer]
# The [FLAGS](https://en.wikipedia.org/wiki/FLAGS_register) are set after many instructions
After many arithmetic instructions, the Carry Flag (unsigned) and Overflow Flag (signed) are set automatically ([more info](https://stackoverflow.com/questions/791991/about-assembly-cfcarry-and-ofoverflow-flag)). The Sign Flag and Zero Flag are set after many arithmetic and logical operations. This can be used for conditional branching. 
Example: 
```
d1 f8                   sar    %eax
```
ZF is set by this instruction, so we can use it for condtional branching. 
[Answer]
# `lea` for math
This is probably one of the first things one learns about x86, but I leave it here as a reminder. `lea` can be used to do multiplication by 2, 3, 4, 5, 8, or 9, and adding an offset. 
For example, to calculate `ebx = 9*eax + 3` in one instruction (in 32-bit mode):
```
8d 5c c0 03             lea    0x3(%eax,%eax,8),%ebx
```
Here it is without an offset:
```
8d 1c c0                lea    (%eax,%eax,8),%ebx
```
Wow! Of course, `lea` can be used to also do math like `ebx = edx + 8*eax + 3` for calculating array indexing. 
[Answer]
# Use `fastcall` conventions
x86 platform has [many calling conventions](https://en.wikipedia.org/wiki/X86_calling_conventions). You should use those that pass parameters in registers. On x86\_64, the first few parameters are passed in registers anyway, so no problem there. On 32-bit platforms, the default calling convention (`cdecl`) passes parameters in stack, which is no good for golfing - accessing parameters on stack requires long instructions.
When using `fastcall` on 32-bit platforms, 2 first parameters are usually passed in `ecx` and `edx`. If your function has 3 parameters, you might consider implementing it on a 64-bit platform.
C function prototypes for `fastcall` convention (taken from [this example answer](https://codegolf.stackexchange.com/a/119057/25315)):
```
extern int __fastcall SwapParity(int value);                 // MSVC
extern int __attribute__((fastcall)) SwapParity(int value);  // GNU   
```
---
Note: you can also use other calling conventions, including custom ones. I never use custom calling conventions; for any ideas related to these, see [here](https://codegolf.stackexchange.com/a/165020/25315).
[Answer]
# Combinations with CDQ for certain piecewise-linear functions
`CDQ` sign-extends EAX into EDX, making EDX 0 if EAX is nonnegative and -1 (all 1s) if EAX is negative. This can be combined with several other instructions to apply certain piecewise-linear functions to a value in EAX in 3 bytes:
`CDQ` + `AND` ‚Üí \$ \min(x, 0) \$ (in either EAX or EDX). (I have used this [here](https://codegolf.stackexchange.com/a/241451/104752).)
`CDQ` + `OR` ‚Üí \$ \max(x, -1) \$.
`CDQ` + `XOR` ‚Üí \$ \max(x, -x-1) \$.
`CDQ` + `MUL EDX` ‚Üí \$ \max(-x, 0) \$ in EAX and \$ \left\{ \begin{array}{ll} 0 & : x \ge 0 \\ x - 1 & : x < 0 \end{array} \right.\$ in EDX.
[Answer]
The loop and string instructions are smaller than alternative instruction sequences. Most useful is `loop <label>` which is smaller than the two instruction sequence `dec ECX` and `jnz <label>`, and `lodsb` is smaller than `mov al,[esi]` and `inc si`.
[Answer]
# Entry point doesn't necessarily have to be first byte of submission
I came across [this answer](https://codegolf.stackexchange.com/questions/118444/stay-away-from-zero/151115#151115), and didn't understand it at first until I realized that the intention is:
```
; ---- example calling code starts here -------------
                MOV     ECX, 1
                CALL    entry
                RET
; ---- code golf answer code starts here (5 bytes) --
       41       INC     ECX
entry: E3 FD    JECXZ   SHORT $-1
       91       XCHG    EAX,ECX
       C3       RETN
; ---- code golf answer code ends here -------------
```
Does not seem to conflict with any of the conditions of "[Choose your calling convention](https://codegolf.stackexchange.com/questions/132981/tips-for-golfing-in-x86-x64-machine-code/132983#132983)" and is otherwise valid assembly language.
[Answer]
# Use a good assembler
There are dozens of x86 assemblers out there, and they are **not** created equal.
Not only can a bad assembler be painful to use, but **they might not always output the most optimal code.**
Most x86 instructions have multiple valid encodings, some shorter than others.
For example, I saw one user with a 16-bit assembler that emitted different code depending on the order of `xchg`'s operands. It is a commutative operation, it shouldn't make a difference.
```
87 D8   xchg    ax, bx
93      xchg    bx, ax
```
Life is too short for bad assemblers, and it should **not** be the thing getting in the way of golfing.
The three assemblers I would suggest are:
* [nasm](http://www.nasm.us/) is the first one I would recommend to everyone (and I wish I had learned it first).
	+ It fully supports 16-bit, 32-bit, *and* 64-bit code
	+ It can easily assemble all sorts of object formats, as well as raw binaries/`.com` files (a multi-step ritual with GAS).
	+ **It is officially supported on DOS** as well as all modern OSes.
	+ While it doesn't support C macros, it has a god-tier preprocessor that is much better than C.
	+ The way it handles local labels is really nice.
	+ Good error messages, mostly fairly beginner-friendly pointing you in the direction of why an instruction isn't allowed or what it doesn't like about a source line.
* [GAS](https://sourceware.org/binutils/docs/as/index.html) ([GCC](https://gcc.gnu.org)'s assembler) is another fairly good assembler.
	+ It is the assembler used by GCC and Clang. You might recognize it if you use Godbolt or `gcc -S`.
	+ It supports AT&T syntax which some might prefer
	+ With `.intel_syntax noprefix`, you can switch to Intel syntax
	+ While its built-in preprocessor is pretty limited, it can be easily combined with a C preprocessor.
	+ For x87, beware of [the AT&T syntax design bug](https://sourceware.org/binutils/docs/as/i386_002dBugs.html) that interchanges `fsubr` with `fsub` in some cases, same for `fdiv[r]`. Older GAS versions applied the same swap in Intel-syntax mode, and so did older binutils `objdump -d` versions. (This is AT&T's fault, not GNU's, and current GAS versions do as well as possible, but is an inherent downside in using AT&T syntax for x87.)
	+ Error messages are less helpful than NASM about why an instruction is invalid
	+ Ambiguous instructions other than `mov` default to dword operand-size instead of being an error in AT&T syntax, such as `add $123, (%rdi)` assembling as `addl`. Clang, and GAS in Intel syntax mode, error on this.
* [Clang](https://clang.llvm.org) is, in most cases, completely exchangeable for GAS.
	+ It has much more helpful error messages and doesn't silently treat x86\_64 registers as symbols in 32-bit mode.
	+ While AT&T syntax is fully supported, Intel syntax currently has a few bugs.
		- It is the **only** reason I am recommending GAS over Clang. üòî
		- It still works as a solid linter.
	+ It also supports C macros.
I haven't used enough of the other assemblers to give a good opinion, as I am more than satisfied with those three.
[FASM](https://flatassembler.net/) is also well-regarded, using very nearly the same syntax as NASM, and is supported on <https://TIO.run/> ; It's able to make 32-bit executables on TIO, unlike with other assemblers, using the directive `format ELF executable 3` to emit a 32-bit ELF executable, not a `.o` object file that would need linking.
[EuroAssembler](https://euroassembler.eu/eadoc/) is also open-source; its maintainer is [active on Stack Overflow](https://stackoverflow.com/users/2581418/vitsoft) in the [assembly] and [x86] tags.
[Answer]
To add or subtract 1, use the one byte `inc` or `dec` instructions which are smaller than the multibyte add and sub instructions.
[Answer]
# Try `XLAT` for byte memory access
[`XLAT`](https://www.felixcloutier.com/x86/xlat:xlatb) is a **one byte** instruction that is equivalent to `AL = [BX+AL]`. Yes, that's right, it lets you use `AL` as an index register for memory access.
[Answer]
# Take advantage of the x86\_64 code model
Linux's default code model will put all of your code and globals in the low 31 bits of memory, so 32-bit pointer arithmetic here is perfectly safe. The stack, libraries, and any dynamically allocated pointers are not, though. [Try it online!](https://tio.run/##tZHbigIxDIbv@xRB8EotnvDCZ9g3WEUytc4WMm1Jqoy@/DieZkZQ2IW1l8n3fyUJitgio@MoN6aqdE4hIyjQeaWdT5Y2cvQJS/Ahst25Ul16SwX3F/fyA4xlUyCLwFs3hG92EQagvyLXop0kXjdQEQ7AUkM12RQNEsENfrZJ18ZhiwnXf/vvd@qPiTORf/DGEJ82zTapTri9iUYx7gS9/niyICpXvte2xJrkggd926N6LLSTpuBzmE1fZO6JN3zL1fOq69Qd6mQ5wFxV1Rk "Assembly (gcc, x64, Linux) ‚Äì Try It Online")
Make sure to still use the entire 64 bits in memory operands (**including lea**), because using `[eax]` requires a `67` prefix byte.
[Answer]
# Use interrupts and syscalls wisely
In general, unlike the C calling conventions, most syscalls and interrupts will preserve your registers and flags unless noted otherwise, except for a return value usually in AL/EAX/RAX depending on the OS. (e.g. the x86-64 `syscall` instruction itself [destroys RCX and R11](https://stackoverflow.com/questions/47983371/why-do-x86-64-linux-system-calls-modify-rcx-and-what-does-the-value-mean))
#### Linux specific:
* If exiting with a crash is okay, `int3`, `int1`, or `into` can usually do the job in one byte. Don't try this on DOS though, it will lock up.
	+ Note that the error messages like `Segmentation fault (core dumped)` or `Trace/breakpoint trap` are actually printed by your *shell*, not the program/kernel. Don't believe me? Try running `set +m` before your program or redirecting stderr to a file.
* You can use `int 0x80` in 64-bit mode. [It will use the 32-bit ABI though](https://stackoverflow.com/questions/46087730/what-happens-if-you-use-the-32-bit-int-0x80-linux-abi-in-64-bit-code) (eax, ebx, ecx, edx), so make sure all pointers are in the low 32 bits. On the small code model, this true for all code stored in your binary. Keep this in mind for [restricted-source](/questions/tagged/restricted-source "show questions tagged 'restricted-source'").
	+ Additionally, `sysenter` and `call dword gs:0x10` can also do syscalls in 32-bit mode, although the calling convention is quite....weird for the former.
#### DOS/BIOS specific:
* Use `int 29h` instead of `int 21h:02h` for printing single bytes to the screen. `int 29h` doesn't need `ah` to be set and *very* conveniently uses `al` instead of `dl`. It writes directly to the screen, so you can't just redirect to a file, though.
* DOS also has `strlen` and `strcmp` interrupts (see [this helpful page for this and other undocumented goodies](http://www.fysnet.net/undoc.htm))
* Unless you modified `cs`, **don't** use `int 20h` or `int 21h:4Ch` for exiting, just `ret` from your `.com` file. Alternatively, if you happen to have `0x0000` on the top of your stack, you can also `ret` to that.
* In the rare case that you need to call helper functions more than 4 times, consider registering them to the `int1`, `int3`, or `into` interrupts. Make sure to use `iret` instead of `ret`, though.
```
        ; AH: 0x25 (set interrupt)
        ; AL: 0x03 (INT3)
        ; Use 0x2504 for INTO, or 0x2501 for INT1
        mov     ax, 0x2503
        ; DX: address to function (make sure to use IRET instead of RET)
        mov     dx, my_interrupt_func
        int     0x21
        ; Now instead of this 3 byte opcode...
        call    my_func
        ; ...just do this 1 byte opcode.
        int3 ; or int1, into
```
* [restricted-source](/questions/tagged/restricted-source "show questions tagged 'restricted-source'") tip: Your interrupt vector table is a table of far pointers at `0000:0000` (so for example, `int 21h` is at `0000h:0084h`).
[Answer]
# Use conditional moves [`CMOVcc`](http://www.felixcloutier.com/x86/CMOVcc.html) and sets [`SETcc`](https://c9x.me/x86/html/file_module_x86_id_288.html)
This is more a reminder to myself, but conditional set instructions exist and conditional move instructions exist on processors P6 (Pentium Pro) or newer. There are many instructions that are based on one or more of the flags set in EFLAGS. 
[Answer]
# Use whatever calling conventions are convenient
System V x86 uses the stack and System V x86-64 uses `rdi`, `rsi`, `rdx`, `rcx`, etc. for input parameters, and `rax` as the return value, but it is perfectly reasonable to use your own calling convention. [\_\_fastcall](https://msdn.microsoft.com/en-us/library/6xa169sk.aspx) uses `ecx` and `edx` as input parameters, and [other compilers/OSes use their own conventions](https://en.wikipedia.org/wiki/X86_calling_conventions#List_of_x86_calling_conventions). Use the stack and whatever registers as input/output when convenient.
Example: [The repetitive byte counter](https://codegolf.stackexchange.com/a/160236/17360), using a clever calling convention for a 1 byte solution.
Meta: [Writing input to registers](https://codegolf.meta.stackexchange.com/a/8508/17360), [Writing output to registers](https://codegolf.meta.stackexchange.com/a/8509/17360)
Other resources: [Agner Fog's notes on calling conventions](http://agner.org/optimize/calling_conventions.pdf)
[Answer]
# Save on `jmp` bytes by arranging into if/then rather than if/then/else
This is certainly very basic, just thought I would post this as something to think about when golfing. As an example, consider the following straightforward code to decode a hexadecimal digit character:
```
    cmp $'A', %al
    jae .Lletter
    sub $'0', %al
    jmp .Lprocess
.Lletter:
    sub $('A'-10), %al
.Lprocess:
    movzbl %al, %eax
    ...
```
This can be shortened by two bytes by letting a "then" case fall into an "else" case:
```
    cmp $'A', %al
    jb .digit
    sub $('A'-'0'-10), %eax
.digit:
    sub $'0', %eax
    movzbl %al, %eax
    ...
```
[Answer]
# (way too many) ways of zeroing a register
I remember being taught these by a certain person (I "invented" some of these myself); I don't remember who did I get them from, anyways these are the most interesting; possible use cases include restricted source code challenges or other bizzare stuff.
`=>` Zero mov:
```
mov reg, 0
; mov eax, 0: B800000000
```
`=>` `push`+`pop`:
```
push [something equal to zero]
pop reg
; push 0 / pop eax: 6A0058
; note: if you have a register equal to zero, it will be
; shorter but also equal to a mov.
```
`=>` `sub` from itself:
```
sub reg, reg
; sub eax, eax: 29C0
```
`=>` `mul` by zero:
```
imul reg, 0
; imul eax, 0: 6BC000
```
`=>` `and` by zero:
```
and reg, 0
; and eax, 0: 83E000
```
`=>` `xor` by itself:
```
xor reg, reg
; xor eax, eax: 31C0
; possibly the best way to zero an arbitrary register,
; I remembered this opcode (among other).
```
`=>` `or` and `inc` / `not`:
```
or reg, -1
inc reg     ; or not reg
; or eax, -1 / inc eax: 83C8FF40
```
`=>` reset `ECX`:
```
loop $
; loop $: E2FE
```
`=>` flush `EDX`:
```
shr eax, 1
cdq
; D1E899
```
`=>` zero `AL` (`AH = AL, AL = 0`)
```
aam 1
; D401
```
`=>` reset `AH`:
```
aad 0
; D500
```
`=>` Read 0 from the port
```
mov dx, 81h
in al, dx
; 66BA8100EC
```
`=>` Reset `AL`
```
stc
setnc al
; F90F93C0
```
`=>` Use the zero descriptor from `gdt`:
```
sgdt [esp-6]
mov reg, [esp-4]
mov reg, [reg]
; with eax: 0F014424FA8B4424FC8B00
```
`=>` Read zero from the `fs` segment (PE `exe` only)
```
mov reg, fs:[10h]
; with eax: 64A110000000
```
`=>` The brainfuck way
```
inc reg
jnz $-1
; with eax: 4075FD
```
`=>` Utilize the coprocessor
```
fldz
fistp dword ptr [esp-4]
mov eax, [esp-4]
; D9EEDB5C24FC8B4424FC
```
Another possible options:
* Read zero using the builtin random number generator.
* calculate sine from `pi * n` (use `fmul`).
There are way cooler and potentially useful ways to execute this operation; although I didn't come up with them, therefore I'm not posting.
[Answer]
# Use multiplication for hashing
[IMUL](https://c9x.me/x86/html/file_module_x86_id_138.html), multiplication by an immediate signed number, is a powerful instruction which can be used for hashing.
The regular multiplication instruction hard-codes one of the input operands and the output operand to be in `eax` (or `ax` or `al`). This is inconvenient; it requires instructions for setup and sometimes also to save and restore `eax` and `edx`. But if one of the operands is a constant, the instruction becomes much more versatile:
* No need to load the constant into a register
* The other operand can be in any register, not only `eax`
* The result can be in any register, not necessarily overwriting the input!
* The result is 32-bit, not a pair of registers
* If the constant is between -128 and 127, it can be encoded by only one byte
I used this many times (I hope I can be excused for these shameless plugs: [1](https://codegolf.stackexchange.com/a/193500/25315) [2](https://codegolf.stackexchange.com/a/205522/25315) [3](https://codegolf.stackexchange.com/a/187720/25315) ...)
[Answer]
"Free bypass": If you already have an instruction with an immediate on a register that you only care about the low part of, making the immediate longer than necessary could allow you to insert into its high part other instructions that can be jumped to (but don't execute when coming from before). This works because of little-endianness. [Example](https://codegolf.stackexchange.com/a/244118/104752); [another example](https://codegolf.stackexchange.com/a/245361/104752).
[Answer]
**To copy a 64-bit register, use `push rcx` ; `pop rdx`** instead of a 3-byte `mov`.  
The default operand-size of push/pop is 64-bit without needing a REX prefix.
```
  51                      push   rcx
  5a                      pop    rdx
                vs.
  48 89 ca                mov    rdx,rcx
```
(An operand-size prefix can override the push/pop size to 16-bit, but [32-bit push/pop operand-size is not encodeable in 64-bit mode](https://stackoverflow.com/questions/45127993/how-many-bytes-does-the-push-instruction-push-onto-the-stack-when-i-dont-specif) even with REX.W=0.)
If either or both registers are `r8`..`r15`, use `mov` because push and/or pop will need a REX prefix. Worst case this actually loses if both need REX prefixes. Obviously you should usually avoid r8..r15 anyway in code golf.
---
You can keep your source more readable while developing with this **NASM macro**. Just remember that it steps on the 8 bytes below RSP. (In the red-zone in x86-64 System V). But under normal conditions it's a drop-in replacement for 64-bit `mov r64,r64` or `mov r64, -128..127`
```
    ; mov  %1, %2       ; use this macro to copy 64-bit registers in 2 bytes (no REX prefix)
%macro MOVE 2
    push  %2
    pop   %1
%endmacro
```
Examples:
```
   MOVE  rax, rsi            ; 2 bytes  (push + pop)
   MOVE  rbp, rdx            ; 2 bytes  (push + pop)
   mov   ecx, edi            ; 2 bytes.  32-bit operand size doesn't need REX prefixes
   MOVE  r8, r10             ; 4 bytes, don't use
   mov   r8, r10             ; 3 bytes, REX prefix has W=1 and the bits for reg and r/m being high
   xchg  eax, edi            ; 1 byte  (special xchg-with-accumulator opcodes)
   xchg  rax, rdi            ; 2 bytes (REX.W + that)
   xchg  ecx, edx            ; 2 bytes (normal xchg + modrm)
   xchg  rcx, rdx            ; 3 bytes (normal REX + xchg + modrm)
```
The `xchg` part of the example is because sometimes you need to get a value into EAX or RAX and don't care about preserving the old copy. push/pop doesn't help you actually exchange, though.
[Answer]
# Try `AAM` or `AAD` for byte division operations
If you are working with only 8 bit values, using the [`AAM`](https://www.felixcloutier.com/x86/aam) instruction can sometimes save several bytes over [`DIV reg8`](https://www.felixcloutier.com/x86/div) since it will take an `imm8` and returns remainder and quotient in opposite `AH/AL` registers as `DIV`.
```
D4 0A    AAM        ; AH = AL / 10, AL = AL % 10
```
It can also accept any byte value as the divisor as well by altering the second byte.
```
D4 XX    AAM  XX    ; AH = AL / XX, AL = AL % XX
```
And `AAD` is the inverse of this, which is two operations in one.
```
D5 XX    AAD  XX    ; AL = AH * XX + AL
```
[Answer]
# Avoid registers which need prefixes
Quite a simple tip I haven't seen mentioned before.
**Avoid `r8-r15` (as well as `dil`, `sil`, `bpl`, and `spl`) on x86\_64 like the plague.** Even just thinking about these registers requires an extra REX prefix. The only exception is if you are using them exclusively for 64-bit arithmetic (which also needs REX prefixes). Even still, you are usually better off using a low register since some operations can be done using the implicit zero extension.
Note that this tip also applies to ARM Thumb-2.
Additionally, be careful when using 16-bit registers in 32/64-bit mode (as well as 32-bit registers in 16-bit mode, but this is rare), as these need a prefix byte as well.
However, unlike the extra x86\_64 registers, 16-bit instructions can be useful: Many instructions which would otherwise need a full 32-bit immediate argument will only use a 16-bit argument. So, if you were to bitwise and `eax` by `0xfffff00f`, `and ax, 0xf00f` would be smaller.
]