What 8086 instructions accept REP?

Question

I tried this code in my assembler, set to 16 bit mode:

bits 16
rep mov ds, ax

Surprisingly, no error was thrown. Is this even valid? Wasn't rep only supposed to work with string instructions? Is it just a very fancy way to write this?

mov ds, ax
xor cx, cx

Answer 1

All of them. But it will only have an effect with a select few.

Contrary to what the question implies, the rep prefix is not an orthogonal looping construct that can be combined with any instruction. The 8086 family manual defines the use of rep/repe/repz (0xf3) and repne/repnz (0xf2) prefixes only in conjunction with string instructions, which are movs, cmps, scas, lods and stos; all other uses of those two prefixes are illegal. But since the original 8086 does not have an illegal opcode exception, every instruction necessarily has to do something, even when it is formally undefined. For most instructions, an illegal rep prefix is simply ignored, but this is not always the case; as was relatively recently discovered, using rep on multiplication and division instructions would invert the sign of the result.

While the 8086’s successor, the 80186, added the #UD (illegal opcode) exception, it was apparently triggered only for undefined instructions, not undefined combinations of an otherwise-defined instruction and prefix. Later revisions of the architecture would trigger #UD on undefined uses of the lock prefix, but not rep or repne. It’s not clear to me what Intel’s rationale for this was at the time, although this would prove useful in later revisions of the architecture:

• The pause instruction, available since the Pentium 4, is encoded equivalently to rep nop, which means earlier versions of the architecture will simply ignore it, enabling operating systems to use it immediately without worrying about backwards compatibility.
• A bug in certain AMD CPUs caused a performance penalty upon encountering a return instruction in certain circumstances. To work it around, the return opcode may be encoded as rep ret instead of a plain ret. This avoids the performance problem on defective CPUs, while other CPUs will simply ignore the prefix.
• The Intel MPX instruction set extension for bounds checking consists entirely of opcodes that were reserved and do nothing on (at least some) previous revisions of the architecture which do not implement the extension. This includes a bnd prefix for branch instructions, which is encoded identically to repne, and as such is likewise ignored by CPUs without MPX support.

Nevertheless, undocumented combinations of the prefix with instructions remain undefined, with all the usual consequences of undefined behaviour: they may break on subsequent revisions or alternative implementations of the architecture, they may behave non-deterministically, they may trigger strange behaviours, up to and including naſal dæmons. You are probably better off not using them, unless you have very good reasons to ignore such problems.

As for why the assembler doesn’t stop you from using illegal instructions: historically, assemblers have been pretty lenient in handling coding errors, for reasons both practical and, so to speak, philosophical:

• Resource constraints. Back when assembly languages were dominant, CPU clocks ran at best a couple of megahertz, while memory sizes (working memory and exchangeable media, which in that time meant floppies) were at best a couple hundreds of kilobytes, so performance tuning was very important. (It was quite the feat to implement a reasonably performant spell checker back then.) Adding extensive error checks to a program (any program, not just an assembler) incurs costs, considered significant at the time, in running time and code size, not to mention complexity. Naturally, most programmers just didn’t bother, and instead ill-fatedly directed users to read the fabulous manual.
• Completeness. An assembly language is supposed to be just a human-readable representation of the architecture’s instruction encoding; as such, it ought to be able to represent any possible instruction, whether is actually supported and makes sense to execute or not. For example, current x86 assemblers are able to encode moves into and out of control registers CR5 through CR7, even though those registers currently have no defined use, no such is in sight either, and the instructions simply raise #UD. (I suspect the registers don’t even exist physically on the chip.)
• Forward compatibility. Occasionally, illegal instructions become legal. When that happens, it becomes advantageous to be able to use them without having to upgrade the assembler. If your assembler hasn’t yet added built-in support for the bnd prefix, you can simply write it as repne today without waiting for an updated version. This reason was much more important before Internet-based software distribution became popular, but still remains valid to an extent.
• Support for de-facto standards. An opcode that is officially undocumented may nevertheless still exhibit useful behaviour on actual hardware. A programmer who understands the risks and/or is only interested in programming against the particular implementation of the architecture which provides it may choose to use such undefined instructions anyway, despite them being officially unsupported. When such use becomes popular enough, the manufacturer may unofficially maintain the behaviour for the sake compatibility, and assemblers may start including a mnemonic for the opcode as a de-facto standard. Eventually, the opcode may even become officially supported and documented; such for example happened with icebp.
• Programmers’ arrogance. There is an enduring stereotype of a low-level programmer who takes pride in doing all the menial tasks of programming by hand and proclaims things like ‘Real Programmers don’t use Pascal’ or ‘how dare the compiler check my array bounds’ or ‘if my program ever has a buffer overflow, it’s the user’s fault because they’re holding it wrong’. (If Raffzahn’s answer is any indication, it’s still a somewhat justified one.) Such a person would probably consider an assembler which checks their errors to be an insult to their programming ability and not consider it an attractive feature.

Answer 2

You can't repeat arbitrary instructions with rep.
In asm syntax, rep just means to include an F3 byte as a prefix for this instruction. There is no implication that it actually means repeat, it's just shorthand for db 0xF3. Assemblers exist to help you put the bytes you want into an object file. It's up to you to make good choices.

---

When F3 rep doesn't apply to an opcode, in practice it's ignored¹ (same for other prefixes except lock, in CPUs after 8086). Intel documents the behaviour as "undefined" because future CPUs could use that sequence as a multi-byte opcode that does something different. (e.g. lzcnt = rep bsr, giving different results on CPUs that know about lzcnt vs. older CPUs). So they're not future-proof safe for padding instructions for alignment to avoid NOPs.

Note 1: at least on modern x86 CPUs. As @user3840170's answer points out, rep mul can invert the sign of the result on actual 8086! Perhaps this is why Intel only ever says anything about specific cases, not in general.

From Intel's x86 manuals, vol.2, REP/REPE/REPZ/REPNE/REPNZ entry:

The REP prefix can be added to the INS, OUTS, MOVS, LODS, and STOS instructions, and the REPE, REPNE, REPZ, and REPNZ prefixes can be added to the CMPS and SCAS instructions. (The REPZ and REPNZ prefixes are synonymous forms of the REPE and REPNE prefixes, respectively.) The F3H prefix is defined for the following instructions and undefined for the rest:

• F3H as REP/REPE/REPZ for string and input/output instruction.
• F3H is a mandatory prefix for POPCNT, LZCNT, and ADOX.

(Intel's list is not a complete, for example omitting TZCNT, and use as part of PAUSE. But perhaps they're intentionally omitting cases where backwards compat by ignoring the REP prefix is useful. e.g. tzcnt = rep bsf same result when the input is non-zero, pause = rep nop, F3 = xrelease HLE prefix, and so on. Also omits mention of F3h as a prefix for some scalar SSE instructions, and of F2h as part of crc32 for example.)

Every time Intel has introduced a new instruction using an F3 (or F2) byte as a mandatory prefix, they've documented the fact that older CPUs ignore the prefix when that's useful. This has allowed transparent Hardware Lock Elision when used with lock add or mov stores, for example, as well as pause safely running as nop on older CPUs. This is possible because CPUs do in practice ignore such prefixes; Intel just chooses not do document it except on a case-by-case basis when it's relevant and useful. (I haven't checked AMD's docs but I assume it's basically the same.)

(Some SSE1/2 scalar instructions like addsd use F2 or F3 as a mandatory prefix; I assume a CPU with SSE1 but not SSE2 would run F2 0F 58 as addps instead of addsd, although this is not documented or useful. F3 0F 58 is SSE1 addss, so there wouldn't be any CPUs that could ignore the F3 and run it as SSE1 addps)

Related Q&As for more detail:

• What does “rep; nop;” mean in x86 assembly? Is it the same as the “pause” instruction?
• What does rep ret mean? not documented, but a de-facto standard thanks to widespread use by GCC to avoid AMD branch-mispredict penalties for a 1-byte ret following a branch. This is explicitly taking advantage of the "current CPUs ignore rep" behaviour, not as a new instruction.
• why do repe and repne do the same before movsb? - Apparently (some?) CPUs run F2 movsb the same as normal F3 rep movsb.
• Combining prefixes in SSE - mixing and matching prefixes other than mandatory ones.
• Repeat prefixes and mandatory prefixes in x86 - f2 0f 38 f0 is definitely CRC32, not rep movbe, despite its 0f 38 f0 encoding.

---

F3 to actually repeat something

An F3 byte behaves as a true rep prefix (repeating for [E/R]CX counts) only for the "string"¹ instructions documented in the rep entry in Intel's current vol.2 manual; still only for stos/movs/ins/outs². And somewhat surprisingly also lods, even though just loading usually doesn't have side effects (except in MMIO regions, or faulting on unmapped pages or segment limits on CPUs later than 8086).

The F3 byte can also act as a repe aka repz prefix for cmps and scas, documented in the same page of Intel's manual. In that case, repeating stops if ZF==0 after any step. (As with rep, E/RCX is checked for !=0 before even the first step, but ZF is only checked after the first step so you don't have to create good FLAGS state before using, unless E/RCX might be zero initially and you want to branch on ZF after.) repne aka repnz is an F2 byte, and only applies to those two instructions as a repeat with the opposite ZF condition.

Note 1: In C terms, rep/repe implements memset/memcpy/memcmp/memchr, not str* for 0-terminated C strings. These are "explicit-length string" instructions, which I think was common at the time 8086 was new, and is now back in favour with C++ std::string unlike C char*.

Note 2: insb/w and outsb/w were new in 186.

Note that only rep movs and rep stos are fast on modern CPUs, with optimized microcode that copies or stores in chunks of 16, 32, or maybe even 64 bytes at a time. (SIMD loops can still be better). repe cmpsb is dog slow (e.g. one compare per 2 cycles on Skylake, 3 on Zen) and easily beaten with even SSE2 pcmpeqb / pmovmskb to implement memcmp. Or even scalar bithacks for strlen / strcmp.

8086 design bug: when a rep instruction with multiple prefixes is interrupted, the saved IP points at the last prefix. So cs rep movsb will resume as rep movsb without a CS segment override!

Later x86 CPUs fixed this, but on 8086 / 8088 you either avoid that entirely, or write it as rep cs movsb inside a loop that compensates. So if it's interrupted, on resume it runs one cs movsb (without decrementing CX because the rep got skipped). You can detect the non-zero CX and either re-calculate from the pointers or work out some logic to get CX correct and jump back to before the rep prefix.

Fun fact #2: when single-stepping (with TF=1, the trap flag), rep-string instructions trap after every count.

Fun fact #3: Per discussion in comments on @Raffzahn's answer, 8086 CPUs assert some external queue-status signals every time they decode a prefix or an opcode. Prefixes clearly behave differently from instructions in terms of software-visible behaviour, so this appears to be an 8086 implementation detail. Prefixes aren't truly "opcodes", despite being documented in the manual along with instructions. (rep and lock still are; segment overrides don't have their own entry in Intel's vol.2 manual. Other prefixes like xacquire (F2) and xrelease (F3) also have entries)

Answer 3

[You might want to add a bit more information to start with - like what assembler you're using, or what kind of 'Error thrown' you expect. From assembler? Linker? Debugger? Or some OS/Runtime? Otherwise it's hard to give any definite answer.]

In general REP is simply an opcode that can come at any point like any other opcode. It isn't invalid on its own. Now, when it comes to usefulness it might make more sense in front of any string operation.

So maybe answer for yourself, why do you expect to be stopped from using it - and by what component?

---

The underlaying issue might be more about the assembler used. In general an assembler is meant to allow the programmer to generate any code he wants - even illegal. And while one might appreciate some warning, this case is not illegal in any way. Beside, there is no way the assembler can guess if that instruction is meant as code to be executed or some data, for example used to synthesize instructions during run time. I'd delete any assembler that would stop me from programming right the moment he tries to :)