The missing piece for me was the math background. Will try to connect math and actual implementation. The fully explanation is given by user3840170's post (hope my post is complementary to user3840170's brilliant post). As example, let me calculate 127:10. The result is 12 and reminder is 7:
127 - 10 * 2^5 = -193  * 2^3
127 - 10 * 2^4 = -33  * 2^4
As was said in the comments, this is the binary long division algorithm. The long division is performed by carefully juggling the bits between the registers and the carry flag.
The algorithm is probably best analysed by looking at each half of the loop as a whole.
MOV D, A
MOV A, E
MOV E, A
The first half of the loop shifts ...
Don't understand purpose of SUB B/ADD B
It's an underflow handling
SUB B ; Subtract divider
JNC loop_1 ; No underflow -> Continue
ADD B ; Correct underflow
JMP loop_1 ; -> Continue
B is subtracted from A.
If there is no underflow (A was larger than B) then the loop continues with the new (positive) value of A.
A is restored to ...