[All following references are in relation to the Archive.Org copy of the Commodore 64 Programmer's Reference Guide ISBN 0-672-22056-3]
VM13 VM12 VM11 VM10 CB13 CB12 CB11 --
It also says, that to set the character pointer addres, you should set VM13-VM10 as the MSB.
The character pointer is a 10 bit counter internal to the VIC, which combined with the Video Matrix (VM13..VM10) gives the memory address where the next character to be displayed is located. See page 436.
The character pointer is nothing one can manipulate (in full). Only the screen address part (VM13..10) can be changed - the pointer (part) to be set.
Similar the character data address is an internal value created from the character base (CB13..11), the character fetched (from screen memory) and the 3 bit row counter. See page 437. It can not be set directly. only CB13..11 is - again the the pointer (part) to be manipulated.
So which of both pointers do you want to manipulate?
The area where the VIC fetches the character to be displayed.
To set the screen memory address for the VIC, the upper 4 bits (VM13..10) have to be manipulated as described on Page 102:
POKE 53272,(PEEK(53272) AND 15) OR A
This will set those to the value in A. Valid values are found in the table on top of page 103. Essentially these are the values 0..15 shifted by 4. If your input is not shifted, this can be done with
POKE 53272,(PEEK(53272) AND 15) OR (A * 16)
That's where the bitmap image for each character is stored. One byte per scan line, 8 in total. See page 107 Character Definition for more details.
To set the address, where the VIC looks up each fetched character for its bitmap, the next 3 bits (CB13..11) need to be set as shown on page 104:
POKE 53272,(PEEK(53272) AND 240) OR A
This will set those to the value in A. Valid values are found in the table right after of page 104. Essentially these are the values 0..7 shifted left by 1. If your input is not already shifted, this can be done with
POKE 53272,(PEEK(53272) AND 240) OR (A * 2)