If you took all the electronics from a monitor like the IBM 5153 and substituted just the picture tube from a color TV, what would still be lacking in terms of display quality?
The answer to your question depends almost entirely on which color TV CRT you wanted to use as a monitor. While any color computer monitor from the 1980s could serve as a color TV, just by adding the needed TV tuner circuitry, the converse of this statement is not true.
The suitability of monitor tube as TV vs. TV tube as monitor comes down to the physics of light, the discernment limitations of the human eye, and the tradeoffs between brightness and sharpness that are inherit in CRT shadow mask specifications. Fundamentally, a TV is (usually) meant to be viewed from across a room while a computer monitor is meant to be viewed from a couple feet away. For the former TV, brightness is critical as the light travels farther before reaching the viewer. For the latter computer monitor sharpness is critical because small text and fine line graphics are best viewed that way, and because broken areas caused by inter-pixel spacing are visibly discontinuous at this close viewing range.
With these considerations, it is obvious that certain specifications, like a higher dot pitch, may be perfectly suitable for a TV while being severely degrading to the quality of a computer monitor display. Specifically, a dot pitch of 0.28 or less is often cited as appropriate for computer monitor CRTs, while TVs can go significantly higher. So, that means you ought to choose a TV CRT that has the right specs to use as a monitor. In the case of you care greatly about monitor quality, you might spend the extra money required to build the CRT to your desired specification when there were no mass-produced CRT's already meeting those specifications.
A case in point would be Apple's early RGB displays. These were very high spec'd and expensive CRT's optimized for sharpness and a small dot pitch. The screens were often only 9 to 12 inches diagonal, which is usually too small for a consumer grade color TV. So Apple opted for the CRT that emphasized the specs that matter for close-up viewing, and their customers paid the extra cost to reap the benefit.
NOTE: I've seen the IBM 5153 dot pitch claimed to be 0.31. This would make it so-so quality, and would explain why many PC users preferred working in MDA/Hercules modes, or springing for a Princeton color monitor.