Leaving leaky coupling/bypass caps in is
going to eventually Bite you
Very true. But in some cases, for example, when you are increasing the size of coupling capacitors to improve the low frequency response, unless you suspect the original coupling caps may be leaky, the easiest and least damaging way is to simply bridge the larger cap in parallel with the original one without removing the latter.
For example, you wish to replace the original 0.001 mfd coupling caps with 0.01's. Instead of carving out the .001 caps, just bridging in a new .01 will result in a total capacitance of .011 (if both capacitors are exactly nominal value). In the real world, the final capacitance will still be within 10% tolerance. Most of the time, 10% doesn't have any significance at all, especially when you are increasing the stock value by 1000%.