My solution was to A) soft-cap advancement. You will reach a point where you can't progress quickly, no matter how good you are. B) I knew the power the main antagonist was going to have as soon as I figured out who he was (I am primarily a Panster), and I knew all power progression had to be kept under that cap.
One of my methods of keeping the power down has been side-grade advancement. Basically, people can split their focus/path into two compatible paths. This allows more flexibility, such as a warrior-monk who is also now a priestess. The religious training was already there, she's following the same goddess, but instead of focusing on just herself and following the ideals of her goddess, she's now helping others pursue their passions. This has slowed down her martial/chi improvements, but she can now heal via prayer and give other blessings.
This is also part of her character development.
She'd be stronger, faster, tougher, and maybe able to do more things with her chi if she had remained focused on just her martial arts, but rounding her out was better for her story wise and makes it easier to keep her power within bounds.