with (good) reason
Having a sound, justifiable cause or purpose. Yes, it was I who denied the defendant's application, but with reason—none of his supporting documentation was from within the last six months, as stipulated on our department's website. Officers may only search a person's private belongings or possessions with good reason that can be proven to the courts.
with good reason
Because something has already been proven to be true or justified. Yes, I'm suspicious, but with good reason—I've already caught her twice.