Al-Shabaab militants have fled the key town of Afmadow fearing attacks from the Kenya Defence.