North Carolina Central University Chancellor James Ammons said Friday it will be tough to leave Durham to become president of Florida A&M University.