The allegations range from nepotism to contracts not being put out on bids, to rent not being collected adequately.
Two employees have been fired. Juanita English and Maxine Holly could also face federal charges.
"All the residents in the housing authority are suffering," says Charles Stewart, interim executive director of the Benson Housing Authority.
The average age of residents living in the duplexes run by the housing authority is 75.
Isaac Moore is happy living here, but other units, including one he recently vacated, are in need of repairs. There is no money to pay for them because the federal government has frozen the funds.
Stewart, the former executive director of the Benson Housing Authority, has been called back to the job to clean up what is being called a financial mess.
"A lot of invoicing stuff has been paid for, but the work was not done and there were people paid for things they shouldn't have been paid for," Stewart says.
Benson Mayor John Johnson says he is "disgusted" over allegations of financial mismanagement.
"Repairs were made, but to the extent that did we get a dollars worth of work for a dollars worth of tax money expended, no we did not," says Johnson. "There's not adequate controls to tell you this. The bookkeeping system that Ms. English was involved in using is just a sham. It's just made up."
English says the allegations are not true and otherwise had no comment.
TheDepartment of Housing and Urban Developmenthas given the Benson Housing Authority 60 days to get the situation back on track or the agency faces a takeover.