The disagreement is over railroad crossings in Clayton. The town council voted to close three crossings, claiming they were not safe.
Two people died at one of the crossings.
Some people believe the decision will segregate the town.
Carolyn Creech said she and other black residents will be cut off from the rest of town.
When asked if she felt she was being discriminated against, Creech said: "Yes, we are."
Town Manager Steve Biggs responded that "the neighborhood on the North side of the tracks, for one thing, is a mixed-race neighborhood, so that's just not even relevant to the dialogue."
Creech shot back: "The council is totally white, so, quite naturally, they're going to say that."
Creech said the town has never put a safety signal up at the crossing where two people have been killed.
"If this crossing was so unsafe," she said, "it should have been the first one to have a safety signal."
Biggs said no signal was put up because "the matter of that particular crossing is that the geometry is unsafe."
Town officials said closing the crossing is the first step toward progress. The next step is to connect Front Street with Highway 42, which will revitalize the area.
People here said that will be a traffic nightmare and that the town has other motives.
"Ten years we've been here," resident Joseph Whitley said. "We haven't seen any progress."
Said Creech: "(Council members) are upset because they have to wait on 70.
"They're upset because they have to wait 20 minutes," Creech said. "So, they're going to make life miserable for us so they can get home quicker, so they can play five more minutes of golf. If that's not discrimination, I don't know what it is."
The crossings probably will not close for at least two years. The town also has federal money to build a state-of-the-art tunnel between the two sides.
People here said they will believe it when they see it.
"I think the bottom line is that people need to relax and wait and see," Biggs said.