Police have charged Seth Parker Price, 28, of Stantonsburg Road, with several counts of breaking and entering, burglary and arson.
Officers responded to a break-in early Sunday morning at a home at 3717 Cattail Lane in the Willow Run subdivision. They also noticed a nearby house that was on fire.
Authorities said an officer saw Price running from the area. Price was later questioned by authorities and released, but he was arrested at his home Friday morning.
Investigators believe Price is responsible for two Jan. 13 fires that destroyed Memorial Baptist Church and damaged Unity Free Will Baptist Church, along with a break-in at Oakmont Baptist Church.
Damage at Memorial Baptist Church was estimated at $1 million. The fire at Unity Free Will Baptist Church was contained to a single room.
Authorities said Price acted alone in all of the cases. Detectives said they believe they know why he commited the crimes, but that motive has not yet been released.
"It's important that this case still needs to be tried," said Greenville Police Chief William Anderson. "We can't try this case in the press and unfortunately, we can't release any of the details as to motive or anything like that. All of that is part of the investigation and will have to come out later."
A task force made up of local and federal investigators was created to investigate the church fires. Investigators said there is no evidence to suggest Price is connected with any hate groups.