Fairfield was a riverboat port town, located 'cross the River. This means that it was on the West Bank of the Tombigbee River, in "the free state of Pickens".
The area was called this because it was so removed from the rest of Pickens Co. and the county government. Residents tended to have more in common with Noxubee Co., than with the rest of Pickens. Even the land was totally different, with different kinds of trees and plants predominating.
I'd look to where people from Noxubee Co. tended to migrate, rather than where people for Pickens tended to migrate.
Since the riverboats were so accessible there, I'd also look up and down river.