There are very few records for Bay County during that period...they were just getting organized. If you check out the Bay County Rootsweb site, though, you'll find quite a bit of info on the early Presbyterian church in town.
The other thing to check out are the state censuses. There would be 3 of them--each taken mid-decade--on film and able to fill in many gaps for you--including when they moved to Michigan. The LDS and your local library (Interlibrary loan) are able to order them for you.