On the surface, it seems that there is no connection.
James BALDWIN who married c1810 in Washington, Mason Co. KY to Sally HARRIS was father to James H. BALDWIN c1813.James SR seemed to have remained in Mason Co. KY thru 1840 with son departing for Platte Co. MO in 1840.
James BALDWIN SR seems to have been born prior to 1790 in Maryland and came to KY with brothers, Samuel and David.
I am speculating they may have been descendants of a Henry BALDWIN of Anne Arundell Co. MARYLAND whose son James remained there while Henry JR removed to Caswell Co. NC.
I do note the name David appears in your line like this family but leave it to you to develop a connection.