I believe the Swayze clan are included with the Parr, Corwin, Horton and other families that were on Long Island in the 17th century. It is my opinion that they originally left the Massachusetts Bay Colony because of differing religious views - in opposition to Governor Winthrop, et al. Several members of these families migrated to Morris County from Long Island, before the Revolutionary War.
As to your Caleb Swayze, Jr (aka Swezy) - my notes from years ago indicate he lived in Black River. I believe there was a 200 pound reward for his capture - and he was actually killed on the premises of Isaac Badgley in the Great Swamp by a party under Captain Carter. Your Caleb was a relative of Badgley's wife. They were Loyalists, thus the families migration to Canada. My notes don't indicate the year of death -- maybe 1782?
There is a Swayze Cemetery in New Jersey - but I'm told that vandals have destroyed much of it.
I track several Swayze descendants starting with Samuel who was born 20 Mar 1689 in Aquebogue, Suffolk County, New York (aka Long Island, etc) He married Abigail Coleman who I believe is related to my third great grandfather's(John Parr) brother, Matthias Parr. Matthias married Amy Coleman in 1779 in Morris Town. Abigal married Israel Swayze in 1776 in Hope, Warren, New Jersey. Israel is the grandson of Samuel - among the first Swayze's in the Colonies.
Let me know if this helps -- and if you are tracking any Parr family members.