While there was a fellow during this era full-named Peter Stanford Duryea, or Duryee, a banker and real estate man in New Jersey, I think it much more likely that your photo is the work of Sanford Duryea. This man, who had studios in NYC and Brooklyn, Long Island, was very definitely a photographer--one of the earliest professionals in his trade. You can pick up some additional information on this Sanford Duryea on the ancestrycom Duryea message board where the discussion concerns his brother Townsend Duryea, of Australia.
Said Sanford Duryea was born 22 February 1833, died 14 April 1904.
He followed his brother Townsend to Australia, married there, then returned to Long Island, New York.
If you will use google.com and search on Townsend Duryea, you will find more on this particular branch of the Duryea-Duryee family. Or, contact me privately at email@example.com; or firstname.lastname@example.org.