I'm sorry for the lateness of this response, but I just today saw your posting on this.
I have a relative (a cousin) who was named Elmer, born about the time you're looking at. He was born 17 April 1895 in Rochester, NY and he married a Ruth WHITE 19 October 1920 in Webster (a suburb of Rochester). Since Perry, NY isn't a large distance from Rochester (much closer than the Trumansburg, NY, area where his other relatives lived), it's possible my Elmer might have boarded with the SPENCERS for a while, but returned to Rochester before he got married. Then again, he would have been three years younger than the Elmer you are looking for, and he definitely did NOT marry Anna SPENCER. Unfortunately, this Elmer died in 1983, and his wife a few years later, so I can't check with them to follow up on the details.