Whoa, don't throw out the baby with the bath water.
Assuming that you've researched your ancestry the right way, beginning with yourself and going backward generation by generation, then the first family name-dit name combination you come across (in reliable sources) should be the correct one. On this basis, it should be the the Dube-Delorme line you discard, not the Lemay-Delorme.
However, I wouldn't throw away anything just yet. Before moving on, you need to confirm the facts regarding Alphonse Delorme first. How was this individual identified as your ancestor, how reliable is this information (not very, it appears), did he actually use the family name Lemay (or Dube) in any records, are there any other Alphonse Delormes who could be your ancestor, and so forth.
Note that statistically speaking, the probability is much higher that someone with the name Delorme is descended from the Lemay-Delorme family, Simon-Delorme family, or Hazeur-Delorme family is much higher than from the Dube-Delorme family. The following is the relative frequencies of the appearance of Delorme with various families in the PRDH records. You may, in fact, be descended from one of these other families.
Frequency Dit Name Family Name 21 DELORME DUBE 37 DELORME FAFARD 278 DELORME HAZEUR 50 DELORME HENAULT 167 DELORME LEMAY 18 DELORME RIEL 198 DELORME SIMON 30 DELORME SOUMANDE
Steve Eno Researching Enaud/Henault dit Canada (and Delorme)and allied families