I am not sure if you have any online subscription access, but there is an UPTON document with lots of information on ancestry.comhttp://content.ancestry.com/Browse/BookView.aspx?dbid=26000&pageno=270http://content.ancestry.com/Browse/BookView.aspx?dbid=26000&pageno=270
It is called "Upton family records : being genealogical collections for an Upton family history" by William Henry Upton, M.A., LL.M.London, 1893
Looking here, I find your Rufus Upton on page 270.His "number" is 1 A F J D A F
(My ancestor incommon is 1 A F J which is Caleb Upton)
This tells me starting with the first American Ancestor:
1 A = John Upton, m. Sarah Thompson (b. abt 1620, d 1699) (Page 184), the founder of the New England family of Uptons-- one of the most distinguished and probably the most numerous of Upton families extant ; a family who representatives are now to be found in every continent and in nearly every important country on the globe.may have been born either just before or just after the year 1620.Probably embarked at London in the "Primrose" July 27, 1635.To Salem about 1658, between 1672 and 1678 removed to Reading.
1 A F = William, b. 6/10/1663; m. Mary Maber.(page 198).Born in Salem, probably at Wood Hill.In the April, preceding, his parents had lost by death a son and two daughters, all in a few weeks.They gave him the name of his deceased brother.He married Mary Maber 5/27, 1701.She was bapt. in Salem Village 4/6/1707- the first known connection of an American Upton with any church.He and his brother Samuel together received, by their father's will, the Wood Hill estate and other lands in Salem.They lived near together, perhaps in the same house, nearly all their lives, and together purchased much land in Salem and Reading.They, however, divided the Wood Hill estate, William taking the west, and Samuel the east half.This was done, says Vinton, in 1708, but the deed of William and Mary Upton, though dated 4/6/1708, was not acknowledged until 5/23/1732, nor recorded til 5 Sept. of the latter year.He died in 1739 or 1740 leaving 10 children
1 A F J = Caleb, b 2/4/1722; m Mary Steward.
1 A F J D = Robert Upton of AMherst, N.H., Salem, Mass., Cavendish, Vt., and Harrington (now Millbridge), Me., born 1758, married Anna Wheelock 1784, and died 1824.He served in the Revolutionary war, and from 1817 to 1820 was keeper of the Petit Menan light station, Me.Soon after the war of 1812 he and his brother Jebuthun were forced to take lands on the Narraguagus river, Me., for a debt due them from James and Robert Foster ; and a son of each ultimately settled there.
1 A F J D A = Hon. Samuel Upton of Salem, Mass.; Castine, Me. ; Boston, Mass. ; Bangor, Me. ; and Washington, D.C. ; was born in Middleton, Mass., in 1784 ; and married Rebecca Allen Peirce in 1810.He engaged in merchandising in Salem and Boston, and iin the shipping business in Castine.He edited the Bangor 'Gazette' and 'Whig,' and was much in politics.He was Collector of the Port of Castine, and in 1819 represened that town in the first General Court of Maine.He died iin 1842, leaving a widow and one of the most distinguished families of children that ever bore the Upton name:
a. Wheelock Samuel
b. Charles Horace
c. Francis Henry
d. Edward Peirce
e. Rebecca Peirce
f. Rufus Allyn
g. Helen Augusta b. 1822, became an artist, lived Paris
So, if you followed all of this, this should give you several more generations back to the first American ancestor.