In the 1850s James C. Smith, esquire, purchased the property from Mr. Penfield. James Smith later became a justice of the Supreme Court and occupied his seat for 34 years. Upon his demise, around the turn of the century, Maynard N. Clement, a prominent attorney, acquired the property.
In 1911, Clement left the Canandaigua area to accept an appointment as state excise commissioner. The property was sold to Dr. & Mrs. Dwight R. Burrell. In 1921, Mrs. Burrell died and the estate was sold to Stewart Brooks Hubbell. Joseph W. Cribb, acquired the property from MR. Hubbell's estate. Mr. Cribb, who became a prominent Judge in the Canandaigua area and owned the property adjacent to the home sold off part of the property to Alfred Morris Gifford in 1945.
Sometime in the 1960s or 1970s, the home was converted to apartments, then later housed a real estate office before becoming a private residence again. In March, 2004 Guy and Jaynee Straw purchased the property to restore to its original grandeur as a Bed & Breakfast, currently known as Inn on the Main.