The first record of a school in the area of the Lee House dates to at least 1734, when one was established by the Second Ecclesiastical Society of Lyme.
On April 2, 1803, Elisha Lee, owner of the Thomas Lee House and farm, deeded seven square rods to the 2nd Society for a school, and by 1805 the present building had been erected. Originally located on the north side of today's Route 156, some 500 feet east of the Lee House, the school and its surrounding neighborhood were given the name "Little Boston" because the quality of the education was considered to be comparable to that of Boston.
From 1865 until its last class in 1922, Little Boston was one of nine school districts operated by the Town of East Lyme. Eventually, the building was moved to its present site, and, in 1926, the building and land were deeded to the East Lyme Historical Society. In 1973 it was restored to its appearance in the early 20th century and opened to the public.