The story goes that Jared Coffin built the brick estate on the sprawling undeveloped lot on the corner of Pleasant Street and Mill Street as an escape for him and his wife at the edge of town. Coffin, a whaler who did well for himself in the early 1800s, called the house Moors End, since it stood at the edge of what was then the entrance to the Middle Moors. But just a few years into their stay, Coffin’s wife decided the brick mansion was too far from town. They left the house and built another brick mansion at the corner of Broad and Centre. That property is known today as the Jared Coffin House.

For the past 200 years, Moors End has been shaped and amended by the changing times as it moved from family to family, and from generation to generation, from the Coffins to the Hallets, the Minturns, the Gardners, the Williams, the Flanagans and the Sandersons. Throughout its history, portions of the property have served as a Prohibition-era speakeasy, a dancehall, a horse stable, a private bistro and a gathering place for garden parties, fundraisers and galas.

When it was all said and done, Moors End had become the crown jewel onan island well known for its Quaker architecture, whaling captains’ estates and pre-Civil-War homes. It’s the only residential lot over an acre in downtown Nantucket. Its slate roof and brick walls bring it back to another point intime, when visitors from around the world would stroll through its long halls and meander around venerable American elms and beeches and through the hedgework of its Victorian garden, perhaps the finest on the island.

Built between 1829 and 1834, the original house fronting Pleasant Street is the oldest residential brick building on the island. Its iconic brick walls around the perimeter were added in the late 1800s after the Coffins had left. A wing was added in the 1920s, after it was purchased by Edward Sanderson, one of the early benefactors of the Nantucket Historical Association. For that project, Sanderson hired architect Fiske Kimball, who was perhaps most well known for renovating Monticello, and for his 30-year tenure directing the Philadelphia Museum of Art.

Along with the new wing—with its second-story library, kitchen and several sitting areas—Sanderson hired Stanley James Roland to paint a mural around all four walls of the dining room, depicting Captain James Cook’s voyage to Polynesia, as well as the trials and tribulations of a harrowing whaling voyage into the South Pacific.

In the back, the barn tells another story entirely, that of Nantucket’s agrarian past and a time before Wranglers and Defenders whipped down Pleasant Street. The barn—an old horse stable—once towered over the edge of the moors, providing sweeping views of the island from a third-story perch that has since been taken down. But perhaps most spectacular is what’s written on the walls in the fourth-story cupola, where signatures line the room, some dating back to 1856. The etching has faded into the wood, leaving only a guess as to the visitors the property has seen: a man visiting from Glasgow, a family caught in a storm. Not all of those oldest words have survived.

From the top floor cupola, the house peeks over the neighborhood, providing a unique view of the chimneys along Pleasant Street and the steeples of the Unitarian Universalist Meetinghouse and Congregational Church. It’s a reminder for anyone who has visited that Moors End has stood the test of time.