Huldah was the third daughter of her parents. Rev. Samuel G. Coe, who was her pastor, described her as always most elegantly attired in black silk, and said that she was usually alluded to as "Lady Stewart." She was descended from Richard Hubbell and also from Captain Matthew Sherwood.