He married, first, on Feb 26,1823 at Colrain,MA, Matilda Peck. She was a daughter of Abraham Peck Jr and Arethusa Bullard Peck. She died in Pennsylvania.
He removed to Blakely, Lackawanna Co, PA following his marriage, and returned to Colrain following the death of his wife.
He married, second, on Feb 13,1853 at Colrain,MA, Mary Ann Oaks. It was her first marriage.
Children(by first marriage): John Brown, Joseph Brown, Arethusa Brown McClellan, Rachel Brown, Elizabeth Brown, Matilda Brown Rogers, Mary Brown Everton, Amanda Brown Wiley, Abram Peck Brown, Alvin Brown, Thomas Brown III, Martha Brown, and William Brown.