From 1863 to 1864 he was connected with the hardware house of Warren, Hart, and Lesley and their successors J.M. Warren & Co. of New York City. In the fall of 1864 he removed to Brooklyn and was connected with an old established hardware house in that city, but failing health led him to take a trip to Virginia in March, 1869, which resulted in his purchasing a farm in Culpepper County (near the then Virginia Midland R.R.), midway between Culpepper Court House and Orange Court House. The tract contained 337 acres, the last homestead of Hon. Daniel F. Stoughten, a descendant of Capt. Stoughten of the Revolution, who was granted a large tract of land by he Government for his services during the Revolutionary War. It was across this farm the Northern troops charged the Rebels at the Battle of Cedar Mountain, or more properly Stoughten's Mountain. The farm when he bought it was without buildings or fences, having been devastated by both armies. Several years were spent building, making fence, improving and clearing up generally, hoping for good returns, which never materialized, owing to the impoverished condition of the soil, the result of the old style of Virginia farming, merely striving to raise corn and bacon to feed their slaves, wherein their available cash was invested. His neighbors treated him very coldly and looked upon him with suspicion, for which, he says, "They were excusable for no one could realize their losses except by personal experience." But to use his own words: "They gradually learned that "Yankee Stewart" was a decent sort of chap, and if one can believe surface indications they were sincerely sorry when that same Yankee went back north." He made many friends, and he looks back to that period of fourteen years residence in old Virginia with many pleasant recollections (Severance, B. Frank. Genealogy and biography of the descendants of Walter Stewart of Scotland and of John Stewart who came to America in 1718 and settled in Londonderry, N.H. Greenfield, Mass. : T. Morey & Son, 1905, pg. 149-151).