Ward was married by a Presbyterian Minister in 1821, and in the same year purchased from Edwin S. Duncan 300 acres on what was then known as Skidmore's Niche, and is now known as South Elkins. Ward apparently settled first near what is now the eastern end of Sixteenth Street, at the mouth of a spring.

In 1832 Reverend James Baber held a meeting of Presbyterians in the Jacob Ward house, Ward apparently having moved by this time to what is now the residence of Dr. Metheny. At this meeting people came from miles around, and there were thirty of forty overnight guests. The Ward family for generations were keepers of slaves, and some of the guests at the religious meeting stayed in the slave quarters. Persons who attended the meeting remembered many years later two girls who rode handsome horses over from the South Branch on Thursday to attend the meeting on Sunday. In February of 1849 Ward sold the northeastern portion of his farm to his son, Levi D. Ward, who two years earlier had been married and was probably living in the old house. The transaction just described may not have included the old house, however, as if one plats the boundaries of the tract, near the Sixteenth Street settlement, the calls produce a small square near the river open at the side facing the Metheny property, as if the lines were surveyed in such a way as to exclude the structure from the transaction. Possibly Levi D. Ward had been living in the old house while building a new one on the property his father would ultimately deed to him. If the older house was excluded from the transaction the elder Ward's motives for keeping it are anyone's guess. In the 1850 census of Randolph County Levi D. Ward is listed adjacent to his father.

Behind the Metheny house next to the large house recently built at the top of Scott Hill there is the small family cemetery of Jacob Ward. A number of the family members are at rest there, the earliest known grave being that of Morgan B. Ward, Jacob's son, who died at the age of 19 on October 17, 1856. In 1858 Jacob Ward's daughter married Andrew Taylor, and several members of their family are in the cemetery, they being the ones who inherited the house, and theirs being the descendants who were to occupy it for generations.

The fact that the house on Taylor Avenue looks like something on a Georgia plantation is no accident. The Wards were of Confederate sympathies when the Civil War divided the people of the nation. During his lifetime Jacob Ward was the owner of at least ten slaves, and other members of the family were also slave holders. When Jacob Ward made his will in October of 1864, he left his home farm to his son William T. Ward if William should have returned alive from the Confederate service. Such an action would have been an unmistakable endorsement of his son's confederate service. He stipulated that if William did not return, Andrew and Louisa Taylor were to have the farm, this being the eventual outcome. Jacob Ward died three months later and is buried in the cemetery behind the house.

Elizabeth Ward, jacob's wife, outlived him a number of years. In 1880 she was living on the farm with the Taylors. In the 1890's the Taylors' daughter, Annie Laurie Taylor became a doctor, but it has been said that the pressure eventually overwhelmed her, as her mother mentions her mental condition in her will in 1915. Blaine W. Taylor, Andrew's and Louisa's son, became also quite successful, and he inherited the house from his parents. It possibly was he who built it into the magnificent structure that it is today. At the time of its occupancy by Dr. Condry, one could still see the date "1882" painted on the brick in very faded paint, suggesting that it had been there for a very long time. Descendants of in-laws of Jacob Ward occupied the house for many years, the last being Doctor Condry, who had been married to one of the Taylor descendants.