One of the most compelling arguments for the infant-baptism position, is the historical evidence.
The following quote from Louis Berkhof is illuminating:
Irenaeus, speaking of Christ, says: "He came to save through means of Himself all who through Him are born again unto God, infants, and little children, and boys, and youths, and old men." This passage, though it does not explicitly mention baptism, is generally regarded as the earliest reference to infant baptism, since the early Fathers so closely associated baptism with regeneration that they used the term "regeneration" for "baptism." That infant baptism was quite generally practiced in the latter part of the second century, is evident from the writings of Tertullian, though he himself considered it safer and more profitable to delay baptism. Origen speaks of it as a tradition of the apostles. Says he: "For this also it was, that the Church had from the apostles a tradition (or, order) to give baptism even to infants." The Council of Carthage (A.D. 253) takes infant baptism for granted and discusses simply the question, whether they may be baptized before the eighth day. From the second century on, infant baptism is regularly recognized, though it was sometimes neglected in practice. Augustine inferred from the fact that it was generally practiced by the Church throughout the world in spite of the fact that it was not instituted in Councils, that it was in all probability settled by the authority of the apostles. Its legitimacy was not denied until the days of the Reformation, when the Anabaptists opposed it. (Systematic Theology p635)
You would think that, from a baptist point of view, if the practice was erroneous there would be a little more by the way of historical skid marks?