These comments are taken from: The Anti-Federalist Papers and the Constitutional Convention Debates -- authored by Ralph Ketcham
Page 150 -
When arranging his notes for the debates of the Constitution of 1787 in old age, Madison observed that his Comments on suffrage did not convey his "full and matured view of the subject". He thus appended two notes, a short one probably written on August 7, 1787, and a larger one probably written in the 1820's.
During the convention there was much debate and rationale offered on the subject of whether only land owners would be allowed to vote in elections for one or both branches of the legislature, or whether other free persons (meaning men of course) would be allowed suffrage. There was a lot of concern for the time when land in America would no longer be free, and when the land owners (mostly by inheritance) would become the minority of the citizenry as it is today. Of interest here is much of the text of the second note which addresses the effects of constituency size on the likely outcome of elections.
Page 155 -
"Should Experience or public opinion require an equal and universal suffrage for each branch of the Government such as prevails generally in the U.S., (then) a resource favorable to the rights of landed and other property, when its possessors become the Minority, may be found in the enlargement of the Election Districts for ONE branch of the Legislature and a prolongation of its period of service. Large districts are manifestly favorable to the election of persons of general respectability, and of probable attachment to the rights of property, over competitors depending on the personal solicitations practicable on a contracted theater. And although an ambitious candidate, of personal distinction, might occasionally recommend himself to popular choice by espousing a popular though unjust object, it might rarely happen to many districts at the same time. The tendency of a longer period of service would be, to render the Body more stable in its policy, and more capable of stemming popular currents taking a wrong direction, till reason and justice could regain their ascendancy.
Should even such a modification as the last be deemed inadmissible, and universal suffrage and very short periods of elections within contracted spheres be required for each branch of the Government, the security for the holders of property when the minority, can only be derived from the ordinary influence possessed by property, and the superior information incident to its holders; from the popular sense of justice enlarged and by a diffusive education; and from the difficulty of combining and effectuating unjust purposes throughout an extensive country; a difficulty essentially distinguishing the U.S. and even most of the individual States, from the small communities where a mistaken interest or contagious passion, could readily unite a majority of the whole under a factious leader in trampling on the rights of the Minor party.
Under every view of the subject, it seems indispensable that the Mass of Citizens should not be without a voice, in making the laws which they are to obey, and in choosing the Magistrates, who are to administer them, and if the only alternative be between an equal and universal right to suffrage for each branch of the the Government and a confinement of the entire right to a part of the Citizens, it is better that those having the greater interest at stake namely that of property and persons both, should be deprived of half their share in the Government; than, that those having the lesser interest, that of personal rights only, should be deprived of the whole."