In a 3-quart saucepan combine the sugars and milk, and bring to a boil over medium heat, stirring constantly. Continue cooking without stirring until the mixture reaches the soft-ball stage (236 degrees F on a candy thermometer). Remove from the heat. Add the butter and vanilla, but do not stir. Let the mixture cool without stirring until it reaches 110 degrees F and the outside of the pan feels lukewarm, about 40 minutes. While the mixture is cooling, butter an 8-inch-square pan. With a wooden spoon beat the mixture until it becomes thick and begins to lose its gloss. Quickly stir in the pecans, and pour into the pan. (Don't scrape the sides, since they may be sugary.) Cut with a sharp knife into 1-1/2-inch squares.