But after the housing bubble burst, the retirees stopped coming. On Main Street, the Wigwam, one of the town's oldest restaurants, now does half the business it used to, according to Moe Royels, who opened the diner in 1961 and sold it five years ago.
"People moved out of town," Royels said from his seat at the restaurant, where he returns every afternoon for a cup of coffee. "Some of these subdivisions are still sitting vacant, with the curb and the gutter in but nothing else."
Due to changing baby boomer migration, rural retirement counties grew 0.4 percent annually from 2007-2012, down from 1.6 percent annually from 2000-2007. During the housing boom, these retirement destinations were growing faster than the rate of the nation as a whole but are now increasing more slowly. The overall U.S. population is now growing by about 0.8 percent each year.
In Florida, almost all counties experienced slower growth or a reversal of boomer population growth since 2010, said Mark Mather, an associate vice president for the Population Reference Bureau who analyzed the numbers.
Other counties showing sharp drop-offs in the boomer population include Forest County, Pa.; Trinity County, Texas; Middlesex County, Va.; and Banks County, Ga.
"The recent decline in migration rates among baby boomers is significant because boomers were expected to jump-start economic growth in rural America," said Mather, noting that parts of the rural Midwest and Appalachia had been losing population for decades. "But since the recession, we've seen more boomers aging in place. This is bad news because as baby boomers get older, they are less likely to move."
Other census findings:
—The 65-and-older population grew 4.3 percent between 2011 and 2012, to 43.1 million, or 13.7 percent of the U.S. population.
—Florida had the highest share of residents 65 and older, at 18.2 percent, followed by Maine and West Virginia. Alaska had the lowest share of older residents, at 8.5 percent, followed by Utah and Texas. By county, Florida's Sumter County was tops in the share of the 65-plus age group, at 49.3 percent.
—The 85-and-older population increased by about 3 percent from 2011 to 2012, to almost 5.9 million. The number of centenarians rose to almost 62,000.
—The nation's median age rose to 37.5, up from 37.3 in 2011.
Dreier reported from Las Vegas.
Copyright 2013 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.