More seniors are living alone. Stop blaming them for the housing crisis.

Ask housing policy experts why housing costs are at or near all-time highs in most cities in America and you’ll get one primary answer: There aren’t enough houses and apartments for people to buy or rent.

There are many reasons for the housing shortage. Home builders haven’t been able to build at a pace that keeps up with demand. Major cities lack enough land to build on. People who already own homes aren’t upsizing or downsizing, which limits the circulation of homes on the market.

But at a time when intergenerational rivalry has become a viral meme, it’s been easier for many to blame one particular punching bag: baby boomers.

The theory goes like this: Senior citizens historically will downsize into smaller houses as their children leave the nest. This allows younger families with starter homes to expand into larger houses. That in turn allows first-time homebuyers to buy starter homes that the younger families are moving out of.

But since the financial crisis, this cycle of homeownership has come to a halt, and baby boomers who “won’t move,” particularly those who live alone in houses that could otherwise be home to multiple people, are often blamed. This chokes off the number of available homes for sale, the homes that are for sale are subject to bidding wars, and the price of housing rises as inventory drops.

“As supply has not kept up with demand, people who already own their homes who would normally trade up or downgrade or whatever it is, they’re finding it harder to move across the tiers,” said Cheryl Young, an economist at Zillow. “We don’t see as much mobility as what people would normally go through during a life stage.“

Boomers are staying in their houses longer because they’re living longer and more independently than previous generations. They’re also working longer, so they have the income to stay in a large house if they want to. They’re also more likely to have a child still living at home than seniors did prior to the housing crash in 2008, meaning they still have a need for larger houses.

But the relationship between solo-living baby boomers and high housing costs is more complicated than the prevailing narrative.

First, there’s no doubt that people are staying in their homes longer, which limits the number of houses for sale in any given market. According to ATTOM Data Solutions, a real estate data provider, the average time a homeowner spends in a single home has grown from about four years to more than eight years since the financial crisis.

It’s also true that baby boomers living alone are occupying more housing. Baby boomers, true to their name, are a large generation, so as they age they’re increasing the total number of senior citizens across different age groups.

Census data show that the percentage of homeowners and renters who are seniors who live alone has been steadily growing since 2005, primarily because of the sheer number of boomers who are moving into the older age demographics relative to previous generations.

Real estate listings site Trulia conducted a study to examine whether the rise in the number of senior citizens living alone is the root cause of the housing affordability crisis. While the trend of boomers living alone and not downsizing is happening in aggregate across the nation, there’s no such thing as a national housing market. A home for sale in Dallas doesn’t matter to someone looking to buy a home in Chicago, and thus any market forces playing out in Dallas aren’t necessarily playing out in Chicago.

This means that to assess what impact boomers living alone are having on the housing market, one has to look at each city’s housing market individually. Trulia looked at the 100 largest metro markets in the United States to see where boomers are failing to downsize the most and what affordability is like in that city.

They defined “able to downsize” as a senior citizen living in a single-family house by themselves. They plotted the percentage of seniors who were able to downsize in each city against housing affordability in that city. In the data visual below, click on the dots to see which cities fall where on the spectrum.

The cities in the upper left quadrant are cities where there are a lot of seniors who can downsize but housing affordability isn’t a problem. This is where most of the cities fall. The cities in the bottom right quadrant are expensive cities where there aren’t a lot of boomers who can downsize and add to the available housing stock.

As you can see, the most expensive cities in the country—New York, San Francisco, Los Angeles—do not have an excess of boomers who can downsize to a smaller house, meaning boomers are not causing affordability problems in the least affordable places.

Furthermore, the cities that have the most boomers who could downsize are among the most affordable in the country, so if they decided to downsize en masse, it wouldn’t alleviate any serious problem. Depending on how many boomers downsized and when, it could theoretically cause a housing market crash, especially given the cities in question are clustered in areas of the Midwest that have struggled to attract jobs and retain younger workers.

“There aren’t job opportunities [in some Midwest cities], so they’re already drained of a younger population,” Young said. “Those markets, you don’t see a lot of home price growth there in general. There are lower levels of demand, especially from those that are interested in moving there for first time, homebuyers or people starting out with families.”

The Wall Street Journal posited this theory in a story that noted that two-thirds of every home will turn over by 2037, fueled by a selloff of the 21 million homes owned by boomers who will downsize, move into assisted living, or pass away.

However, it’s unlikely that boomers would cause what’s currently a housing shortage to turn into a housing surplus. Boomers are generally defined as people born between 1946 and 1964, and cover a broad age range. This means that any selloff of boomer housing would be staggered over the course of 20 years or more. This would hardly be the type of sudden meltdown that caused the housing collapse in 2008.

“This is a big generation,” said Jennifer Molinsky, a researcher and lecturer at Harvard University. “It’s possible that some places could see an effect, but I don’t think it’s going to happen all at once, especially since people do seem to be trying to age in place.”