How Accurate are Zillow Year-Over-Year Home Price Estimates?

Chart from Zillow article below on its home price methodology

Home prices are falling, but how fast, and by what measures?

Down Markets 

Four Major Markets 

Investigating Zillow’s Home Value Index

Please consider Zillow Home Value Index Methodology, 2019 Revision: Getting Under the Hood. 

The Zillow Home Value Index (ZHVI) is Zillow’s flagship measure of both the typical home value as well as housing market appreciation currently and over time. It offers a number of advantages over other housing indices, including better timeliness, better coverage of the market and more visibility into changes in small market segments.

The Zestimate is generated through an ensemble of machine learning models and incorporates data from a variety of sources including public data, user-generated data and real estate data from direct feeds or multiple listing services.

One of the core functions of the ZHVI is to calculate total housing stock appreciation in a market. Total market appreciation is calculated as a weighted average of each home’s appreciation in the property universe.

For example, a home worth $100,000 that appreciates 10% month-over-month adds $10,000 in value, while a home worth $1 million that appreciates 10% adds $100,000 in value. And because the more expensive home adds more dollar value, it represents a larger portion of the total market appreciation.

Zillow Example Based On Its Stated Methodology 

  • 1 home originally priced at $3,000,000 rises 15 percent to 3,450,000
  • 10 homes originally priced at $300,000 decline 10 percent each to $270,000.

In my example above, the Zillow model would say the market is rising despite the fact that 10 out of 11 homes fell in price. 

My estimate says the average home price decline in the example was 7.7 percent (10*10-15)/11 * 100.

Bear in mind that Zillow is not based on repeat sales of the same house but rather “data from a variety of sources including public data, user-generated data and real estate data from direct feeds or multiple listing services.

User generated data? Multiple listings?

Case Shiller Methodology

In contrast to whatever the heck Zillow is doing, please consider the S&P CoreLogic Case-Shiller Home Price Indices Methodology.

The S&P CoreLogic Case-Shiller city indices reflect the average change in home prices in a particular geographic market. The indices are calculated monthly and cover 20 major metropolitan areas (Metropolitan Statistical Areas or MSAs), which are also aggregated to form two composites – one comprising 10 of the metro areas, the other comprising all 20. 

Percentage changes in the indices measure percentage changes in housing market prices given a constant level of quality. Changes in the types and sizes of houses or changes in the physical characteristics of houses are specifically excluded from the calculations to avoid incorrectly affecting the index value.  

The monthly S&P CoreLogic Case-Shiller Home Price Indices use the “repeat sales method” of index calculation – an approach that is widely recognized as the premier methodology for indexing housing prices – which uses data on properties that have sold at least twice, in order to capture the true appreciated value of each specific sales unit.  

And whereas Case-Shiller strips out quick transactions (likely flips with improvements) Zillow doesn’t. 

Case-Shiller Home Prices

Case-Shiller home price data as of September via St. Louis Fed, chart by Mish.

Zillow Says Home Price Correction is Over

“Nationally, the Zillow model thinks the home price correction is over.”

Oh Wait, No It Isn’t.

What a hoot. And what is Zillow’s track record?

Zillow Reports $880M Loss on Failed Home-Flipping Business

Please recall Zillow Zillow Reports $880M Loss on Failed Home-Flipping Business

Listings giant Zillow lost more than $880 million on its failed home-flipping business in 2021, the company reported late last week.

According to the Wall Street Journal, the otherwise profitable home-listing and real estate advertising company ended up losing nearly $530 million overall, with the bulk of the losses coming from its since shut-down Zillow Offers, which was responsible for the majority of Zillow’s income — $6 billion of the $8.1 billion it generated — but none of its profits.

That article was from February 13, 2022. 

Zillow did not make money flipping real estate in an up market. How pathetic is that? 

And we are supposed to have faith in its pricing model and year-over year estimates. 

Case-Shiller Percent Change Year-Over-Year 

The Case-Shiller chart above is from current, September data.

But September is based on three-month average of repeat sales which are hugely lagging (June, July, August), and it’s now the end of December. 

We will have new data tomorrow for October. I will post new charts with more closeups after the St. Louis Fed posts the data. 

Timely vs Accuracy

Zillow says “It offers a number of advantages over other housing indices, including better timeliness, better coverage of the market and more visibility into changes in small market segments.”

All of the above is at the key expense of accuracy so bad that it could not use its own data to make a profit in a strongly rising market.

Case-Shiller is not timely, but it is the best methodology available. I don’t care how timely Zillow pretends to be if the methodology is flawed.

Curiously, Zillow and Case-Shiller have San Francisco as the weakest market, but it’s difficult to really say how much prices are down year-over-year.

Zillow’s statements that the home price correction is over is truly a joke. 

What we can say is home prices have peaked in every major national market with prices falling rapidly in most. There is no reason to believe a major trend change is at hand.

This post originated on MishTalk.Com.

Thanks for Tuning In!

Please Subscribe to MishTalk Email Alerts.

Subscribers get an email alert of each post as they happen. Read the ones you like and you can unsubscribe at any time.

If you have subscribed and do not get email alerts, please check your spam folder.

Mish

Subscribe to MishTalk Email Alerts.

Subscribers get an email alert of each post as they happen. Read the ones you like and you can unsubscribe at any time.

This post originated on MishTalk.Com

Thanks for Tuning In!

Mish

Subscribe
Notify of
guest

19 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments
MikeC711
MikeC711
1 year ago
As a landlord, I use Zillows rent zestimate more than their cost estimate. That said, people praising the drop (and the cost zestimate on my rentals has been going down) as a great thing for affordability are not considering interest rates. Several of my homes have seen the zestimate (selling) go down 10% … but in that same time the mortgage rates went from 3% to 7%. Affordability only got better for people who can pay cash (who are not as impacted by affordability). I believe a recession is looming and it will be interesting to see what happens with mortgage rates, house costs, rents, etc..
shamrock
shamrock
1 year ago
Zillow used to have an accuracy statistic comparing estimate to actual sales, it was something like 90% of sales are withing +-5% of the z-estimate. Not sure if that is still on their website somewhere.
shamrock
shamrock
1 year ago
Reply to  shamrock
hmk
hmk
1 year ago

For the past 2 years I have been getting updates daily from Zillow and Trulia on new listing of homes for sale new and used. I moved into a new house less than years ago but keep looking at the market just out of curiosity. The Zillow estimate of my new home is 100k more than I paid. Even though it was new you have to add a lot like landscaping, appliances and window treatments etc. Zillow has it at listed at basically my breakeven cost. Anyway I have only been getting updates on the NE suburbs of Detroit at the edge of the county where pavement intersects with dirt road and farmland. Wanted to get as far away from that Shole of Detroit as possible without having to drive to far for work. Prices have stabilized and are not going parabolic like they were when I was looking in the spring of 2021. Any price declines are about 2-5%, homes are not selling as quick and I haven’t seen any bidding wars. One thing for certain new listings are way down and the price of new listings have stabilized. Common sense would lead you to believe that prices will be declining a lot going forward. This is my own anecdotal non scientific data.

RonJ
RonJ
1 year ago
“Case-Shiller is not timely, but it is the best methodology available. I
don’t care how timely Zillow pretends to be if the methodology is
flawed.”
Relative Risk Reduction 95%
Absolute Risk Reduction .86%
The methodology of reporting only efficacy was flawed. In fact, it was a deception. ARR was the important number.
Lies, damned lies, and abuse of statistics.
Ronski
Ronski
1 year ago
I like the joke “Q: What the does ‘A’ in Zillow stand for? A: Accuracy”.
When I look up my own house on Zillow, the Zestimate is way off. How can I can I trust Zillow when it can’t get this one thing right?
LawrenceBird
LawrenceBird
1 year ago
Zillow is unable to give comparable estimates for identical units in an HOA of townhomes that I am very familiar with. If it can’t agree on what the price of a two bedroom, no garage unit is why would anyone trust it for anything?
vanderlyn
vanderlyn
1 year ago
excellent analysis mish. thanks for the effort to explain the scam of zillow numbers. case/shiller is fantastic. shiller not only nailed the top of the dot com bubble, he explained the reason for the timing of inside restricted stocks coming unrestricted in beginning of 2000. he also nailed the top of the r/e bubble in his book, too. as you did, too. for so many reasons. i tip my hat to you, to thank you for this stellar analysis. you have helped me with my r/e “trading” since the last huge bubble circa 2005. i’m keeping powder dry to pick up some nice cap rate properties. happy new years to you and everyone.
Ethangregory
Ethangregory
1 year ago
I am a Realtor in Jacksonville, FL and on the board of directors for our local MLS. Both the statistics and my anecdotal experience indicate that we are in a slowing market. The price appreciation YoY is just a lagging indicator.
Inventories are rapidly increasing, days on the market are increasing, percent of list price received is shrinking and affordability is getting worse.
I don’t see how we can avoid a price reduction in the near future.
The old saying in the stock market is you don’t fight the fed. That is the case in the housing market this cycle as well. Powell has said they are dedicated to fighting inflation and specifically said housing costs. The fed has driven interest rates up creating affordability issues. With interest rates likely to stay higher for the foreseeable future the only way to increase affordability is to lower prices.
The change has already started just the stats are lagging.
worleyeoe
worleyeoe
1 year ago
Reply to  Ethangregory
Totally agree. All too often, gurus are clamoring that we need to build more houses. While that might be true in certain areas of the country, the real issue is affordability. You don’t have 100-125% price increases in 4 years without a significant downside to prices to bring about the needed market correction. A 100% increase in prices is removed with a 50% drop in prices. While I don’t see 50% drop on the horizon, anything less than 25% doesn’t really get us anywhere with affordability, due to inflationary forces keeping 30YFRM higher than buyers want.
What matters is when does the 30YFRM drop towards 5% at which point housing stabilizes. Also, does rent & mortgage relief return if housing drops by 20% or more, putting millions of homes underwater during a recession that’s looming in 2nd the 2nd half of 2023?
Doug78
Doug78
1 year ago
You cannot compare Zillow and Case–Shiller. Even though they both are supposed to give a snapshot of the housing market with predictions that resemblance is superficial because their clienteles and objectives are completely different which makes their methodologies equally different. Zillow is a realter and its objective is to stimulate transactions because that is how they make their money while Case–Shiller provides economic studies of the housing market principally to housing market professionals. Case-Shiller publishes their methodology because they want people to look for flaws while Zillow doesn’t because they absolutely do not want people to find the flaws. Zillow uses the tool to sell where Case-Shiller sells the tool. It is easy to judge which one is more accurate.
Mish
Mish
1 year ago
“I’d take the Zestimate for my house right now, and I am not looking to move…”
That is exactly my experience as well when we went to sell our house. Anecdotes don’t constitute data but what about machine learning models?
Felix_Mish
Felix_Mish
1 year ago
Reply to  Mish
Ha, ha. I’d take the county’s assessment for my place right now! As a mechanism to enforce that property taxes are not up in outer space, it might make sense that a county or state be required to buy a place at, say, 90% of the assessed value if the owner wishes it. Keep the tax guys honest. 🙂
Anyway, it might be instructive to compare Zillow’s estimate with the real numbers at the end of the next month. The closer Zillow is, the more “accurate” they are, right? Ditto the other guys.
That house prices are going down is pretty easy to understand. Most “houses” are sold as a bundle that includes the loan. And since the loan price has gone up, to keep the same bundle price, the house’s unbundled price must go down.
Mish
Mish
1 year ago
Case-Shiller certainly is not timely. But at least they describe their model fully, in 35 pages.
I have no idea what Zillow is even doing.
Mish
Mish
1 year ago
“The Zestimate is generated through an ensemble of machine learning models and incorporates data from a variety of sources including public data, user-generated data and real estate data from direct feeds or multiple listing services.”
Their ensemble of “machine learning models” failed in an up trending market. And we are supposed to have faith in it?
hpie
hpie
1 year ago
Wow, this blog post is spectacularly uninformed. Makes me question the accuracy of the other stuff I read from you. I recommend this reading, which largely validates the hedonic approach vs known biases of the repeat sales method. link to philadelphiafed.org
Mish
Mish
1 year ago
Reply to  hpie
Lovely
Write one post that does not agree with someone and they immediately go into their own echo chamber willing to dismiss everything someone says.
This is despite the fact Zillow, lost money using its own methodology in the biggest bull market in history.
More importantly, this is despite the fact that Zillow’s weighted average methodology of calculating average gains is blatantly stupid. 1 big sale up 15% more than wipes out 10 lesser sales each dropping 10%. That is what they state. I gave the example.
Maybe there is a better approach than Case-Shiller but it sure as hell isn’t Zillow’s “user-generated data and real estate data from direct feeds or multiple listing services” with a weighted average that over-emphasizes more expensive homes.
Sheeesh
Return to your echo chamber where you discount everything anyone says if they disagree with you once.
worleyeoe
worleyeoe
1 year ago
Reply to  hpie
From the paper’s concluding remarks:
“Initial releases of repeat-sales HPIs may
also be subject to upward bias, resulting in downward revisions.”
This paper clearly summarizes it’s initial claim: housing prices are very difficult to measure.
I’m going with Mish on this one. I wouldn’t put $0.02 worth the support of Zillow saying the housing price drop has abated. That’s just crazy! If 2022 was a rollercoaster for housing, then 2023 is going to be when the rollercoaster goes off its tracks.
The Fed & BLS are stacking up a nice string of economic predictive setbacks. IMO, the only downside to the CSI is that it’s lagging 2-3 months.
The sheer amount of profuse spending on the part of the federal government since Biden took office will single-handedly prop up inflation above the Fed’s PCE target by at least a factor of 2 through 2024.
Maximus_Minimus
Maximus_Minimus
1 year ago
Reply to  hpie
Philadelphia FED, nice. Now we know why were in the housing bubble of the millenia that nobody could possibly see coming.

Stay Informed

Subscribe to MishTalk

You will receive all messages from this feed and they will be delivered by email.