In February, Zillow seemed so sure of its AI home valuation capabilities that it introduced a new feature: for some houses, the so-called “Zestimate” would also serve as an initial cash offer to buy the property.
A firm executive hailed the change as “an exciting advancement,” and its goal was to make things easier for homeowners who were thinking about selling to Zillow for its home-flipping business. Zillow advertised this feature as a means to facilitate house sales with minimal human contact during the pandemic. But now, only eight months later, the firm is completely winding down Zillow Offers.
Zillow suffered a crushing loss with the announcement of the decision last week. Due to recent purchases of homes at greater prices than the firm believes they can sell them for, the real estate listing company recorded a $304 million write-down of its inventory in the third quarter. The stock price dropped, and now the corporation is planning to lay off 2,000 workers—or 25% of the workforce.
The consequences of this commercial endeavor highlight more than just the difficulties of purchasing and selling houses for profit. It also shows how challenging it is to utilize AI to assist with costly, real-world decisions, especially in a dynamic market where prices might be based on gut feelings as much as objective evidence, and where changes can happen months or even weeks in advance. Zillow Offers was shuttered because to “unpredictability in forecasting home prices” that “far exceeds” what the business had anticipated, according to Zillow CEO and cofounder Rich Barton.
Zillow and other realty firms use a business model called “iBuyer” wherein they buy houses directly from sellers, do some little repairs, and then put them back on the market. Zillow uses a process called the “Zestimate” to determine if a house is worth buying. This estimate is based on a lot of data, including tax and property records, information provided by the homeowner (such as any additions to the house), and pictures of the house. Zillow also uses machine learning to help with this estimate. Comparative estimates using comparable data are available on competing platforms like Redfin.
The Zillow Offers page explains to homeowners that they utilize “the Zestimate, facts you provided, and comparable homes nearby to calculate an estimated sale price” when estimating the value of a home to sell to potential buyers. (As of this writing, the page states that the firm is no longer making new offers on houses and is instead “winding down” the business.) According to the page, after that estimate, Zillow visits the home to assess it in person, calculates the cost needed for repairs before reselling it, and finally makes an offer. From April 2018 to September 2021, Zillow bought 27,000 homes, but only sold about 17,000 of them, according to its quarterly results. This is despite the fact that Zillow Offers has resulted in tens of thousands of home purchases.
The request for an interview with Zillow’s VP of analytics, Krishna Rao, was refused. The business utilized the Zestimate for Zillow Offers “the same way we encourage the public to use it: as a starting point,” Zillow spokesperson Viet Shelton stated in an interview with Zobuz.
The capacity to reliably predict inventory prices three to six months out was a difficulty for Zillow Offers, according to Shelton. This was especially true in a market where property values were changing at a faster rate than ever before.
In fact, real estate markets have undergone extremely unanticipated changes since Zillow got involved in the home-flipping industry in 2018. A brief halt in the property market due to the epidemic and subsequent mismatch between supply and demand led to an all-time high in home prices. Because of this, Zillow may have been hesitant to incorporate the Zestimate into the Zillow Offers process in over 20 cities. The Zestimate is not an appraisal, but rather a “computer-generated estimate of the value of the home today, given the available data,” as Zillow explains.
When determining a reasonable asking price for a property, artificial intelligence can examine a plethora of data points, such as recent sales in the region, the number of individuals actively searching for a home in that particular neighborhood, and more, in a fraction of the time it would take a human. Mike DelPrete, a real estate technology strategist and scholar-in-residence at the University of Colorado Boulder, said that real estate agents can “pick out one critical factor of the valuation that just doesn’t exist as ones and zeroes in any database” when they look at a house.
Important to Zillow
When Zillow initially launched its website in 2006, the Zestimate was an integral aspect of the brand. Millions of Zillow house listings prominently feature the term; the company trademarks the term; and the IPO documentation from 2011 mentions the term 61 times.
Zestimates for over 70 million US homes are generated using over 500,000 individual valuation models that are built atop 3.2 terabytes of data. This process occurs three times a week, according to the company’s 2011 securities filing. Zestimates for over 100 million US residences are published by the company more than a decade later.
Zillow prominently displays the Zestimate in every listing on their website or app, regardless of whether the home is for sale or not. A red dot will be displayed next to the words “House for sale” if the house is currently listed for sale, and the Zestimate, if available, will be presented on the same line for that home.
Even though the firm clarifies that Zestimate is not an appraisal, many have questioned the feature’s veracity. Take the 2017 instance of homeowners suing it as an example. (The court rejected that case.)
Zillow has invested a lot of time and effort into making the Zestimate better over the years. They even held a data science competition that lasted several years in an effort to refine the algorithm that powers it. In early 2019, the firm handed out a $1 million prize to a trio.
According to Shelton, the current median error rate for the Zestimate for properties on the market is 1.9%. This means that for half of the homes on the market, Zillow’s predictions are within 1.9% of the actual selling price. According to Shelton, the margin of error for residences that are not for sale is 6.9%. Even a small discrepancy of 1.9% on a house valued at $500,000 would result in a loss of roughly $10,000 when applied across a large number of residences in various areas across the US.
Something more than a science
Creating a model online, which is typically fairly accurate, is one thing. Using that model in a real-world setting to make expensive bets on a large scale is a whole different ballgame, according to Nima Shahbazi, CEO of Mindle.AI (which assists enterprises in using AI for prediction) and a part of the winning Zestimate algorithm team. He gave the example of a house Zillow bought that had hidden concerns, such a foundation crack that was undetected, and argued that the Zestimate couldn’t have predicted those difficulties.
“There are numerous potential points of failure between developing a good model and putting it into production,” he said.
Zillow relied on the Zestimate to guide its home-buying decisions, with the goal of eventually turning a profit. However, USC assistant professor of marketing Nikhil Malik remarked that algorithms excel at producing very specific, very short-term predictions, like stock price predictions made one second in advance. Malik, an expert in algorithmic pricing who has examined the Zestimate extensively, claims that there is insufficient evidence for algorithms to understand prolonged booms and busts.
According to DelPrete, there are numerous intangibles when it comes to valuing a property, such as the significance of returning to a familiar neighborhood or being in close proximity to family. These can differ from one individual to another, which further complicates the task of automating a house appraisal.
DelPrete stated, “It’s a good tool for what it is” regarding the Zestimate, but he cautioned against relying on it as a reliable indicator of future home prices. In his opinion, it is more of a “almost a toy” that is designed to pique your interest when you search for your own or your neighbor’s house online.
“The ability to accurately value homes, not just in the present but also three to six months from now, is crucial if you intend to participate in iBuying and make thousands of offers daily,” he said. “And it’s both an art and a science too.”
Veronica Baxter is a writer, blogger, and legal assistant operating out of the greater Philadelphia area.