Editor’s note: Inman News, in citing a Zillow corporate blog announcement at Zillow.com, had originally reported that ComScore’s data showed that Zillow had pulled ahead of Realtor.com in monthly visitors, though according to this article Zillow had based its report on incompatible statistics.
Zillow.com jumped the gun in claiming to have surpassed Realtor.com in November as the most visited real estate Web site on the Internet, according to the purported source of the claim, Web metrics provider comScore Inc.
Zillow’s claim that it had surpassed Realtor.com was based on a non-public report issued by comScore, which recently began using a new methodology for tallying visits to Zillow.
ComScore had not yet implemented the new methodology for other top real estate Web sites in November, according to a copy of the report Zillow provided to Inman News.
In a recent policy bulletin, comScore advised clients that in comparing themselves to their competitors, companies should use a separate set of reports employing only the previous "panel based" methodology across all sites. Otherwise, sites still making the switch to the new "hybrid methodology" would be "unfairly penalized in ranking during the transition."
The bulletin warned that "changes in rank that are a result of mixed methodology comparisons are not allowed."
In post on the company blog Thursday, Amy Bohutinsky, Zillow’s vice president of communications, cited a comScore report that showed Zillow.com tallying 5.231 million "unique visitors" during November, compared to Realtor.com’s 5.223 million.
Inman News published a story repeating the claim Thursday evening, and retracted the story Friday morning after Realtor.com operator Move Inc. questioned its accuracy.
A comScore spokeswoman said Friday that an "apples to apples" comparison using the "panel-based" methodology to evaluate traffic across all top real estate sites showed Zillow with 3.013 million unique visitors in November, behind Realtor.com, Yahoo Real Estate, and AOL Real Estate.
"Zillow.com is now being measured with our hybrid methodology which caused its figures to grow largely from (October to November), such that a significant portion of its growth was attributable to this change and not necessarily due to organic growth at the site," said comScore spokeswoman Sarah Radwanick in an e-mail. …CONTINUED
Reston, Va.-based comScore announced its new Media Metrix 360 "panel-centric hybrid" methodology in May. The new technique was designed to blend measurements comScore gets from a panel of 2 million Internet users worldwide with traffic metrics from clients’ own Internet server logs.
Bohutinsky said that in reporting its Web site traffic, Zillow is careful to cite data from multiple sources, and that there was no intent to be "funny with the numbers."
She said Zillow never received the comScore bulletin, "Best Practices for Using Panel-Centric Hybrid Data in Media Metrix," and was following terms of use posted on comScore’s Web site.
"Zillow reported data as it was reported by comScore this week, and followed the terms of use we were given," Bohutinsky said.
In her blog post, Bohutinsky also noted that another online metrics company, Hitwise, ranked Zillow second behind Realtor.com in November. Zillow’s own internal numbers show traffic up more than 60 percent from last year, to 8.3 million visitors a month, she said.
"Our blog post is about the large amount of traffic Zillow experienced during the month of November, citing internal data and two external audience measurement services," Bohutinsky said in an e-mail.
"The thesis of the post is that Zillow is growing fast and at more than 8 million visitors a month, has hit a number of significant milestones in less than four years of business. It would be unfortunate if this broader message were tarnished by comScore’s restatement of their own reporting methodology."
Radwanick declined to comment on whether Zillow, in publishing results that mixed panel based data with Zillow.com’s hybrid methodology score, violated comScore’s best practices policy.
When Inman News first contacted comScore about Zillow’s claim, comScore initially provided a table that also mixed "panel only" data for top real estate sites with "hybrid methodology" numbers for Zillow.com — the very practice it advises clients not to engage in.
After Radwanick was asked if those numbers provided an "apples to apples" comparison, she provided another table listing only the "panel based" numbers, which showed Zillow with 3 million unique visitors in November instead of 5.2 million. The unique visitor count is supposed to count each person who visits a site once, no matter how many times they return to a site in a given time period. …CONTINUED
During a six-month transition period ending May 1, comScore is producing both panel only and hybrid data sets in parallel.
"During this time, sites that transition to hybrid may not publish changes in either level or trend that are based on a mixed methodology and without specifically citing them as hybrid," comScore said in its best practices bulletin.
Julie Reynolds, a spokeswoman for Move, said the episode demonstrates that "accurate and fresh information is critical."
"What’s important to us as an industry is that we use services and methodologies that result in accurate and consistent information," Reynolds said. "When inconsistencies like this occur, we have to look and determine what has occurred."
ComScore Chief Executive Officer Magid Abraham and Executive Vice President Jack Flanagan declined to comment.
In a blog post in October, "The Dawn of Hybrid Audience Measurement," Abraham laid out a case for the new methodology. Abraham said that while server logs represent a "census of usage," including page views and sessions, it’s difficult to determine the true number of unique users from them.
That’s because many users on the same computer may be represented by the same "cookie," an identifying code that can be fed and retrieved from a visitor’s browser, and a single user using multiple browsers on can be represented by multiple cookies.
Panels of known users allow Web metrics companies to track individuals across sites and time without depending on cookies, Abraham said, but may miss Web surfing that goes on at work because companies will prevent panel metering software from being downloaded onto employees’ computers, he said.
By using both server logs and panels, comScore can match page view counts and better estimate the number of unique visitors to a site or a group of sites, Abraham said.
***
What’s your opinion? Leave your comments below or send a letter to the editor.
