In a controversial move, Craigslist, the well-known national online classified ad and discussion site, has asked Oodle, a search engine for online classified ads, to stop displaying its listings.
Both sites offer consumers a way to look at real estate for sale or rent, along with a number of other online ads for such things as vehicle and jobs.
“We got an e-mail from Jim Buckmaster, Craiglist’s CEO, asking us not to use their listings any more,” said Oodle CEO Craig Donato. Donato e-mailed back asking why and requesting an opportunity to talk to Buckmaster, but has not gotten an answer, Donato said.
“I’m a huge fan of Craigslist and I think it’s a great service. We really thought we were helping them out,” said Donato. Donato revealed the news Oct. 13 on his blog on the Oodle site. His company has complied with Craigslist’s request.
“Craigslist gets so many hits from Oodle’s robots spidering the site to gather listings that it puts a strain on Craiglist’s server and slows the system down,” said Jim Buckmaster, Craigslist’s CEO. Buckmaster said Oodle is also subscribing to all of Craigslist’s categories, which include housing, cars and jobs, which also puts a strain on the system, costing his organization money.
“If they (Oodle) had consulted our terms of use on our site, they would have seen that this type of activity is outside our guidelines,” said Buckmaster. “If you are going to assume it’s okay and just start doing it, you need to provide an easy way to opt out,” which Oodle has not done.
Craigslist “ended up blocking their IP addresses” in an attempt to keep Oodle from spidering Craigslist, Buckmaster said. After Oodle circumvented the blocks several times, Buckmaster sent the e-mail asking Oodle not to display the listings, he said.
Launched in April of this year, Oodle is a free search engine similar to Google. A consumer looking to buy, say, a one-bedroom condominium in the San Francisco Bay Area, can type in that information and get search results from sources such as RE InfoLink’s MLSlistings. To read the ads, the consumer must click through to the original site where the ads appear.
Oodle finds online classified listings in 22 metropolitan areas. The Oodle search engine displays results from the major suburban papers, major metropolitan dailies, eBay and online verticals such as Cars.com and Homescape – and, until recently, Craigslist.
Ironically, mainstream newspapers that make money from classified ads, such as the Los Angeles Times, have not protested at Oodle scraping their ads.
Donato emphasized that when people read listings on his site, they must go back to the original site to get the information.
“There are actually sites that don’t usually allow search engines to crawl them that have made allowances so we can. The Orange County Register, for example,” Donato said.
“We deliver folks to their (The Register’s) site which makes their advertisers happy,” Donato said.
In response to Donato’s comment, Buckmaster said, “We don’t get any traffic from them (Oodle). The traffic we get from them is from their robots, not users. They don’t have any users.” Craigslist gets 6 million classified ads each month, Buckmaster said.
According to Buckmaster, “As a result of their activity, fewer people are seeing the listings than otherwise. Because by competing with our users for access to our listings, our users have incrementally a more difficult time using pages and there aren’t any users seeing our listings coming from their site.”
It doesn’t seem as though Craigslist is any slower than it ever has been since April when Oodle launched. Asked about this, Buckmaster said, “Even if it was one-tenth of one percent, do you think you could tell if it was one-tenth of one percent slower? That would be enough to offset the amount of traffic we’re getting.”
The move has stirred comment on various blogs and from industry observers.
“When I wrote about Oodle a while back, I wondered if the sources of its listings would ever get cranky,” blogged John Battelle on his Searchblog October 13. Battelle, a media maven who co-founded Wired Magazine, offers commentary on his Searchblog and recently published “The Search,” a book about Google and other search engines.
“I never thought the first grump would be craigslist – but indeed, the company has asked Oodle to stop scraping its listings. It feels counter to the vibe craigslist has always had,” Battelle wrote in his blog.
Greg Sterling of the Kelsey Group, an expert in the local search field, said Craigslist’s request might indicate that the national classified ad site sees Oodle as a threat.
“It’s a bit of a surprise because of the seeming friendliness and openness of the Craigslist philosophy. But on another level, I believe they wanted to protect their status as a preferred destination site for people looking for certain kinds of information such as jobs or local services,” said Sterling, who noted that he has not talked with Craig Newmark, Craig’s List’s founder, or Jim Buckmaster.
In other words, the concern is that instead of going to Craig’s List to look for things, consumers might decide to start with Oodle instead.
“I can speculate that maybe the comprehensiveness of the Oodle site made them feel like they were going to lose traffic over time or the value of their brand because all their listings were brought into the Oodle data set, cars, jobs, real estate,” Sterling said, though Buckmaster denies that this is the case.
Sterling noted that Indeed, a meta-search engine for jobs, includes Craigslist jobs information.
“This would be inconsistent with what Craigslist did with regard to Oodle,” Sterling said, except that possibly the policy might be to allow Indeed to do it because it focuses only on jobs, and is not spanning the full spectrum of Craiglist’s offerings.
Buckmaster said that Indeed “may be grabbing certainly less than a tenth of the number of listings” that Oodle is grabbing. ” He said, “It’s not so taxing on us because they’re not using as much and they’re not calling attention to themselves by using their name in our marketing and press releases.” Nonetheless, Buckmaster said, it’s possible that Indeed might get an e-mail asking them to stop.
“In a way, they’re doing this because they can and implied in this is the recognition that what Oodle offers is a better user experience,” Sterling commented. On Craigslist, ads are organized by subject, such as “Real estate for sale,” by area and the time they are posted.
For example, on Oodle, it is possible to search for “San Francisco condominiums,” so readers wouldn’t have to look at every type of house offered for sale in a given week, as is necessary on Craigslist.
Buckmaster pointed out that Craigslist also has a search engine.
“It’s a dilemma for everyone dealing with online distribution on search engines or aggregator sites,” Sterling said of the conflict. “Everybody wants the traffic, the promotion, but nobody wants that at the expense of their own user acquisition,” Sterling said.
***
What’s your opinion? Send your Letter to the Editor to janis@inman.com; (510) 658-9252, ext. 140.
