This report is available exclusively to subscribers of Inman Intel, a data and research arm of Inman offering deep insights and market intelligence on the business of residential real estate and proptech. Subscribe today.
The newest artificial intelligence models like ChatGPT have taken a huge step forward in generating human-sounding language, but have yet to change much about the way that real estate brokerages do business.
Their eventual impact on this heavily regulated industry might come down to two questions of trust: Should agents, brokers and clients completely trust these AI models right now? And can they ever trust them in the future?
The answer to that first question for Anywhere Chief Product Officer Tony Kueh is “no.” And the answer to the second question will only be resolved in time as the creators improve on the factual accuracy of the models, he said.
TAKE INMAN’S INAUGURAL SURVEY ON AGENT COMMISSIONS
Kueh met recently with Intel by video call to discuss some of the risks posed by new generative AI models, including their tendency to make up false facts in a poorly understood process that AI technologists refer to as “hallucination.” He also detailed some of the tantalizing opportunities for real estate, if this major obstacle is ever resolved.
The conversation below has been edited for length and clarity.
Intel: The arrival of these sophisticated large-language models has had a lot of brokers and agents sitting up and paying attention and thinking about how they might employ AI in their daily business.
I’m curious from your standpoint, what are the big AI-related topics you’re discussing right now on a weekly basis, both in team meetings and maybe even with brokers?
Kueh: From our perspective, the AI and machine learning has been used for quite a while. We use this to run predictive modeling. There are tools that we use internally for agent recruiting. We use this to predict things like ebbs and flows of the business so that we can apply resources appropriately. Those mechanisms have been in place for a while.
The [newer] generative AI is about generating things — generating content. And the way we look at the question is: Where are we generating content? Now, the simple ones are like property descriptions. But there are opportunities in many of our consumer-engagement points — whether it be email communications in different forms, marketing collateral — where things that normally would have taken at least a few hours or a few days to get through, now is a matter of minutes, sometimes even seconds.
So it really increases the productivity. Essentially when someone has to put fingers to the keyboard and generate content, generative AI tools like ChatGPT become extremely powerful.
Some of the more advanced-use cases then become the more experimental, and we still need to prove it out.
Image generation, for example — lots of people are tinkering with, ‘Hey, what if I can take a picture of an empty room and place furniture into it?’ Certainly you can imagine that use-case being well used or beneficial.
But the problem is that the way we take photos today, without the true depth perception, it’s very difficult to get accurate, 3-D modeling of furniture into that photo. And so those are sort of the core of that last 10 percent of perfection. You certainly can’t put a picture that’s got the furniture running into a wall for a luxury listing. The expectations are going to be significantly higher.
So those are things that we are going to continue to evolve. And we’re going to work both internally and with our technology partners to get to a place where we feel good about the quality of that output, where we can use that as part of our day-to-day process.
Are there any applications of some of these new AI products that Anywhere has already embraced or are actually in use by your brokers and agents?
From a generative AI perspective, no.
From a predictive-modeling perspective, absolutely. Our brokers today have access to tools that do prospecting, and that’s how they run their franchises and run their brokerages.
From a generative AI perspective, we do allow and we have seen agents themselves — the ones who are a little more tech-savvy — use that to generate emails or property descriptions and things like that.
The ‘hallucination’ problem is one where we need to figure out the right balance. Because this is a regulated industry. There are rules around what we can say and what we cannot say and what our agents can and cannot say.
To blindly generate something knowing that there’s a risk of hallucination on the content that’s created is an increased risk that we have. Because at some point, is it the AI’s fault or is it the person that generated the content? And if we endorse that generation, where is that liability and risk?
So those are the systems and controls that — as one of the leaders in the industry — we believe we have to solve.
I know that once in a while we get these emails that [say] one little boutique [brokerage] over here, they’re using that. Yes: The risk exposure for them is significantly less compared to us, being the largest real estate company in the United States.
So we are taking a very concerted effort to create a mechanism and a system in which this is going to be highly scalable, but also adhere to all the legalities and the compliance concerns that we have.
I’ve played with some of these language models, including ChatGPT, particularly for troubleshooting code. It has a remarkable ability to understand my questions, which are sometimes very technical, and return plausible-sounding answers.
But I’ve also run into dozens of cases where facts were fabricated with confidence by the AI, which is that known issue you referred to called, ‘hallucination.’
What discussions are you guys having right now to try to account for this hallucinated info and protect the transaction from false information?
Man, I think the whole industry is trying to figure that one out. It should provide a warning when guys like [OpenAI CEO] Sam Altman says, ‘We have no idea why it’s hallucinating or how it hallucinates.’ The hallucination patterns also vary from time to time, even around the same topics.
I internally joke LLM is kind of like the most sophisticated parrot. It just learns what you say and repeats it back. It has certain triggers, and it says, ‘When that word comes up, I say this.’ It may sound like the parrot knows what it’s talking about, but it really doesn’t. And that’s really the hallucination when that occurs.
There’s a couple of techniques that I’ve seen that people [put] in play. No. 1 is this notion of prompt engineering, which is that if you give it enough context and narrow it down enough, the probability — just by design — of hallucination is much, much lower. Because you’ve sort of narrowed the scope down to a place where you’re essentially saying, ‘I believe the right answer is somewhere within this circle; please give me an answer within that circle.’ And so the wrong answers will be fairly probabilistically lowered and filtered out. So that’s one.
The second thing is at some point the technology stack will have to allow for some real-time learning training, machine learning. The LLMs are pre-trained, and it’s a lot of computing power to train and re-train. And the way that LLM models work is that whenever you train, it’s not like you can make a small adjustment here or there. The baseline models, like Google, Microsoft, ChatGPT — those models will continue to get trained, retrained, and they will get better.
But some of the hallucination, candidly, could come from the fact that the training source is the internet. And so unfortunately, all the good content on the internet is being used to train; but along with it, some of the garbage content is used to train it too. So maybe that’s where the hallucination is coming from.
I think the language model will enhance. I think there will probably be some kind of enhanced layer that allows for finer, more granular tuning. The tools available to us from an Anywhere perspective would be — prompt engineering is the No. 1 thing — and then some kind of manual auditing.
Even if you say, ‘I need a manual step, where a human has to be involved for compliance and sanity check,’ it’s still significantly faster than if we had to do the whole thing the old way without AI.
If these models improve enough in the coming months and years, and they improve in accuracy, what might that open up for the industry? Like, once you can rely on it, what are the next-level applications that might be particularly exciting?
The thing is, I think in general right now in the world — it’s not just real estate — we do have a question around content authenticity, content accuracy.
Unfortunately with AI, we’re not getting closer to the source. We’re actually getting away from the source, because it’s generated. It’s kind of like taking everything that it’s been trained with and compiling an answer. I appreciate that there are people working on referencing the source, and I think that’s really important. I also think that being able to authenticate the source and make sure that that is indeed fact and truth is really important.
Ultimately it comes down to trust. I think this industry, more than anything else, is around trust. I think once you establish trust, then you have the opportunity to create solutions that really help problem-solve.
Everyone’s looking for agents because they’re looking for a trusted adviser. But sometimes the problem they’re solving doesn’t necessarily translate to a real estate transaction.
I can imagine a world where once you have a mechanism to create a trustworthy, minimal- or no-hallucination type of AI service, the clients then would have access to that to really help them problem-solve. The trust to the extent where [a client might say], ‘Here’s my W-2, here’s my tax statement, here’s my bank account: Can you give me the best way to structure my remodel so I get the best tax benefit?’
Just to do what I said there, you can imagine attorneys jumping up and down and saying, ‘Oh my God; that’s got lots of red flags there.’ And it’s a hard, hard problem. Today that requires not just humans, but certified people that have the right credentials to provide that type of advice. Imagine if that was now scalable in a way that [it] could be offered as part of a real estate brokerage service. I’m talking about this as years down the road of course. I think that it will be an evolution.
But that would be the dream of being able to offer that level of sophistication in an automated way. It would be extremely powerful.
Yeah, it’s exciting stuff to think about. And your point is well taken that a lot of this stuff feels like it could be a long way off, or a few years off at least, to work out some of the issues. Is there anything else that you think we or our readers should be keeping an eye on in this space?
There’s a whole thing going on in Hollywood right now with the labor union and stuff like that. I think there are a lot of people either embracing AI, or they’re fearful of it because of what this could mean [for] jobs.
Will AI replace agents? I’ll address that head-on. One of the things I would say is, technology will continue to evolve. There was a time when we had to go to a store to rent a movie. There was a time when thinking about getting into the car of a stranger would be insane. Now with Uber, we do it all the time. So once you create that trust, it will change behavior.
Now, I don’t think that when we’re talking about the largest financial transaction someone will make in their life. It is hard to imagine that will be completely done behind a computer.
For the foreseeable future though, I would say that AI is more like Tony Stark’s Iron Man suit. What we’re really looking for is a way to enhance the power and capability and get to a level of consistency of service for our household brands that are under the Anywhere umbrella to really empower them to deliver the best possible service.
And the machines will have hallucinations; the machines will have errors. [Iron Man’s] JARVIS cannot win a war on its own. It really needs the capability of a human mind and the empathy.
That’s a completely different conversation: Can machines have empathy? That’s what we need. That’s what our agents do today. We look at the sophisticated agents, they’re the ones who can really step into the shoes of their clients and the family and guide them to the solution.
It’s going to take a while before AI can have that level of simulated empathy. And even then, at best, it will only be simulated, because it is artificial.
