Google will one day be able to drive a car for you. It can help you compose your e-mails and finish your thoughts now, automatically. And now, in a spooky view of the future, it can even make phone calls on your behalf.

Google’s display of artificial intelligence and what computers are now capable of this week at its I/O developer’s conference was both the most exciting breakthrough in tech of the year, and the scariest.

Via a program it calls Duplex, it showed how a smartphone app could replicate a human voice, down to the “umms” and “ahhs” in pauses. The bot sounded more real than the person it was calling. (Listen to my Talking Tech podcast here for audio examples.)

While the idea is a practical one — saving us the hassle of making phone calls to set up reservations at restaurants and hair salons — Google’s breakthrough has the potential to conjure for some pretty bad stuff.

Consider that in its demo, Google essentially tricked the person on the other side of the phone call into thinking they were talking to a fellow human being.

Fittinly, while the Internet giant was hailed for the tech breakthrough, it also was criticized for pulling a form of high-tech “con” since at no point did Google in its recorded demo let the human on the other end know they were talking to a robot.

For three days, Google basically ignored growing criticism, presumably because it was worried that announcing “Hello, you are talking to a Google robot” would result in a hangup.

But by late Thursday, Google was forced to bow to pressure and agreed to further develop the software with a “disclosure” of its robotic nature.

That’s good news. But will it go far enough? And really–what was Google thinking? After all the outcry about privacy in the past year and Internet sites that go too far with our data, Google didn’t have the smarts to see this one coming?

Think about what might happen when rogue actors get ahold of this technology and start making realistic, human-sounding calls on their behalf? You know it will happen. What will Google do to prevent that? Or now that the genie is out of the bottle, is it too late, as YouMail’s Alex Quilici suggested to me this week, when he called Google’s tech innovation “a massive Christmas present to robocallers.”

Remember, it was just two years ago that our elections disrupted by Russian tech experts in state-sponsored attacks via Facebook. (Read below for more on just how bad Russia played Facebook, in our news roundup.)

The upshot of Duplex, should it work as advertised, is that Google has just put one foot forward and two steps backward for our lives, says Jim Boerkoel, a professor at Harvey Mudd College in Claremont, California.

“If Google drives us to work, then we just added another hour of work to our day, because we’ll be expected to work during the commute,” he says. “If we don’t have time to make appointments, and now the robot can do it for us, we’ll use the time we just freed up for more work. This goes both ways. Because we have more time, we’ll be expected to do more.”

But for all the heavy talk about a coming future where tech runs amuck, let’s get real for a moment. Artificial intelligence is creepy, but it could also be great, making our lives better and saving us time. And it’s not about to take over our lives. Not yet.

Google can’t eat for us, do our exercise or take over most of our jobs. We still need to get dressed in the morning —no computer will do that for us. But who knows, give the company time, it could get there, right?

Google plans to begin its Duplex test in the summer.