The digital breaches of the past week have been alarming. Equally alarming has been the amount of speculation and lack of real information. One theory I’ve seen posited is that this is the typical Q4 dash to deliver. Another is that this is all AI slop code coming back to bite us.
Both seem plausible to me, and they are in no way mutually exclusive, but what feels so odd is the number of events in the same timeframe and across so many organizations.
Security has always been a challenge with technology, long before software or the ‘cloud’ entered the mix. Antics to spy on typewriters truly show the creativity inherent in hacking and the never-ending spy vs. spy games being played. (Nerd out here).
But what’s different is the reach of the impact now. Robbing one bank is very different than accessing data for more than 100 million Capital One customers (International Business Times).
I remember eons ago working on a pitch for Samsung. One of the categories we were asked to consider was home appliances. We sat in a room for at least an hour trying to find value in a ‘connected refrigerator.’ It was, in corporate speak, challenging. And that was not even considering the downsides we now know.
I was also working at Capital One when it was decided that the chief priority was to be the first bank fully in the cloud. This was a company that actually had a very strong product practice. We would frequently talk to customers and ‘human centered’ product thinking was zealously preached.
Yet, the call was made to brush our entire roadmap built around those customer needs aside in pursuit of saying ‘FIRST’ in some non-existent thread on financial cloud dominance. Ultimately, the cloud migration was achieved and shortly after came a breach by a former employee at Amazon (Search It). Because there is no such thing as perfectly secure software. Someone, somewhere, can fuck things up.
And the reality is as much as we plan and test and eval, the world is so complex and ever-changing, it’s impossible to cover every scenario. Regulators faulted the bank for rushing dangerous work (International Business Times), but you cannot fault Capital One for not anticipating one Ms. Paige Thompson as part of their modernization efforts.
It would be equally hard to blame the person who designed Fiona’s enclosure for not considering the scenario in which a small child threw their hat in the air, which was then caught by a ceiling fan, and somehow entered a one-foot opening. This is a Final Destination level sequence of events and as the movie teaches us in each release, you cannot escape YOUR DESTINY.
But the thing that we can do better is pause to truly evaluate the risk and reward. What could be the impact? Is it worth it? Maybe we decide we cannot make that call for users, in which case, our job is to clearly communicate the risks and rewards so they can make the best decisions for themselves, not bury it in your “Online User Agreement”.
I also feel strongly that security is not a premium service. No one should be in a position where they can’t afford to use your product in the safest way possible. This email from yesterday, trying to sell me safety at a mere $19.99 per month, is incredibly gross timing by American Express.
Recent developments put responsibility for AI mischief in the hands of the consumer (TechCrunch). If a customer connects their account to an AI service, they’re on their own. This may seem fair, but it’s counter to the precedent set with other digital services like payments. Why is a customer responsible for sharing account information with AI, but not Shop Pay?
I still don’t know how an internet-enabled refrigerator would meaningfully impact my life. And I can attest that I never heard a single customer say, “I wish your digital products were hosted on a different server immediately.”
Similarly, I don’t think any of us are saying “Man, I wish this had AI.”
We’ve developed a zealousness in tech, where every new capability becomes a rush to adoption or brace yourself for irrelevancy. “You can’t afford to be left behind. Everybody is doing it. You need to start having an affair with Jensen Huang. Or your competitors will.” (Bsky)
Each time it’s framed as anticipating consumer demand. But the actual demand is for what the technology provides, not the technology itself. Users don’t want AI, but they do want assistance.
If a child’s knit hat could help organize my life the way Instinct or Claude Duet is offering to do, I’d want a child’s knit hat. Now, if that child’s knit hat has the ability to lodge itself into my small intestines and end my life, I might reconsider.
Fiona, RIP queen. I hope that hat was truly delicious.