The Real AI Threat?

Thu Dec 10 00:34:33 UTC 2020

Miles,

My point is that there is no reason to wrap this engineering problem in the mythology of AI. There are no new problems here. The only way automated threat detection & response combined with automatic resource allocation allocation will lead to a disaster is if an incompetent (sometimes called “agile” :) software engineer fails to design in appropriate safeguards.

The problem isn’t the technology. It’s the lack of competent engineering. You can’t equate  "AI fighting for survival" with "bad engineering”, anymore than you can equate time travel with bad engineering.

Incidentally, Alan Turing’s test, which posits that a computer can be said to possess human intelligence if it can fool a human with its responses, was debunked long ago by an undergraduate CS student, who innocently asked, at an AI conference attended by CS luminaries “Does it follow, then, that if a computer can fool a dog with its responses, that it possesses dog-level intelligence?” ROTFL!

So don’t be fooled by Siri and Google voice response. There is no intellect there, only pattern matching. Which we’ve been doing with machines since the Jacquard Loom.

 -mel

On Dec 9, 2020, at 2:32 PM, Miles Fidelman <mfidelman at meetinghouse.net<mailto:mfidelman at meetinghouse.net>> wrote:

Mel Beckman wrote:

Miles,

You realize that “AI” as general artificial intelligence is science fiction, right? There is no general AI, and even ML is not actually learning in the sense that humans or animals learn. “Neural networks”, likewise, have nothing to do at all with the way biological neurons work in cognition (which science doesn’t understand). That’s all mythology, amplified by science fiction and TV fantasies like Star Trek’s character “Data”. It’s just anthropomorphizing technology.

Well, duh.  I'm old enough to remember the old aphorism "it's AI until we solve it, then it's engineering."

We create unnecessary risk when we anthropomorphize technology. The truth is, any kind of automation incurs risk. There is nothing related to intelligence, AI or otherwise. It’s all just automation to varying degrees. ML, for example, simply builds data structures based on prior input, and uses those structures to guide future actions. But that’s not general behavior — it all has to be purpose-designed for specific tasks.

We create unnecessary risk when we deploy technology with positive feedback loops.
Machine learning (not AI) + automated threat detection & response + automatic resource allocation = a recipe for disaster.
Call it "AI fighting for survival" or bad engineering - either way, it will kill us a lot sooner than any of the more fictional varieties of AI.

Since the academics’  promised general intelligence of AI never materialized, they had to dumb-down their terminology, and came up with “narrow AI”. Or “not AI”, as I prefer to say. But narrow AI is mathematically indistinguishable from any other kind of automation, and it has nothing whatsoever to do with intelligence, which science doesn’t remotely yet understand. It’s all automation, all the time.

Then again, Google's "AI" has gotten awfully good at answering relatively free-form questions.  And allowing for really, really dumb people, Siri comes pretty close to passing the classic Turing Test.

All automated systems require safeguards. If you don’t put safeguards in, things blow up: rockets on launchpads, guns on ships, Ansible on steroids. When things blow up, it’s never because systems unilaterally exploited general intelligence to “hook up” and become self-smarted. It’s because you were stupid.

Yup.  And folks are looking in the wrong place for things to protect against.

Miles

For a nice, rational look at why general AI is fiction, and what “narrow AI”, such as ML, can actually do, get Meredith Broussard’s excellent book "Artificial Unintelligence - How computers misunderstand the world".

https://www.amazon.com/Artificial-Unintelligence-Computers-Misunderstand-World/dp/026253701X

Or if you prefer a video summary, she has a quick talk on YouTube, "ERROR – The Art of Imperfection Conference: The Fragile”:

https://www.youtube.com/watch?v=OuDFhSUwOAQ

At 2:20 into the video, she puts the kibosh on the mythology of general AI.

 -mel

On Dec 9, 2020, at 11:07 AM, Miles Fidelman <mfidelman at meetinghouse.net><mailto:mfidelman at meetinghouse.net> wrote:

Hi Folks,
It occurs to me that network & systems admins are the the folks who really have to worry about AI threats.

After watching yet another AI takes over the world show - you know, the
same general theme, AI wipes out humans to preserve its existence - it
occurred to me:

Perhaps the real AI threat is "self-healing systems" gone wild. Consider:

- automated system management
- automated load management
- automated resource management - spin up more instances of <whatever>
as necessary
- automated threat detection & response
- automated vulnerability analysis & response

Put them together, and the nightmare scenario is:
- machine learning algorithm detects need for more resources
- machine learning algorithm makes use of vulnerability analysis library
to find other systems with resources to spare, and starts attaching
those resources
- unbounded demand for more resources

Kind of what spambots have done to the global email system.

"For Homo Sapiens, the telephone bell had tolled."
(Dial F for Frankenstein, Arthur C. Clarke)

I think I need to start putting whisky in my morning coffee.  And maybe not thinking
about NOT replacing third shift with AI tools.

Miles Fidelman
--
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why.  ... unknown

--
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why.  ... unknown

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20201210/a6fa2cb6/attachment.html>