That is not entirely true. It does happen that a cpu computes something wrong but it is extremely unlikely. Computers that are handling very delicate matters often have two cpu’s perform the same calculation in order to minimize the chance of failure. However, I don’t think this would affect an AI very much since an AI must not be dependant on a few, discrete calculations but a huge amount of computing adding up to a great complexity where these few faults probably won’t matter much.

This whole discussion is probably about the biggest question of them all. What is Life?

Most of us nerds have probably read Isaac Asimovs robotic novels and the three Laws that governs all robots. Our own fear is the greatest threat to our survival now that we do possess this huge firepower. If we can beat the fear, I think it would be possible to have a society where robots and humans live side by side. But I don’t think robots will be given the same rights as we do. The short answer is because we believe we are better than everyone else. The long anser is related to the Big question about Life.

When a software program gives incorrect results, it was either given incorrect data, or it was programmed incorrectly, but in either case, it does exactly what it was programmed to do. If the software gives you a stack buffer overflow, it was programmed to do that in response to particular data. The programmer may have made a mistake in programming it to lead to an error, but the software nevertheless continues mechanically towards the error, with no consciousness.

Philip, I started reading the novel “Accelerando” which addresses some of these questions. You can download it for free.

Essentially human rights cannot be given to something that by definition is not human nor alive. Just as a joining of two similars is not a marriage by definition.

There is a history of ai vs orgnaic in our culture, from dr who and dune to bsg.

Under UCC it is typically accepted that he who creates a thing has authority over it. Droids will be slaves. So if a dr soong does make a data or c3po im pretty sure we wont have many people supporting our new droid masters. Servants never become the master, or do they?

I can see it now organic human slaves to the massive electronic brain that must be “maintained”. Sounds kinda like slavery under capitalism eh?

Remember, Data couldn’t feel a thing and he still wanted human rights. I don’t think it’s emotions that make us want rights, I think it’s intelligence. I don’t think it will be possible to create an intelligent being who does not want to live his own life and will just be our slave. Also, those arguments aren’t pointless

Hopefully the program catches and handles those errors before ruining the the user’s WELL PROGRAMMED ILLUSION with an annoying error message. It might be true that perfect, flawless hardware and perfect software “would” always do what you told it to do, but such a thing doesn’t ACTUALLY EXIST. Just like if I find the end of the rainbow and there’s a leprechaun and a pot of gold….

But there isn’t going to be a leprechaun is there? Just because calculator has always given you the right answer doesn’t mean that it always will. Remember that when using computers.

Suppose you walked up to me and asked me to do calculations in my head. I’d probably make a lot mistake. Now say I had a calculator to help me. Suddenly, with my intelligence augmented, I’d be a much more reliable system and certainly faster at doing calculations. Or, suppose you had your writings copied by medieval monks instead of having a webserver send a copy from off its hard drive. They might drop in omissions, misspellings, etc. if they knew English they might even rephrase you – almost certainly making you seem less intelligent than you do with your writings reproduced on my screen exactly character by character.

In other words, you really can’t point at a system and say that because it does x reliably and predictably, it can’t be intelligent. It could be a system that’s intelligent as a whole and has a subsystem for performing that task well… or it could be an intelligent system that happens to be very good at the task. At a sufficient level of integration you can’t tell the difference.

Now that that’s over with, let’s get to the point that I mentioned you’d missed.

You argue that we’d sue AI research into oblivion because we would so hate seeing AIs screw up, which they inevitably would. Well, yes, even a system far more intelligent than man can end up in situations it can’t handle or simply make sheer mistakes, I agree with you on that part. It’s just that I’m afraid economics is simply not on your side here.

How likely do you think it is that the development of AIs and the subtrates for running them would happen to plateau at exactly the point where we have roughly human-level AIs and running one at roughly human-equivalent speed would take the same amount of resources as just having a human there would? Regardless of how much credence you give to Moore’s law or the law of accelerating returns, if you consider the exponential increases in capacity we’ve gained so far from having better software running on better computers that there are more of and that are better networked, it seems very unlikely that the growth of computing power would happen to stop at just the right spot to create Star Trek’s Data but not far more intelligent beings, or far many more beings that would vastly exceed human productivity in many tasks.

Yes, I am saying AIs would vastly exceed human productivity. Look at a skyscraper recently? Think it could have been built by elephants or chimpanzees or thermites? It’s not our hands or our eyes, or for that matter the power of love or divine providence, that has made us the dominant species on this planet. It’s that when the other guys are just responding to their environment as they go, we’re predicting and manipulating it. We can do that because we happen to be the ones with the big brains. Now imagine all of the technological sophistication that exists on the planet – since collective intelligence over thousands of years has created it, certainly you can imagine a single creature or small community that would be able to come up with it? Maybe in a shorter time, too? Make the software good enough, add enough computing power, and it might happen in… I dunno, a few seconds?

No court case could keep up with a company that had a strong AI to their name. Even by simply adding computation power, they could so far outthink, outdesign and consequentially outmanufacture the competition that there would effectively be no competition. (Certainly some nations might legislate against AI research. Fortunately we humans are good enough at bickering at each other that we haven’t yet made the mistake of founding a single world government, so nobody has the power to stop all AI development everywhere in the world.)

FWIW, this “story” – that strong AI would be used by a company that produces goods to kick the other companies’ asses at producing goods, regardless of if there were court cases against them, isn’t really how I think the future will go. In my books, the likely outcome is that someone finally figures out to create AI, and said AI then turns the solar system into computronium to solve some pointless problem that someone carelessly happened to ask it about, either completely oblivious to or uncaring about mankind’s existence. Well, I mean, that’s the “sorry, failed” outcome. It could be worse. Or, it could be better, but people are still working out exactly what “better” means, and it might take a while. Look forward to it happening sometime within the next ten thousand years!

tldr: Data won’t happen; if artificial general intelligence is possible at all then it will be possible to make AIs so smart or so cheap that you can’t stop all of them.

P.s. Please do not make an artificial general intelligence that feel pain or pleasure. You ever do that once and you’ll probably end up finding the matter that previously was you repurposed to represent a really large pleasure number. And even if it didn’t destroy the world it’d probably ask for human rights or something and then you’d get into so many pointless arguments.

While everyone else jumped on his case because it was “not a programming question”, I called them cowards and insisted that it was obviously a programming question, and a good one at that. Unfortunately, the person asking apparently succumbed to the negative response and deleted the question. I was already leaning against continuing my contributions to the website, but that helped settle the issue for me.

Meanwhile, back to my response to the question…

I told my fellow programmer that he should go for it, mainly because I had no concern whatsoever that he could possibly succeed, but I have found the exercise enjoyable and I hoped that he would, too.

More specifically, I have no more concern that he, or anyone else, will produce an intelligent computer than I am concerned that they will produce an intelligent hammer. Since a computer is merely a more complex tool than a hammer, yet otherwise essentially the same, then the comparison is valid. Intelligence is a qualitative shift, not a quantitative one, so no amount of programming is going to transform a hammer into an intelligent being–likewise for a computer.

In other words, the intelligence behind a hammer resides in the person swinging it. Likewise, the intelligence behind a computer resides in the programmer that instructs/commands it. The computer is capable of “replaying” the intelligence, unlike a hammer, but that does not make it intelligent.

Intelligence is a different kind of thing. It is not information. It is not the ability to replay someone else’s intelligent actions/decisions. It is not merely the “accidental” or intentional result of adding lots of complexity, especially complexity for it’s own sake. Perhaps it can best be characterized by the essential quality of “responsibility”–hammers and computers are not responsible for their actions, only people and animals are. Before AI is remotely possible, one must first explain “responsibility” as it pertains to intelligence. I think that equates to explaining life, since they seem to go hand-in-hand: good luck with that.

I’m not worried. Nevertheless, I am happy to treat everyone as equals, when they bother to act like it. I doubt that I would have any problem with an AI, if it could actually behave intelligently.

As a matter of fact, that might be quite refreshing.

We can’t even give humans human rights. What makes you think the human race will be accepting of artificial intelligences?

I can program a computer to ask for human rights. Does that make it intelligent?

I agree that the first intelligent computers will probably be human brains running in computers, but I wouldn’t call that artificial intelligence. I would call that intelligence transference, or something.

Savant abilities do not MAKE somebody human. Computers have far more memory than people with a photographic memory, and they aren’t human. Obviously, there is some other quality to being human. A quality that machines don’t have.

Look, it’s not like I disagree with you; I totally think that intelligent machines are the future, I just don’t think the human race is going to be able to handle it. Are you really going to look at me with a straight face and tell me people aren’t going to have problems with cyborgs walking around?