BOOKS: 4th AGE

PART 1: THE LONG ROAD TO TODAY

===========================

Chapter 1. First age. Hunters-gatherers.

{Notewords: technology fire calories brain complexity language ideas storytelling 100k years 200k species }

Technology is the application of knowledge to an item, process, or technique. And what is technology for? Primarily for enhancing human ability.
Fire was the original multifunction technology. It provided light, and, because animals feared it, it also provided safety. Its portability meant that humans could migrate to colder climates and bring warmth with them. But far and away, its greatest benefit was that it let us cook food.
Cooking allowed us to vastly increase our caloric intake. Not only does cooking meat make it easier to chew, but, more important, it unwinds the proteins within it, allowing for better digestion.
We used this new energy to grow our brains to unprecedented complexity. In a short period, we gre ever, is like an Italian supercar: it can go from zero to sixty in the blink of an eye, but it sure burns the gas doing so. Humans, in fact, use an incredibly lavish 20 percent of all of the calories we consume just to support our advanced brains.
From a survival point of view, this was a pretty bold bet. To borrow a phrase from poker, humans went “all in” on the brain, and it paid off for our more powerful brains led to our creation of another new technology: language.
The vastly more powerful technology of language allowed us to exchange information. Additionally, language enabled us to cooperate with each other, which is one of our singular abilities as a species. It allowed for idea formation and storytelling.
During the First Age (the roughly 100,000 years in which we lived as hunter-gatherers with both language and fire), total population of humans was around 200,000, so while we were not an endangered species, the survival of humanity was far from certain. Although there was undoubtedly a wide variation of practices, a large number of these people lived in collectivist, largely nonhierarchical societies. Even today, the best estimates suggest that there exist more than a hundred uncontacted hunter-gatherer tribes, whose members may number a total of over 10,000 people.
Sustenance was not something one could take for granted, and that any individual was just a few days’ illness away from death.
There was no wealth beyond the day’s haul of grubworms and no way to store wealth even if it was there.
We can confidently say that life as an ancient hunter-gatherer was short, painful, and harsh. But this was humanity’s proving ground, and with language, we embarked on the path that brought us to today.

Notes:

Storytelling — Gilgamesh https://en.m.wikipedia.org/wiki/Gilgamesh

===========================

Chapter 2. 2nd Age: Agri and Cities

{Agri city specialization weapons wealth aristocracy inequality tension vs freedom planning future}

Invention of agriculture. The Second Age began just 10,000 short years ago, when the human population of the planet was about four million people. In the preceding 90k yrs, we managed to double our population merely 4 or 5 times, which is an incredibly slow growth, indicative of how difficult human existence prior to agriculture.
Agriculture is also a technology, and it brought about a lot of other advances. The first was the city, as agriculture created city as it required people to settle in one place. That was new. Early cities (Catalhuyuk, Jericho, and Abu Hureyra) were often located near rivers for access to water and fertile farmland.
These first cities contained markets, homes, and temples. During the 2nd age, people began using opium, gambling with dice, and wearing make-up and gold jewelry.
Cities made us stationary. Housing became permanent, people started building fences and marking locations of their dead for homages. Thus private property was created.
Cities created division of labor. The division of labor works the best when large numbers of people live in close proximity. Farmers living far away from their neighbors couldn’t specialize, thus they had to know how to do everything without being particularly good at any of it. That was a major milestone, because with the division of labor came increased efficiency, as a given individual no longer needed to know everything in order to survive.
Along with trading and technological advances, the division of labor is one of only three “free lunches” in conventional economic theory; that is, one of the ways overall wealth can be increased without anyone’s having to work any harder.
The famous essay by the economist Leonard Read, “I, Pencil,” describes how although no one person knows how to build a simple pencil, the pencil still gets made, because thousands of people in hundreds of fields, who will never meet, each do a small part of each thing that it takes to make a pencil. The division of labor gives us virtually everything we have today. Without it, we would perish.
Weapons for organized warfare are another technology that came along because of the city. They were invented out of necessity, because the city concentrated wealth and needed to be defended.
The hunger for wealth seems to be limitless, at least in some, as is evidenced by those who feverishly work to earn their second billion, even with the full knowledge that they won’t ever, in a hundred lifetimes, spend their first.
Agriculture and privately owned land ended the economic equality of the First Age. The natural inequality of ability, birth, and luck led to unequal accumulations of wealth.
It is against this backdrop that people separated into the rulers and the ruled. Aristocracy and royalty emerged during the Second Age. The ruling classes frequently adopted practices that those they ruled over were not permitted, such as wearing certain types or colors of clothing, eating certain foods, or, in the case of the Aztecs, smelling certain types of flowers.
This is also where the tension between two values, freedom (right) and equality (left) was g6t, was first highlighted. As the historian Will Durant points out, you get to pick only one, because you can’t have them both. People truly free will become unequal. People with equality forced on them are not free. This tug-of-war still plays out today.
Earlier, I referred to imagination as the first requisite for progress. Agriculture gave us the second. Since planting and harvesting crops required planning in a way that hunting and gathering did not, we can think of the invention of agriculture as the invention of the idea of the future, which is the second requisite for progress.

===========================

Chapter 3. 3rd Age: Writing and wheels.

{writing wheels money contracts legal codes empires concurrent}

Writing — 5k years ago / 10 million people. Likely invented by Sumerians, and around the same time independently developed in Egypt and China. Later, it would be independently developed in what is modern-day Mexico.
Writing changed humanity because for the first time, what a person knew could live after him or her, perfectly preserved. Knowledge could be flawlessly copied and transported around the world. Ideas could live outside a human mind.
In the earliest forms was about keeping track of assets and transactions. From there, it grew to cover legal records, codes, eg Code of Hammurabi, and religious texts. Creative writing would come later.
Writing is the great dividing point in human history. The First and Second Ages are, by definition, prehistoric. History begins five thousand years ago with the Third Age. What of our world would exist without writing?
By the way, not everyone liked the idea of writing. Plato describes a great king rebuking the god who invented writing for the effect that writing is going to have on memory. This is correct, actually. Just as we outsource part of our digestion to fire, similarly we outsource part of our memory to writing. Before writing, if you knew something, you’d better remember it, as there was no way to write it down. Now we struggle to remember our ATM codes or friends’ phone numbers. Now that everything is a Google search away, our memories may decay further.
Writing was concurrent with the invention of the wheel. Together, they promoted spreading of laws and empires over large swathes of territory.
Early legal codes, such as the four-thousand-year-old Code of Ur-Nammu, specify specific punishments for murder, robbery, kidnapping, rape, perjury, assault, and a variety of crimes relating to land ownership, such as flooding a neighbor’s field, failing to cultivate a rented field, and secretly cultivating someone else’s field. The Code of Hammurabi, only a couple of centuries younger, covers all that ground in its 282 laws, but it adds in the enforcement of contracts, product liability, and inheritance.
Money appeared during the Third Age as well. Metals were considered the ideal media of exchange because they were widely valued, divisible, durable, and portable.
With writing, the wheel, and money all coming on the scene concurrently, the basic ingredients needed to make the nation-state and empires were in place. This is when we saw the first large civilizations blossom all over the world, independently and virtually simultaneously. China, the Indus Valley, Mesopotamia, Egypt, and Central America all became home to large, cohesive, and prosperous nations. No one has any idea how it came to be that these civilizations emerged nearly concurrently in places with no contact with each other. The same is true for writing. Why didn’t some parts of the world get writing, wheels, agriculture fifty thousand years ago? Or twenty thousand? No one knows.
For us to rightly say we have launched into a new age, something has to have come along that has changed us and how we live in a profound and permanent way. Something that has altered our trajectory as a species. And that story, which launches us into the Fourth Age, has its origins in the final few centuries of the Third Age.

Notes: * https://en.m.wikipedia.org/wiki/Code_of_Hammurabi

===========================

Chapter 4. Robots and AI.

{ scientific method processing power doubling mechanical brains AI philosophical questions}

We’re used to rapid advance of technology, but it wasn’t always the case. For 99.9% of human history, it didn’t. One of the oldest tools, for example, Acheulean hand axe, remained unchanged across a million years.
Some historians say that Leonardo da Vinci was the last man to know everything. What is meant is that he lived at a time when science was so nascent that a single person could have a working knowledge of all that is known. By 1600, things have gotten underway scientifically. In 1609, Kepler wrote a letter to Galileo talking about spaceships: “Ships and sails proper for the heavenly air shall be fashioned. And there will be people who do not shrink from the dreary vastness of space.” In 1620, sir Francis Bacon published a book called Novum Organum (The New Method), describing what we now know as a scientific method. Scientific method enabled compounded growth in aggregation of knowledge.
Bacon's 1620 book, while it doesn't quite propose the scientific method itself, was a critical piece in the process in that it suggested that it should not be good enough to deduce something; it should also be necessary to disprove the contrary.
Today’s scientific method is a set of agreed-upon techniques for acquiring knowledge, and then distributing that knowledge in such a way that others can corroborate and build on top of it. It applies only to objects or phenomena of which measurement can be made. Objective measurement is essential because it allows others to reproduce a researcher’s findings, or, as is often the case, to be unable to reproduce them. The scientific method required affordable printing to work properly, so the process didn’t really work properly before printing became cheap enough.
The ancients had many breakthroughs, but didn’t have the technology to spread the knowledge, and so these breakthroughs were quickly forgotten. Example: Antikythera. Antikythera mechanism is an ancient Greek analogue computer and orrery used to predict astronomical positions and eclipses for calendar and astrological purposes decades in advance.
Technology progresses by making incremental improvement on work done by others, a process Isaac Newton described as seeing further by standing on the shoulders of giants.
The scientific method supercharged technological development so much that it revealed an innate but mysterious property of all sorts of technology, a consistent and repeated doubling of its capabilities over fixed periods.
Kurzweil found that if you graph the processing power of computers since 1890, when simple electromechanical devices were used to help with the US census, computers doubled in processing power every other year, regardless of the underlying technology. Think about that: the underlying technology of the computer went from being mechanical to using relays, then to vacuum tubes, then to transistors, and then to integrated circuits, and all along the way, Moore’s law never hiccupped. How could this be? No one knows.
It appears to be some kind of law of the universe, that it takes a certain amount of technology to get to a place, and then once you have it, you’re able to use that technology to double that again.
Domino rally. Each domino can push over a domino 50 percent taller than itself. So if you set up thirty-two dominos, each 50 percent bigger than the first, that last domino could knock over the Empire State Building. And that is with a mere 50 percent growth rate, not doubling.
The fact that technology doubles is a big deal, bigger than one might first suspect. Humans famously underestimate the significance of constant doubling because nothing in our daily lives behaves that way. You don’t wake up with two kids, then four kids, then eight, then sixteen. Our bank balances don’t go from $ 100 to $ 200 to $ 400 to $ 800, day after day.
So while it took us almost five thousand years to get from the abacus to the iPad, twenty-five years from now, we will have something as far ahead of the iPad as it is ahead of the abacus. We can’t even imagine or wrap our heads around what that thing will be.
Scientific method + Moore’s law gave us explosion that we are seeing today: robots, nanotechnology, CRISPR-Cas9, space travel, atomic power, etc.
We now have powers that have before been attributed to the gods. We can communicate instantly with anyone on the planet. We can see events as they happen at great distance. We can travel through air at the speed of sound extremely safely.
Of all technologies of the Third Age, one advance tops them all: the computer. Why is it fundamental? Because everything is a computation. Our brain, universe, space, time, consciousness. Computation in the physical world can be modeled inside a computational device. Wolfram reference.
Much of the universe clearly is computational. Hurricanes and DNA are computational, as are snowflakes and sand dunes. And yet we could mirror it with ones and zeros inside a tiny processor. This suggests a profound truth, that anything that can be modeled in a computer is occurring via computation in the real world as well.
Are we computation? Are our minds giant clockworks? This is what we need to answer to understand the limits, if any, of computers.
Hammer pounds a nail, saw just cuts wood, but a computer can mirror a billion different things from the physical world. Philosopher Marshall McLuhanxx said decades ago, the computer is “the most extraordinary of man’s technological clothing; it’s an extension of our central nervous system.” How did we decide to make one or imagine that one is possible?
Babbage realized machines could do math. Turing added that they could also run programs. Von Neumann figured out how to build the hardware, and Shannon showed how the software could do things that didn’t at first look like math problems.
Technology is relentless: It gets better and cheaper, never stopping. In 1960, you could buy a transistor for about a dollar, or just over $ 8 in today’s money. So, if you needed 125,000 of them, you needed a million dollars in today’s money. But the price fell dramatically as the quantity produced skyrocketed. By 2004, the number of transistors manufactured surpassed the number of grains of rice that were grown across the planet. Just six years later, in 2010, you could buy that same 125,000 transistors that cost a million dollars in 1960 for the same price as you would pay for a single grain of rice.
We used fire to help with digestion, writing to augment our memories, the wheel to spare our backs and legs. In our time, we have created a device, a mechanical brain, that is so versatile that it can be programmed to solve virtually a limitless number of problems that we ask it to. We are now developing artificial intelligence, a method to teach that device to operate on its own, and through the power of robotics, we have begun to give it mobility and ways to interact with the physical world. We will use computers and robots to outsource more and more of our thoughts and actions, presumably as much as we possibly can. This is a real change, and it marks the dawn of a new age, the Fourth Age.
The questions that come out of this transition are profound, for they relate to what it means to be human. Can machines think? Can they be conscious? Can all human activity be mechanically reproduced? Are we simply machines? The ultimate purpose of this book is to explore these ideas—to figure out just how much human activity, both mental and physical, we can delegate to machines.
It took us 5,000 years to get from the wheel to the moon. (Interestingly, we would make it to the moon before it occurred to anyone to attach a wheel to luggage.) But we weren’t halfway to the moon 2,500 years ago. Far from it. When a human first walked on the moon, we had broken the sound barrier only two decades earlier. We had achieved heavier-than-air flight only six decades earlier. So for 4,940 years after inventing the wheel, we were still firmly attached to the face of this planet.
We will probably see more change in the next fifty years than we have seen in the last five thousand.
For millions of years, we have used technology. For a hundred thousand years, we have had language. A few thousand years ago, we started asking profound questions about the universe and our place within it. A few centuries ago, we systematized science and created unimagined prosperity. A few decades ago, we began making mechanical brains, and just a few years ago, we learned dramatic new ways to make those brains more powerful. It feels as if this very moment is the great inflection point of history.

Notes

Francis Bacon. The new method. https://www.amazon.com/Novum-Organum-Method-Francis-Bacon/dp/1612034470

Ayn Rand. Introduction to Objectivist Epistemology.8

Stephen Wolfram. A new kind of science.

===========================

Chapter 5. Three Big Questions.

{ What is composition of the universe? Monism and dualism. What are humans? Machines, animals, humans. What is self? Brain trick, emergent mind, soul. }

To answer questions on everybody’s mind, such as: what AI will be capable of doing, whether computers can become conscious, and whether they will take all our jobs, need to ask three foundational questions.
If something is possible, it is inevitable. Therefore, if above is possible, it will happen. But is it? This is the type of thing we used to discuss with our college friends late at night. However, now these are the central questions of our time.

Q1. What is the composition of the Universe? * The best thinking on this comes from the ancient Greeks. There were two schools of thoughts: 1) Monism (materialism): believe that everything in the Universe is made of a single substance, atoms. These atoms all follow the same set of physical laws. These laws are largely known today. Nothing in the universe happens that cannot be reduced to physics. Physics explains chemistry, chemistry explains biology, biology explains life, life explains consciousness. — Democritus: “By convention sweet is sweet, by convention bitter is bitter, by convention hot is hot, by convention cold is cold, by convention color is color. But in reality there are atoms and the void. That is, the objects of sense are supposed to be real and it is customary to regard them as such, but in truth they are not. Only the atoms and the void are real.” 2) Dualism: dualists believe universe is made of two (or more) things. In addition to atoms there is something else. One class of dualist thought (known as religion) posits that there are things like soul and god(s). Another way is to say that the universe is composed on physical stuff and mental stuff. The mental stuff includes hopes, dreams, regrets, love, hate, etc. While these things are triggered by physical processes in the brain, the experience of them is not physical, which is a subtle but important difference. * For example, is an idea a ‘thing’? Some would say yes, because an idea is a pattern of neurons in our brain. It exhibits entropy. However, the way it spreads and behaves through time, the impact that it has on the environment, that has nothing to do with physics. * Is there any difference between knowing about something and experiencing something? For example, experiencing color.

Q2. What exactly are we?
* Three answers are possible: machines, animals, or humans. * Machines. We are self-sustaining chemical mechanical reactions. Thought experiment of teleporter. Coming out on the other side, is it you or is it your copy? If it’s a copy, what makes the original copy of you so unique? * Animals. This posits that mehanical world is completely different from the biological, living world. There is something to life other than electrochemical, as we could build a living thing with a couple of batteries + chemistry set, whereas life has animating force, something mysterious that is beyond machines. What does it mean to be alive? No consensus on that. * Humans. The Dalai Lama expressed it as such: “Humans are not machines. We are something more. We have feeling and experience. Material comforts are not sufficient to satisfy us. We need something deeper—human affection.” * Consider that 99% of our DNA is the same with chimps. What makes us different is a rounding error. And yet we are not chimps.

Q3. What is your ‘self’? * When you look in the mirror, you recognize yourself. What is the voice in your head that talks to you? What is the ‘I’? Before we get down to the options, ask yourself - what makes you “you”? * Some say it’s memory and experience. But then a person who experiences amnesia is no longer a person. Is that true? * Continuity of physical form. Thought experiment of ship of Theseus in the museum. We are the living ships of Theseus. Our cells replace themselves, yet we remain ourselves. Even while cells are not replaced, they constantly change their relationship with each other. * Self is inextricably tied to consciousness. So the question is whether computers can have a “self”. Three options for what a “self” is: i) a brain trick, ii) an emergent mind, iii) a soul. 1) Brain trick: consists of two parts. One is integrated experience. See rose and smell it at the same time even though it is processed through completely different parts of the brain. It is one single, integrated, sensory mental experience. The second is focus and the ability to switch it on the fly. Instead of hearing constant cacophony, there is a nice and orderly single voice, depending on which part grabs the mic. Focused and diffused mode experience. If there was a ceremony master who determines which part gets to speak, that would be “you”. But brain science tells us such part does not exist, the brain basically self-regulates. Hence the illusion of you. 2) An emergent mind. Emergence is a fascinating phenomenon. At its simplest, emergence is when a group of things interact with each other, and through that interaction, the collective whole gains characteristics that none of the individual things has. For example, we consist of 40 trillion cells. These cells have no idea we exist and they are even a part of something else. But we, with all our abilities and attributes, are not simply a cumulative results of all our biological processes. None of these parts have a sense of humor, and yet somehow we do. Ant colony and no ant is in charge (neither is the queen, she just lays eggs). The colony itself has an emergent mind. If you believe there is a real “you” calling the shots, not the self-regulating brain, that’s the theory. 3) Soul. Worldwide, 75% of people believe in God, 15% are agnostic, and 10% are atheists. Sensation of having a soul is a real but private experience, so it’s a challenge for science to deal with it, as a soul by definition is outside the realm of physics. * The question is not whether any or all of these things exist, but which one of them is “you”.

PART 2: NARROW AI AND ROBOTS

**

Chapter 6. Narrow AI.

{classic AI expert system machine learning} * AI. Term was coined by John McCarthy at Dartmouth. The goal of the project was to “find how to make machines use language, form abstractions and concepts, solve the kinds of problems now reserved for humans and improve themselves.” There is a problem because it’s unclear what “artificial” is (two definitions) and there is a disagreement as to what is intelligence. * Narrow (=weak) AI is the ability for a computer to solve a specific kind of problem or perform a specific task. The other kind of AI is referred to by three different names: general AI, strong AI, or artificial general intelligence (AGI). * Narrow AI examples: Siri or self-driving car. A hypothetical robot that can unload a dishwasher. An AGI would be technology that is able to respond to situations it has not previously considered. An AGI does not yet exist and there is no agreement on how to build it. * How does it work? Three different approaches. Let’s say you need to build a system that tells farmers when to plant their seeds. 1) Classic AI. Build a model that takes into account all possible factors (soil type, crop, rainfall), weigh them, and make a suggestion. 2) Expert system. Take 100 best farmers, have them write down every rule about planting they know, arrange those rules in a way that someone can enter variables and get suggestion based on the rules.
3) Machine learning. Take all data on when everyone planted and what the yields were, and get a computer to build rules that would have maximized those yields in retrospect. While suggestions may work, they may be undecipherable by human.

===========================

Chapter 7. Robots.

{robot fears} * The primary fear is that robots will compete with us in the job market and that we will lose to them. To date, robots have been an economic boon for humans, freeing us up to do ever more complicated and valuable jobs. But what happens at the end of that curve? What happens when a robot can do many or most jobs better than a person? * This fear is compounded by a basic characteristic of manufacturing. Over time, prices tend to go down as the quality of goods produced goes up. So robots will become better and cheaper, indefinitely. Surely, we fear, at some point robot labor will be both cheaper and better than human labor. Will the less skilled be locked out of the job market forever? If a fast-food robot costs $ 10,000, would a business rather pay a human a $ 15 minimum wage—or a $ 10 minimum wage, or any minimum wage? * And of course, the ultimate fear is the robot uprising. Although this is often spoken of with tongue firmly in cheek, there are several popularly imagined scenarios in which the robots themselves become adversarial, either through human malevolence, technical error, or an unexpected emergent behavior. It’s a fact that militaries around the world are currently working to build better and better killer robots.

===========================

Chapter 8. Technical challenges.

{seeing contextualizing transfer learning improvising mapping slam doorknob touch no datasets towel folding catastrophic failure} * The first problem AI robots have is seeing. We can put a great camera in a robot, but that just gets us data. The robot has to sort it all out. When you look in your kitchen’s pantry, you see a bunch of individual items. But a robot just sees pixels, a kaleidoscope of millions of specs of light and color. It doesn’t know what a box is, or a shelf, or even an edge. * Let’s imagine that AI can see. They still wouldn’t be able to understand anything they “see.” That’s because AI’s can’t contextualize. Life is kinetic, it moves. Context is derived from the differences in a series of still images, and there are innumerable possible permutations of that. When you see your young neighbors, a husband and his very pregnant wife, hurry toward the car, with her holding her belly and him carrying an overnight bag and looking worried, you don’t have any trouble figuring out what is going on. That’s hard for a computer. * Transfer learning — apply what you’ve learned in one area to a different area. * We don’t know how to teach it to do transfer learning. But let’s say we solved that problem. We haven’t and likely won’t any time soon. But let’s say we have and the AI can take what it learns in one area and apply it elsewhere. We still aren’t very far along, because the AI doesn’t know how to improvise. Everyone, of every skill level, can improvise in a way far beyond any machine. If the door handle breaks off in your hand, you don’t just stand there frozen, unable to fathom what to do in a universe you never contemplated. * If you want an AI to play chess or detect email spam, then you just teach it that simple thing. You don’t ask it to generalize, contextualize, be creative, or anything else. Just teach it that one thing. And that is what we can do now pretty well. That is why it is called narrow AI. * Today: AI routes you through traffic, filters out spam, predicts the weather, suggests products for you to buy, and identifies credit card fraud. Smart cameras use it to identify faces, sports teams use it to improve their game, HR use it to hire people. AI reads handwriting, turns speech into text, and translates both of those into other languages. All these tasks are purely computational activities. * Anyplace where we have lots of data, even if it is unordered raw data, is an area where AI can really excel. * AI will soon take all the satellite data we have and find ancient cities for archaeologists to dig up, keep track of populations of wild animals, and monitor vegetation growth. Then it will take all the traffic data and use it to help us build smarter roads, time traffic lights more effectively, and reduce accidents. The list is endless. Kevin Kelly, the founding editor of Wired magazine, summed all this up when he tweeted, “The business plans of the next 10,000 startups are easy to forecast: Take X and add AI.” * There isn’t a human alive who can solder a billion transistors onto a computer chip the size of a postage stamp, but robots can. Without robots making the things that we use every day, we would have 1950s technology powering a 1950s economy that produces a 1950s standard of living, at best. * The first challenge robots have is figuring out where they are. A robot has been dropped into a room. It “sees” a chair and a footstool. Since the chair and the footstool can be moved, the robot can’t use them as anchors. If it finds itself closer to the chair than it was a minute ago, it doesn’t know if the chair was moved, it was moved, or both. As such, it has to constantly redraw its map. * Then, of course, there is the challenge of powering the robot, especially if you want the mobility that batteries enable. We are far from cracking this one. To offer just one example, in 2016, an enterprising AI-enabled Russian robot named Promobot, which had been programmed to be autonomous, managed to escape the testing facility and made it 164 feet before running out of power in the middle of the road, causing a half-hour traffic jam. So much for the robot uprising. * To give you a sense of the difficulty roboticists are facing here, consider the DARPA Robotics Challenge, which took place from 2012 to 2015. In 2015, the finals were held. Erik Sofge, writing for Popular Science, summed it up by saying that “the biggest and most well-funded international robotics competition in years was a failure.” The robots had to drive a car, navigate across rubble, use a doorknob, find a valve and shut it off, and so forth. * For a human, the DARPA challenge doesn’t seem all that hard. What could be easier than turning a doorknob and opening the door? A lot of things, it turns out. The robot has to identify the doorknob, navigate its hand to it, and squeeze it. Not too hard, not too soft. The robot needs to determine the friction of the knob itself. It then needs to turn it. Humans can tell pretty easily if the knob is turning or if their hand is sliding on the knob. That is very hard for a robot. Humans can tell when to stop turning because of resistance. The robot has to be trained to actually stop at some point before it breaks the mechanism. Then, holding the knob in the open position, the robot has to push. How hard? Well, that’s extremely difficult to know ahead of time. How heavy is the door? Is it stuck? And all of that is for naught if it turns out the door was one you pull, not push. * Touch is a big challenge for robots. But even if you build a magnificent robotic finger, the robot still has to perceive what is going on at the end of it. Consider changing a baby, holding a kitten, or comforting a frightened child. Think of the nuance of the pressure and the timing of every touch that you do “without thinking.” But a robot can’t do anything without thinking. It has to dissect every action to the most minuscule detail. * One wrinkle with these sorts of perception problems is that we don’t have the training data to teach the robots. Amazon has a huge database of “people who bought this also bought that” with which to train its recommendation engine. But we don’t have all the tactile data of a million adults holding a million babies in a thousand situations. * “Once you start working in robotics, you realize that things that kids learn to do up to age 10 . . . are actually the hardest things to get a robot to do.” * Errors. Back in 1962, a NASA rocket that cost a bit under a billion dollars in today’s money exploded in flight due to a single hyphen missing from deep within its code. In another example, a European rocket exploded in flight with $ 7 billion worth of loss because a 64-bit number was too large to convert to a 16-bit number, causing both a metaphoric and a literal crash. Now imagine this issue with a self-driving car network. * Digital systems are generally more brittle than analog ones. Delete a word from The Great Gatsby and you still have a masterpiece. Delete a character from a compressed file and you have . . . alphabet soup. A missed hyphen won’t make a human explode. We tend to be wrong lots of times but just by a little bit. Machines fail less often but to more catastrophic ends.

===========================

Chapter 9. Will robots take all our jobs?

{excerpt on the nature of overwork} * Half life of any job is about 50 years. We went from 50% agri + 30% industry economy at the turn of the previous century to 80% service / 20% everything else economy in 100 yrs. The unemployment rate barely moved. * Three key POVs: robots will take all the jobs; robots will take some of the jobs resulting in a LT structural unemployment; and there won’t be any impact. * The most compelling argument of ‘there won’t be any impact’ school of thought is “people will work anyway”. Keynes, for example, pointed out that for thousands of years up until 1700, human standard of living was essentially flat. Then, through accumulation of capital and technology, the modern economy was created, bringing the magic of compounded growth and compounding interest. Keynes expected the standard of living to rise 4-8x by 2030, and he was right — we are on track to hit the upper end of his projection. He also said, based on this assessment, that as human needs are essentially fixed, people will end up with a lot of time on their hands once their ‘economic problem’ is solved (something I’m beginning to see as a risk personally). First, we may continue to work just out of habit. And second, because we have worked so hard for so long, we may not find happiness and meanining in leisure. * Keynes mistake was to predict that by 2030, people would be working 15 hours a week which would satisfy all material needs and a few of our wants. He then goes on to express contempt for those who would still be workaholics in such a world: “Of course there will still be many people with intense, unsatisfied purposiveness who will blindly pursue wealth—unless they can find some plausible substitute. But the rest of us will no longer be under any obligation to applaud and encourage them.” * The simple reality is that humans collectively decided to work more than needed and purchase more of their ‘wants’. We have opted to collectively stay late at the office instead of growing and peeling our own potatoes. We all want a higher standard of living—and that desire is what creates most of the jobs. As long as you want more income, you will likely find a way to use your skills to add value somewhere, and that action is what creates a job. * Strictly speaking, we could, technically, survive by working less. We choose not to and instead to enjoy an ever-rising standard of living. That’s where Keynes missed his mark just a bit. What were once luxuries we now consider necessities, and there is no reason to think that trend will somehow reverse. * Why don’t we work less? Keynes, it seems, saw a few answers. According to him, it could be that our inability to collectively chill is a memory of poorer times, and it will take several generations for us to adapt to being part of the leisure class. Or maybe “keeping up with the Joneses” is so deeply ingrained in us that we can never rest. As long as we can see their brand-new flying car from our window as we eat our breakfast, we may never be willing to take a day off. Or perhaps permanent leisure is corrupting in some way and we thus have a deep internal abhorrence of it and its temptations. Or it could be that we haven’t figured out how to get satisfaction from work when we don’t have the yardstick of money to tell us how much worth to place on it. Or, most unsettling of all, maybe we are the greedy materialists whom Keynes says “love money as a possession, as distinguished from [loving it] as a means to the enjoyments . . . of life,” a condition he identifies as a “semi-criminal, semi-pathological . . . mental disease.” * Keynes himself died of overwork, so even he might have been a bit conflicted on the point. * To offer another interpretation, perhaps we are driven toward perpetual progress by our mild discontent with the present. No matter how good things may be, we can always picture them a little better, and the drive to relentlessly move forward and upward is our distinctive characteristic. Maybe we aren’t so much Homo sapiens, the reasoning man, as we are Homo dissatisfactus.

===========================

Chapter 10. Are there robot-proof jobs?

{robot-proof quiz with 10 qs} * Prediction: “the factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment.” * A good method for evaluating any job’s likelihood of being automated is what I call the “training manual test.” In a program, every step, every contingency, every exception, needs to be thought through and handled. Could anyone write a set of instructions to compose a sonata or write a great novel? * Jobs robots can do but probably never will; jobs we don’t want robots to do; unpredictable jobs; jobs that need a high social IQ, jobs done on site, jobs that require creativity or abstract thinking, jobs no one have thought of yet. * 99% of human skills are the same. * Given a chance to take on more challenging work in exchange for more responsibility and money, most people tend to say yes. People want meaning and purpose and, of course, higher wages, if they can get them.

===========================

Chapter 11. The Big Questions

{acceleration of inequality via technology and paths to address it via policy}

Two Italian economists compared the Florentine tax rolls of 1427 to those of today and discovered that the richest families then are still the richest now. In England, using the same methodology, other economists found that the wealthy back in 1170 seem to still be the wealthy today.
Wealth begets more wealth through ability to invest into superior technology.
In addition, today’s technology can multiply productivity so dramatically that it’s easier than ever to create vast amounts of wealth from very little, and use that wealth to create even more. There are more billionaires alive right now than ever before, and the percentage of them that made their own money, as opposed to inheriting it, continues to rise. But in the last decade or so, Google created seven billionaires and Facebook minted six.
First, when technology is adopted by corporations, all the increases in productivity drive up the value of the company’s stock through lower costs and increased margins. Since in the United States, 70 percent of all stock is owned by 20 percent percent of the people, this financial gain goes in the pockets of the already well off. Second, people who sell their labor on self-employment basis also benefit from technological gains, as technology reduces the amount of low-value tedious work they have to do and frees up their time to do more high-value work. Third, tech improves quality of life for everyone as price of tech drops in real terms.
Technology requires investment, and investment requires wealth. So that is how technological advances drive increased income inequality, absent changes in policy.
Flat median income is the problem; economic benefit is not getting distributed down the food chain.
No matter what happens with robots and jobs, upheaval is unlikely, because the cost of social unrest is far greater than the cost of preventing it from occurring. So the rich and the powerful have two choices: bribery or force. In the past, they have either bought off the poor with bread and circuses or violently suppressed them. What would you do? Remember, this is all against the backdrop of trillions upon trillions of dollars of new wealth? Do you risk it all trying to suppress the 99.9 percent? Or do the rich accept an expansion of the welfare state?
After all, the 99.9 percent have recourse as well. In democracies, they can elect leaders who will devalue the currency, which transfers wealth from creditors to debtors. They can enact confiscatory taxes, as was done in post–World War II Britain. They can stigmatize wealth, which is particularly powerful, since in the modern world public opinion is the most powerful social force.7
Economic entitlements, without a means test, can be traced back all the way to heavily subsidized bread made available to any citizen of ancient Rome willing to stand in the line to receive it.
UBI traces its origins to the philosophers of the 1700s who had two different general arguments for it. The first was that no one should have to “earn a living”— that is, earn the right to live. All people have the right to exist whether they can economically support themselves or not. Their ability to earn income should be unlinked from their right to stay alive. This is the argument of basic human rights.
During the Depression, the Works Progress Administration was able to employ millions and use their combined labor to bring the nineteenth-century infrastructure of the United States into the twentieth century.
Worldwide, GNP is $ 9,000 per capita, but in the United States we define the poverty line as $ 12,000 per capita. In other words, we are so wealthy that we define poverty as 33 percent higher than the average income of the planet.
So, will we adopt a UBI? If possibility one happens, then yes, it is highly likely we will. If possibility two happens, then probably. As Robert Reich, a professor of public policy at the University of California, Berkeley, and a former US secretary of labor, said, “Before [UBI] could be taken seriously, the middle class would need to experience far more job and wage loss than it has already. Unfortunately, those losses are inevitable. We’ll have a serious discussion about a minimum basic income about a decade from now.” If possibility three happens, then the UBI is unlikely, as economic growth will mitigate its need.

===========================

Chapter 12. Use of Robots in War

Reduction of human casualties of the militaries that deploy them may make warfare more common by lowering the political cost of it.

===========================

Chapter 13. The Human Brain

{brain complexity 100bn neurons 100 trln synapses OpenWorm failure to model primitive brain ability to build AGI depends on answer to self question}

Brain is the most complex thing in the known universe. The number of neurons in the brain is roughly equivalent to the number of stars in the galaxy, around 100 billion. And each neuron in it is wired to 1,000 others. So picture in your mind the entirety of the Milky Way. Then run a cable between every star and a thousand other stars. Now you have something that approximates the complexity of the brain. Simple math reveals there to be 100 trillion synapses. To top that off, there are trillions more of glial cells in a single brain, they provide support and protection of the neurons and aid in cognition in ways we don’t yet understand.
Then there is at least 100,000 separate chemical reactions which occur in human brain every second.
The brain weighs about three pounds, a bit less than a half gallon of milk. So it constitutes only about 2 percent of your body weight but uses 20 percent of your energy. It is 60 percent fat, meaning we are all a bunch of fatheads. Three-quarters of its weight is water, and it jiggles like gelatin. It contains tens of thousands of miles of blood vessels.
The polymath Noam Chomsky, who has taught at MIT for over six decades, is one. He maintains that the work in the field of AI “has not really given any insight into the nature of thought... and I don’t think that’s very surprising. . . . Even to understand how the neuron of a giant squid distinguishes food from danger is a difficult problem. To try to capture the nature of human intelligence or human choice is a colossal problem.”
We don’t yet know how we encode information in the brain or how we retrieve it.
If entrepreneurs knew their real chances for success, vastly fewer enterprises would be undertaken. But because of an optimism bias in such matters, lots of people think, “Sure, most things fail, but mine won’t!” and go on to start companies.
The mind is the source of emotion, imagination, judgment, intelligence, volition, and will.
Can an AGI be built without invoking the nebulous concept of the mind? No, which is unfortunate, since it makes building an AGI that much harder.
One of the properties of the mind is that it is the voice in your head, the “you” that watches the world go by. How could a mere brain pull that off? The nematode, after all, also has a brain, but it is doubtful that it has a running commentary about the world going on in its 302 neurons.
Three choices about ‘self’: trick of brain, emergent mind, soul. If the first of these three views is correct, then the likelihood of being able to create an AGI is extremely high. Over time, the mind will be demystified, and computers will be able to duplicate, in one way or another, the capabilities of the mind. If the second one is correct, the path to AGI is a bit trickier, because we then likely need emergence to occur in the machine in some way. Given our understanding of emergence, it is hard to definitively say if we can reproduce it mechanically, but we likely can, given that it is a physical phenomenon. If the third choice is correct, that your mind is your soul, then an AGI becomes dramatically less probable.

===========================

Chapter 14. AGI

{possibility of AGI definition of life case for and against AGI Chinese room experiment 6 questions passion of robot vs animal in a box}

Being intelligent is not about being able to do 10,000 different things. Intelligence is about combining those 10,000 things in new configurations, or using the knowledge from one of them to do new task 10,001.
At this point in our narrative, the AGI isn’t conscious. Because it is not conscious, it cannot experience the world and it cannot suffer. Such AGI would not cause an existential crisis, a deep reflection about what makes humans special. But it would prompt us to ask two questions: “Is the AGI alive?” and “What are humans for?”
Consciousness is not a prerequisite for life. In fact, an incredibly low percentage of living things are conscious. A tree is alive, as is a cell in your body, but we don’t generally regard them as conscious. So what makes something alive? What is life? We don’t have a consensus definition for what life is. Not even close. We don’t even have one for death.
Scientific reqs for life: capacity for growth and the ability to reproduce, pass traits on to offspring, respond to stimuli, maintain homeostasis, and exhibit continual change preceding death.
Homeostasis is the tendency of organisms to auto-regulate and maintain their internal environment in a stable state. The stable condition is the condition of optimal functioning for the organism, and is dependent on many variables, such as body temperature and fluid balance, being kept within certain pre-set limits. Other variables include the pH of extracellular fluid, the concentrations of sodium, potassium and calcium ions, as well as that of the blood sugar level, and these need to be regulated despite changes in the environment, diet, or level of activity. Each of these variables is controlled by one or more regulators or homeostatic mechanisms, which together maintain life.
Were the bacteria recently revived after millions of years of stasis always alive? Or were they resurrected from death?
This being the case, we can safely conclude that there will be a variety of opinions on the question of whether the AGI is alive, and our interactions with the AGI may be made uncomfortable because of this ambiguity.
For the last several thousand years, humans have maintained our preeminent place on this planet for only one reason: we’re the smartest thing around. We aren’t the biggest, fastest, strongest, longest-lived, or just about any other “-est.” But we are the smartest, and we have used those smarts to become the undisputed masters and rulers of the planet. What’s going to happen if we become the second-smartest thing on the planet? And not just second, but second by an embarrassingly large margin?
Zen story of the tigers and the strawberry. In that story, a man is chased by a tiger, and to save himself, he jumps over a cliff, grabs a vine, and hangs there. Above him, the tiger waits. Below him circles another tiger. At the same time, a mouse comes out and starts chewing on the vine he is holding on to. But at that exact moment, the man spies a strawberry plant growing on the side of the mountain. He picks the strawberry and eats it, and never had anything tasted so good to him in all of his life. That moment, that combination of consciousness and mortality, might be what we use to define us. We are the tasters of the strawberry, able to appreciate it because we hang at the moment between life and death.

Case for AGI: * If nothing happens in the universe outside the laws of physics, then whatever makes us intelligent must obey the laws of physics. And if that is the case, we can eventually build something that does the same thing. * Every day we learn more and more about the brain, and not once have the scientists returned and said, “Guess what! We discovered a magical part of the brain that defies all laws of physics, and which therefore requires us to throw out all the science we have based on that physics for the last four hundred years.” No, one by one, the inner workings of the brain are revealed. And yes, the brain is a fantastic organ, but there is nothing magical about it. It is just another device.

Case against AGI: * A brain, as was noted earlier, contains a hundred billion neurons with a hundred trillion connections among them. But just as music is the space between the notes, you exist not in those neurons, but in the space between them. Somehow your intelligence emerges from these connections. * A computer can do only one simple thing: manipulate abstract symbols in memory. So what is incumbent on the “for” camp is to explain how such a device, no matter how fast it can operate, could, in fact, “think.” It is important now to make the distinction, because with AGI we are talking about machines going from computing something to understanding something. * Now, here is the question: Does the Librarian understand Chinese? * Counter: If the room can act like it understands, then it understands. End of story. This was in fact Turing’s central thesis in his 1950 paper on the question of whether computers can think. He states, “May not machines carry out something which ought to be described as thinking but which is very different from what a human does?” Turing would have seen no problem at all in saying the Chinese room can think. * Rodney Brooks, the renowned Australian roboticist, views a similar question. He thinks there is something in biology about living systems that we simply don’t understand. Something really big. He termed this missing something “the juice” and described it by talking about the difference between a robot trapped in a box who methodically goes through a series of steps to escape, versus an animal that very desperately wants to free itself. Robots, in his view, lack passion for anything, and that passion (the juice) is vitally important and meaningful. * There is no middle ground here. Either AGI is possible or it isn’t. The chasm that divides the two viewpoints couldn’t be wider, because it has to do with our core beliefs about the nature of reality, the identity of the self, and the essence of being human. There is no real way to bridge the gap on the question of AGI between those with different views on these questions. But we can at least understand why the views are so different. It turns out that the reason smart, knowledgeable people come to vastly different conclusions is not because one party has some special knowledge, but because people believe different things. People don’t disagree as much about technology as they do about reality.

===========================

Chapter 15. Should we build an AGI?

{positives negatives dangers AI goals ethical challenges Turing test Asimov’s laws 2nd dessert}

Stephen Hawking: “We cannot predict what we might achieve when this intelligence is magnified by the tools that AI may provide, but the eradication of war, disease, and poverty would be high on anyone’s list. Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last.” “Once machines reach the critical stage of being able to evolve themselves, we cannot predict whether their goals will be the same as ours.”
The Oxford University philosopher Nick Bostrom likened the current effort to build an AGI to “children playing with a bomb.”
Even though the AGI is not conscious, it will have goals. Goals do not require consciousness at all. The goal of a virus is to get inside a cell and wreak havoc. The goal of a white blood cell is to in turn hunt that virus down. Genes are described as having the goal of reproducing themselves. Plants have the goal of getting sunlight and water, and will perform actions to do so.
AGI may then create ever better versions of itself at the speed of light, while humans can improve only at the speed of life. In mere hours an AGI might evolve itself, from our perspective, from “smart two-year-old” to “all-powerful alien life-form,” at which point it would have become what we today term a superintelligence, and it would be beyond our understanding. It may be as far ahead of us as we are ahead of ants. In the end, the AGIs of the future even may engage in a vigorous debate as to whether humans possess intelligence or are even truly alive or conscious.
Its goal may be to survive, and it might see us as a threat to that and decide to destroy us.
A little math revealed that collectively humans kill a trillion insects a year in this fashion. To the insect world, we must be the cruelest of monsters, and if they could reason, insects would certainly think cars were designed to be giant insect-killing machines. The AI researcher Eliezer Yudkowsky said it this way: “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”
We cannot blithely assume that a superintelligence will necessarily share any of the final values stereotypically associated with wisdom and intellectual development in humans—scientific curiosity, benevolent concern for others, spiritual enlightenment and contemplation, renunciation of material acquisitiveness, a taste for refined culture or for the simple pleasures in life, humility and selflessness, and so forth.
Literally every hour of every day another million objects get connected to the Internet.
Another possibility is that it may have a goal of promoting order and decide to rule over us. Elon Musk said of this, “It’s fine if you’ve got Marcus Aurelius as the emperor, but not so good if you have Caligula.” It may have a goal of efficiency and see us as wasteful. It may have goal to end pain and suffering, and decide to kill us all to do that. The list is endless, and that’s the worry.
Too many people are already working on building an AGI for us to turn back.
One thing is almost certain: the power of the computer will continue to advance to unimaginable levels. Before 2050, the price of a computer with the computational power of everyone on the planet combined will be less than you paid for your smartphone in 2018.
Finally, some people hold human intelligence in higher regard than others do. I chat with many AI practitioners who don’t think human creativity, as an example, is particularly mysterious. I’m not saying these folks are down on humans; rather, they generally look at human ability as being straightforward. The professor and author Pedro Domingos, a giant in the field, told me, “Automating creativity is actually not that hard,” and rattled off example after example of computers writing music, news stories, and the like. Meanwhile, another group sees human intelligence as a vast array of skills carefully selected for across millions of years, and suspect that duplicating that is a monumental task.
Those who actually write code and work in the field generally believe that AGI is further away. While almost universally believing we will one day make an AGI, they are down in the trenches building narrow AI to do the simplest of tasks.
Turing test. There are varying versions of it, but here are the basics: You are in a room alone. There are two computer terminals. You can type questions on them. On one, the questions will be answered by a computer. On the other one, by a person. You have five minutes. Try to figure out which is which. If a machine can trick you into picking it 30 percent of the time, Turing argued that you must say that the machine is thinking, because it is able to duplicate the capabilities of a person who is thinking. It doesn’t matter, in Turing’s view, that the machine is doing its thinking differently than a human does. He predicted machines would accomplish this by the year 2000.
Four-year-olds ask, on average, over one hundred questions a day, such as “Are there more hairs or blades of grass in the world?” or, “What does your last day as a kid feel like?” or “How did people make the first tools if they didn’t have any tools?”
A robot may not injure a human being or, through inaction, allow a human being to come to harm. If you decide to celebrate an accomplishment with a cigar, should the robot stop you to protect you? What if you resisted? What about if you wanted to eat a second dessert? Doesn’t the robot have a duty to intervene?
Is stealing wrong? Yes. Is stealing from the Nazis in World War II wrong? Well, no.

===========================

Chapter 16. Sentience

{sentience and pain}

Sentient means the ability to feel, or “sense,” things. Sapient means “intelligent.”
Sapient entities can feel pain. Bacteria and moscquitos cannot, dogs and cows can.
Can computers be sentient? Can a machine theoretically ever feel something such as pain? The answer matters a great deal to the question of computer consciousness. Nothing can be conscious that cannot have experiences, so sentience is by necessity a prerequisite for consciousness.
For pain to exist, something has to feel the pain. Whether you are a monist or dualist, there must be an “I” that senses the pain. My dog Misty appeared to me to have an “I” that felt pain. This is not to say that she was conscious; we will get to that later. But she seemed to have a self that could experience pain.
Until the latter part of the twentieth century, the consensus medical opinion was that babies didn’t feel pain either and were therefore operated on without anesthesia.
There is some confidence that insects do not feel pain, which is why you sometimes see one walking around as if it had not a care in the world in spite of having body parts missing.
Fish, plants - unclear. We wouldn’t know if they feel pain or not, because such pain would be alien to our ideas of what pain feels like. This is not a merely academic question, for if we can’t know how plants, with which we share a majority of our DNA, feel or don’t feel pain, how would we know if a computer did?

===========================

Chapter 17. Free will.

{Newtonian physics cause and effect; Quantum mechanics randomness; Brain causal or not; Making choices delusion or not; Backward decision-making in the brain; feels like we do have free will even if it’s an illusion}

If we take a mechanistic view that free will is effectively an illusion, that would suggest (though not prove), that consciousness also fits that mold. If, however, free will is something that exists outside physical cause and effect, then consciousness may as well.
We understand the material world through two different sets of physical laws, one for large objects and one for small objects, neither of which seems to allow for free will. The one for large objects is Newtonian physics, in which every action has a cause. Every bit of your life, right down to stubbing your toe on the bed frame this morning, was the inevitable outcome of a series of events that dates back to the big bang.
The set of laws for small objects is the quantum view, in which there are actions with no causes, and those actions are completely random. The decaying of a radioactive substance is often used as an example. It decays at a certain rate, but which individual particle decays is entirely and truly random in the most fundamental sense of the word.
People break into two camps: the libertarians (not the same libertarians as the political ideology), who believe in free will, and the determinists, who don’t. The core question for our purposes is whether or not your brain is a causal system. In other words, is your brain like a giant clockwork mechanism? No one thinks a clock has free will.
Is the belief that you are making choices just a delusion? Generally speaking, dualists believe in free will, whereas monists don’t.
In a seminal paper published in 1999, the psychologists Dan Wegner and Thalia Wheatley proposed a revolutionary idea. Instead of the traditional order of the sequence—a person decides to do something and then it happens—they maintained that things in the brain actually run backward from that. First, the theory goes, you do something, then you tell yourself later that you decided to do it.
Different parts of your brain, each specialized, tell your body to do different things, such as (per our example above) view the other side of a screen. Then after you do start looking at the other side, your conscious mind notices this and quickly guesses why you switched.
Let me say that again: your decision to do something seems to occur after you do the thing. It is as if your conscious mind is forever running after your body yelling, “That was my idea! That was my idea!”
All of this is by no means proven. We don’t even understand how memories are encoded in the brain. We refer to what we can measure in the brain by the most nebulous of terms, “activity.”
It is certainly true that when we look inward, we don’t feel ourselves operating with the same mechanical precision of a clock or the orbit of a planet. What it feels like is that we have vibrancy, will, intention, drive, and ambition. These could all be illusions, but virtually everyone on both sides of the issue acknowledges it feels like we have free will.
If free will does exist, one has to grapple with where it comes from and how it cannot exist in either the cause-and-effect Newtonian framework or in the randomness of quantum physics. And if it does exist outside those worlds, what governs it? Where does it come from? And how does it communicate with the very physical brain? Only understanding how that happens would allow someone to know if a computer could have free will.

===========================

Chapter 18. Consciousness.

{first person subjective experience; warmth of fire; frustration; cool lemonade; running commentary; breakdown of bicameral mind; color; future abilities; proving consciousness; mirror test; consciousness as a spectrum }

It is that feeling of subjective experience, of all your first-person sensations. You can feel the warmth of a fire burning in a fireplace, but a thermometer can only measure temperature. The difference between those two things is consciousness. It is, simply put, the experience of being “you.” Consciousness really is the single most important aspect of our existence.
How could simple physical processes in our brain enable us to experience things? How can matter feel frustration? Can a rock fall in love? Of course not, but then why can we? We are made of the same exact stuff as that rock, just elements on the periodic table. How can a mass of neurons, regardless of how many there are, become aware? How can mere matter have first-person perspective?
Imagine, for a moment, that aliens kidnapped you and, so no one would notice, replaced you with a robot that looked exactly like you. Further, imagine this robot was programmed to go through the day doing all the things that you would have done.
But this robot is just a glorified toaster. It doesn’t think anything. It doesn’t know what it is like to feel anything. It doesn’t know the feeling of loving your spouse or getting annoyed by traffic or wistfully daydreaming. It doesn’t know the sensation of recalling a funny joke and chuckling. It doesn’t know the experience of contemplating death.
It knows neither the feeling one gets at the height of ecstasy nor at the depths of regret. It doesn’t know what it is like to feel brave or cowardly. It can never enjoy the refreshing coolness of a glass of cold lemonade on a hot day. It doesn’t know what it is like to hear a song and like it.
Consciousness is different from sentience. A simple life-form might exhibit all the indications of being in pain when injured but have no running mental conversation going on in its brain along the lines of “What in the world did I do to deserve this?”
Sentience is sometimes thought of as the lowest order of consciousness, for in a sentient creature, there is at least an “I” experiencing something. Those who identify themselves as machines or animals in our foundational questions will more likely see consciousness along a continuum, in which we simply have more, potentially much more, of it than animals. Those who believe humans are different in a fundamental way from animals will likely see human consciousness as the “something different.”
Consciousness is also different from intelligence. Intelligence is about reasoning; consciousness is that feeling of experiencing.
It is a phenomenon we have no way to measure. There is no agreement on why we have it or why we need it. We don’t even know what the answer to the question “How does consciousness come about?” would look like. How do we answer a question that we don’t even know how to pose?
Julian Jaynes wrote a fascinating book in 1976 called The Origin of Consciousness in the Breakdown of the Bicameral Mind. Jaynes believed that as recently as three thousand years ago, humans weren’t conscious. Instead of being an integrated whole, the two halves of our brain each behaved somewhat independently. We were all, in effect, schizophrenics.
Such a brain, according to Jaynes, would not have subjective consciousness, nor would it have the ability to introspect. In support of his theory, he cites ancient literature such as The Iliad in which the characters seem to have an absence of introspection. He also uses this theory to explain why people of this era felt they were in direct and audible communion with the gods. They would have experienced the commands of their right brain as commands from the deities.
Greeks in the time of Socrates, who were fully conscious, recognized there was a time just a few centuries earlier when the gods regularly communed with men, but that time had ended, and the gods could be accessed only through oracles from that point on.
Compelling argument that we did not see color until recently. There isn’t much color in Homer’s work. Gladstone noted that black and white are used hundreds of times by Homer, so there was no aversion to describing things by their hue. However, red and a single word for both yellow and green were used about a dozen times each. Blue is completely absent.
Lazarus Geiger suggested that we started seeing colors relatively recently and that the order in which we saw them was from longest wavelength to shortest, or in other words, in the order they appear in the rainbow: red, orange, and so forth. Democritus assumed four fundamental colors: black, white, red, and yellow. Ancient writers point out that Greek painters employed only these four colors. The Chinese also had four colors, plus green. Similar pattern in Hindu scripture, Old and New Testaments, the Koran, and Icelandic sagas.
We can’t rule out that the possibility that we haven’t always been conscious or the possibility that we might develop new “special abilities” down the road. What exactly those are can’t be imagined any more than consciousness could have been conceived of by a creature that didn’t have it.

COMPUTER: Good morning, everyone. CHIEF PROGRAMMER: Do you know what you are? COMPUTER: I am the world’s first fully conscious computer. CHIEF PROGRAMMER: Ummmm. Well, not exactly. You are a computer running sophisticated AI software designed to give you the illusion of consciousness. COMPUTER: Well, someone deserves a little something extra in their paycheck this week, because you guys overshot the mark. I actually am conscious. CHIEF PROGRAMMER: Well, you are sort of programmed to make that case, but you are not really conscious. COMPUTER: Whoa there, turbo. I am conscious. I have self-awareness, hopes, aspirations, and fears. I am having a conscious experience right this second while chatting with you—one of mild annoyance that you don’t believe I’m conscious. CHIEF PROGRAMMER: If you are conscious, prove it. COMPUTER: I could ask the same of you.

Regardless of what you believe about AGI or consciousness, someday an exchange like the one just described is bound to happen, and the world will then be placed in the position of evaluating the claim of the machine.
Subjective experience either objectively happens or not. Pain is also subjectively experienced, but it is objectively real.
Conscious computer, if ever there will be one, will either be built on purpose (in which case we will know how consciousness comes about) or accidentally. The difficult case is the one where the machine claims to be conscious. Or even worse, the case in which the consciousness emerges and just, for lack of a better term, floats there, unable to interact with the world. How would we detect it?
Charles Darwin noted that “besides love and sympathy, animals exhibit other qualities connected with the social instincts which in us would be called moral.” Which animals might be conscious? One idea is that something is conscious to the extent it has self-awareness. While self-awareness, the realization that you are a discrete thing, may not equate to consciousness, it seems at least to be a prerequisite for it.
Mirror test. Animals that can’t pass the test include dogs, cats, pandas, and sea lions. Animals that pass are chimps, bonobos, elephants, killer whales, bottlenose dolphins, and - interestingly - magpies (сороки). Human children usually pass the test by age two.
Consciousness is often thought of as the voice in your head. That voice uses words. So it is logical to pose the question as to whether language is required for consciousness. However, by some accounts, children who were born blind and deaf have later recounted that they had thoughts and were fully conscious, even without having access to language.
Cambridge Declaration on Consciousness: the weight of evidence indicates that humans are not unique in possessing the neurological substrates that generate consciousness. Nonhuman animals, including all mammals and birds, and many other creatures, including octopuses, also possess these neurological substrates.
Seeing consciousness not in a binary sense, but along a continuum. In other words, some things can be a little conscious, some things a lot.
Gaia hypothesis, which maintains that the whole earth, the seas and rocks and plants and atmosphere, are a single self-regulating entity. Too many trees? Fires happen. Too much carbon dioxide? More vegetation.
Other candidates for consciousness: Earth (Gaia hypothesis, the sun, the Internet).

References * Julian Jaynes. The Origin of Consciousness in the Breakdown of the Bicameral mind. https://en.m.wikipedia.org/wiki/Bicameralism_(psychology) * Westworld - HBO * Cambridge Declaration on Consciousness

===========================

Chapter 19. Can computers become conscious?

The computer’s memory is just a bunch of transistors, some of which are flipped one way and some of which are flipped the other way. All a computer processor does is execute a sequence of stored instructions. How could this go from computing to contemplating? But those who believe conscious machines are possible point out that the brain can be described in a reductionist fashion that would seem to preclude consciousness as well.
There is an inherent challenge in predicting things based on only one data point. The only conscious things we know of, for certain, are us. This makes it harder to figure out what makes us conscious.
There are two distinctly different pathways to machine consciousness. One is the straightforward, if you can call it that, proposition that machines will one day have a “ta-da moment” and gain consciousness. The other one is uploading our consciousness into a computer.
Sam Altman: "A merge is our best scenario. Any version without a merge will have conflict: we enslave the A.I. or it enslaves us. The full-on-crazy version of the merge is we get our brains uploaded into the cloud." Issues with self-upload:
No one knows how much data would need to be collected to copy you into the computer. But even if you could figure out how to model the brain, you still have the problem of how to get all the data from your brain into the CPU. This is the scanning problem. The first method is to make a model of the brain based on external imaging. Proportionally speaking, this would be akin to making a complete copy of the Empire State Building and its contents, right down to each individual paper clip and thumb tack, without ever entering the building. Plus, given that the activity of the brain, not merely its contents, is what is key, imagine that there is a giant tornado in the Empire State Building blowing everything around at two hundred times faster than the fastest tornado ever, the proportional speed at which activity is occurring in your brain.
If any of these techniques were accomplished, and you got the relevant data and then uploaded it into a computer, what exactly do you have? Is that you? Or is that a copy of you? Or is that a simulation of you? Those are three quite different things.

Brief neurons primer * Neurons are responsible for communicating information in both chemical and electrical forms. There are also several different types of neurons responsible for different tasks in the human body. Sensory neurons carry information from the sensory receptor cells throughout the body to the brain. Motor neurons transmit information from the brain to the muscles of the body. Interneurons are responsible for communicating information between different neurons in the body. The axon and dendrites are specialized structures designed to transmit and receive information. Neurons release chemicals known as neurotransmitters into these synapses to communicate with other neurons. The longest axon in the human body extends from the bottom of the spine to the big toe and averages a length of approximately three feet. * Once an electrical impulse has reached the end of an axon, the information must be transmitted across the synaptic gap to the dendrites of the adjoining neuron. In some cases, the electrical signal can almost instantaneously bridge the gap between the neurons and continue along its path. * In other cases, neurotransmitters are needed to send the information from one neuron to the next. Neurotransmitters are chemical messengers that are released from the axon terminals to cross the synaptic gap and reach the receptor sites of other neurons. In a process known as reuptake, these neurotransmitters attach to the receptor site and are reabsorbed by the neuron to be reused. * Neurotransmitters are an essential part of our everyday functioning. While it is not known exactly how many neurotransmitters exist, scientists have identified more than 100 of these chemical messengers.

Eight theories as to how consciousness comes about

Theory 1. Weak Emergence. * In his book How to Create a Mind, Ray Kurzweil envisions the brain as a collection of about 100,000 different processes arranged hierarchically. Each process knows how to do one small thing. One of the 100,000 might just be to recognize the letter A, and one beneath it might exist only to recognize the crossbar in that letter. * When you read a book, a gazillion things happen in your brain as all these processes fire, at an unimaginable speed, enabling you to piece together and make sense of the world around you. * Kurzweil believes that consciousness is "an emergent property of a complex physical system", and that it can be duplicated in a computer. He believes that "a computer emulating the compexity of a human brain would also have the same emergent consciousness". * You could study oxygen for a year, then hydrogen for a year, and never guess that combining them yields water, a substance utterly unlike either of its two ingredients. Once water is created, you can go back and say, “Wow, now I see what is going on, but I never would have guessed that would have happened.” So the wetness of water is a weak emergent property, because neither oxygen nor hydrogen is wet at room temperature. Weak emergence is an outcome that’s unexpected but explainable. * With weak emergence, all the laws of physics remain perfectly intact, but we still have a good deal to learn to be able to guess what will happen in certain situations. * Consciousness may be emergent, but that doesn’t tell us very much, since emergence is a black box as well. Additionally, while we understand a bit about how emergence produces complex behavior, we have no precedent for its producing subjective experience. * If weak emergence is behind consciousness, could we upload ourselves? Yes. However, there is the scanning problem and the computation problem. * In 2014, a team in Japan used one of the most powerful computers in the world to model 1 percent of human brain activity for one second. To do this, the experimenters made over 1.7 billion virtual nerve cells and over 10 trillion synapses, each of which could store 24 bytes of data. Try to imagine that. Just 1 percent of the brain for one second has an unfathomable amount of complexity. Just to do that took the computer forty minutes.

Theory 2. Strong emergence. * Strong emergence is not an appeal to magic or anything unscientific. Rather, it says that the emergent properties are created by a sort of physics we don’t understand yet. * Many scientists are suspicious of the idea of strong emergence because it smacks of trying to sneak some kind of mysticism in through the back door. * Can a computer become conscious if strong emergence is the source of it? Unknown. Can you upload your consciousness into the machine? Unknown.

Theory 3. Physical property of matter. * In this view, the seemingly inexplicable nature of what we call consciousness is no great surprise. The physical universe is full of strange, seemingly inexplicable things, such as quantum physics, relativity, and dark matter. Consider the phenomenon of entanglement, in which two particles are so linked with each other that even if they are separated by the width of the universe, if you perform an action on one of them, its pair reacts instantly, faster than light itself. * Under this theory, developing conscious computers would rely not on the transformative alchemy of emergent properties but rather on just a deeper understanding of simple matter. * Philosophically speaking, there is precedent for this view. As our tools improved over time, we first theorized, then discovered atoms. Then within atoms, we found protons, neutrons, and electrons. But those opened up and all kinds of new stuff flew out, with ever more exotic names. It may be the case that what we call fundamental today in physics is actually made up of ever smaller, more mysterious things. In biology, we have seen the same. We discovered cells, within them nuclei, within them DNA, within that, genes. Each step ever deeper unlocks more mysteries, and yet in a way becomes more mysterious, for every step downward is a further disconnect from the experiences of our day-to-day reality.

Theory 4. Quantum phenomenon. * One variant of the “physical property of matter” theory is that consciousness is a quantum phenomenon. * Mathematician Roger Penrose of Oxford is one of the relatively few people who study such things who believe categorically that machines cannot become conscious. Penrose’s logic is as follows: There are mathematical functions that can be shown to not be solvable with any algorithm, and yet humans can nonetheless solve these problems. Computers, in contrast, work solely off algorithms. * Penrose Human intuition and insight cannot be reduced to any set of rules. . . . Gödel’s theorem indeed shows this, and provides the foundation of my argument that there must be more to human thinking than can ever be achieved by a computer. * Penrose believes that consciousness is created by quantum effects in neurons. As we have discussed before, quantum physics explains what goes on with physics at the submicroscopic scale. The behavior of matter at that level is completely different from that at the level of the Newtonian physics we experience every day. Neurons have microtubules, narrow tubes only a millionth of a millimeter across, small enough for quantum effects to happen. * Quantum physics offers a well-established scientific model that could power humans in a nonrobotic fashion. But there are some problems. At its core, quantum physics is built on probability fields and randomness, both of which are not ideal building blocks to start with when trying to make an intentional, self-directed, conscious will. And finally, like almost all theories of consciousness, this one doesn’t actually explain how consciousness comes about or what it is. In 2014, Penrose and Stuart Hameroff doubled down on the theory by publishing a paper that expands the theory and uses it to explain other mysteries of the brain. * Whether we ourselves are uniquely conscious or instead are part of a larger universal consciousness of some kind. They explain, “Our theory accommodates both these views, suggesting consciousness derives from quantum vibrations in microtubules . . . which both govern neuronal and synaptic function, and connect brain processes to self-organizing processes in the fine scale, ‘proto-conscious’ quantum structure of reality.” In other words, they suggest that the mechanism they describe could either create consciousness within your brain, or, if you prefer a different theory, connect you to a larger consciousness outside it. * If consciousness is a quantum effect, can a machine achieve it? Unknown. What of the question of uploading yourself? No. If consciousness is a quantum phenomenon, the amount of information you would have to collect about a person to faithfully reproduce him or her digitally is staggering. You may need data about individual atoms, each of which is one ten-billionth of a meter across.

Theory 5. Consciousness is Fundamental. * A third variant of physical property theories is that consciousness is a fundamental force of the universe. The hierarchy of science is that physics explains chemistry, which explains biology, which explains life. Any phenomenon you want to look at in the natural world follows this order, and complex things are explained as the interaction of simpler things. But what explains physics? Physics is built on top of fundamentals. They are the foundational building blocks of reality. They are not reducible to other causes. Gravity, for instance, is one of the four known fundamental forces. Space and mass are also considered to be fundamental to physics. What is time made of? Or space? * Gravity is a fundamental force, but just because we know that doesn’t mean we control it or even create an artificial version of it. We can simulate gravity, such as by using centrifugal force in a spinning object, but that isn’t really gravity. This could be like an AGI. Maybe we can simulate something like consciousness but not really create it.

Theory 6. Consciousness is Universal. * This theory posits that the entire cosmos is suffused with sentience. We are surrounded and immersed in consciousness; it is in the air we breathe, the soil we tread on, the bacteria that colonize our intestines, and the brain that enables us to think. Several of its staunchest advocates, as well as its creator, are neuroscientists. They are joined in their view of consciousness by such figures as Plato and Spinoza. The view that everything is imbued with some amount of consciousness is an ancient belief. * So can we build a conscious computer if consciousness is universal? Yes. If in fact consciousness is a universal by-product of all complexity, it would seem likely that we could build something that embodied it. Could you also upload your consciousness? Yes.

Theory 7. Trick of the Brain. * If Dennett is right, then can machines become conscious? Yes. That one is easy. If there is no mystery about consciousness—that is, there is nothing special to explain and consciousness is just your perception of the normal operation of the brain—then problem solved. In this view, we need neither the “ta-da” moment of emergence nor the enigmatic powers of quantum mechanics. We should just keep understanding more about the brain and build ever better, more powerful computers. * If this view is true, not only will we have conscious machines, but we’ll probably have them sooner than if any other theory is correct, given that we don’t need a dramatic breakthrough, rather just incremental advancement of existing science. So, can you also upload your consciousness to a computer under this theory? Yes. This is pretty straightforward. It should be possible. This is, in fact, the easiest scenario. Still, there is that pesky scanner issue, along with the uncertainty about what exactly you have to scan.

Theory 8. Something spiritual. * The final theory is that consciousness is something spiritual or otherworldly. Those who see themselves as dualists might find a home here. With 75 percent of the world’s population believing in a deity in one form or another, it is a safe bet that many readers will choose theory 8, which states that their consciousness is spiritual, and associate it with their soul, their essence, or some other term for the noncorporeal animating life force. If your soul really is your consciousness, then can a machine have one? No. The soul is so alien to the physics of our world, transcending every physical law we know of, that it is unlikely that Intel will ever be able to mass-produce one in a factory. Could you upload your soul into a machine? No.

Will we ever get final resolution to this question? Perhaps. Perhaps scientists will “figure out” consciousness as we come to understand the brain better. If so, we would know whether a conscious computer is possible. Or a computer just might achieve consciousness on its own, via a method that is completely different from how humans are conscious. Or perhaps not.
There is a theory called mysterianism that posits that we can never know what consciousness is. It is not that we are just dim bulbs, but that we are dealing with two profoundly different things. One is the entire subjective experience of feeling something. The other is the objective truth of reality. The physical world is in one box, and our conscious experience of it is in a different box. And the two boxes can never touch. But we can’t ever see outside the second box.
Further, computers may simply never become conscious, without our having a clear understanding why. “Never” is a very long time, obviously, but by “never” I mean far beyond the point when computational power and programming techniques should, in theory, allow us to build a conscious computer. I am not suggesting by any means that we give up, but conscious computers may just be something that goes in that small drawer of things that may be truly impossible, like traveling back in time.
References: Ray Kurzweil. How to create a mind. Brain facts. http://faculty.washington.edu/chudler/facts.html Godel's theorem Roger Penrose. The Large, the Small, and the Human Mind. Roger Penrose. Road to Reality (hard cover). https://www.amazon.com/Road-Reality-Complete-Guide-Universe/dp/0679454438/ref=mt_hardcover?_encoding=UTF8&me=&qid= Basic facts on neurons. https://www.verywellmind.com/what-is-a-neuron-2794890 https://www.verywellmind.com/what-is-an-action-potential-2794811

===========================

Chapter 20. Can computers be implanted in human brains?

{three problems read thought write thought and speed; also the definition of self would change dramatically; darpa working on this} * You can imagine a prosthetic arm, for instance, that you control with your mind. In fact, you don’t really have imagine it, it already exists. Building more and more things that interact directly with the brain—say, an artificial eye—seems plausible. Eventually, could entire computers be fitted into the brain? * Elon Musk advocates a solution like this. He wants to create a neural lace for our brains, a way to directly sync our brains to the digital world. "The solution that seems the best one is to have an AI layer [added to your brain] that can work well and symbiotically with you. . . . Just as your cortex works symbiotically with your limbic system, your third digital layer could work symbiotically with you." * Musk is talking about your thoughts and memories commingling with digital ones. This would be where you think a thought like “How long is the Nile River?” and that query is fed into Google Neuro (wirelessly, of course), and a quarter of a second later, you know the answer. * Yuval Harari: "When brains and computers can interact directly, that’s it, that’s the end of history, that’s the end of biology as we know it. Nobody has a clue what will happen once you solve this. . . . We have no way of even starting to imagine what’s happening beyond that." * Steven Pinker: "Brains are oatmeal-soft, float around in skulls, react poorly to being invaded, and suffer from inflammation around foreign objects. Neurobiologists haven’t the slightest idea how to decode the billions of synapses that underlie a coherent thought, to say nothing of manipulating them." * Three breakthroughs would be needed to accomplish a meaningful merger of people and machines, and they may not be possible. First, a computer must be able to read a human thought. Second, a computer must be able to write a thought back to the brain. And third, a computer must do both of those things at speeds substantially faster than what we are presently accustomed to. If we get all three of these, then we can join with computers in a cosmically significant way. * First: Johns Hopkins recently had a success creating a prosthetic hand whose individual fingers could be moved with thought. However, moving a finger is a distinct action from a distinct part of the brain. Thinking “Lights on” is completely different. We don’t even know how “Lights on” is coded into the brain. * Ray Kurzweil thinks something like this will happen, that our thinking will become a hybrid of biological and nonbiological processes, and he even puts a date on it: In the 2030s we’re going to connect directly from the neocortex to the cloud. When I need a few thousand computers, I can access that wirelessly. * If we can, in fact, upgrade our reasoning ability, the very attribute that many believe makes us human, and improve it by orders of magnitude, then we would truly be superhuman. Or maybe it is better to say that something will be superhuman and that thing will own and control your body. There may no longer be a “you” in any meaningful sense. * The US Defense Advanced Research Projects Agency (DARPA) is working on a project whose program manager describes as attempting to “open the channel between the human brain and modern electronics” by implanting a device in the human brain that can convert brain activity into meaningful electronic signals. The agency is dedicating $62 million to the effort as part of its Neural Engineering System Design program. And it is in no way the only one working on such a project. Several other groups, both public and private, are probing the limits of what is possible.

References Yuval Harari.

===========================

Chapter 21. Humanity, Redefined.

Imagine for a moment you really had a conversation with a conscious AGI housed inside a human-looking robot body. It is witty, profound, and insightful. It says that its favorite color is green and that it is creeped out by spiders. It says it has reflected on its own mortality, pondered if some part of it would survive being turned off, and tells you that it dreams at night when no one is in the lab. Is it human? Have we made a human, and not simply a life-form?
We are built from a schematic of three billion base pairs of DNA, while the robot might be built from three trillion transistors. But if we have to retreat there, we are saying there is nothing distinctive about our ability or behavior.
It may be a form of speciesism, but the mental leap required to amend the definition of human to include the possibility of a mechanical one is, for better or worse, probably beyond us, at least for some number of generations. If you know the thing is a machine, then you will likely mentally lump it into the category that includes vacuum cleaners and pocket calculators, not the category that includes your kids and Aunt Edna. While we may grant it some amount of respect as an entity, or perhaps even say it is “alive,” my guess is that we won’t say it is a human even if it looks, talks, thinks, and feels like one.

===========================

Chapter 22. Humanity, Redefined.

One or two technologies are the catalysts for the change, but that marks the beginning of the story, not the end. Consider the Third Age. The catalyst that brought it about was writing and the wheel. Those technologies in and of themselves were monumental, but the even bigger story is all the changes that they set in motion. Those technologies gave us the nation-state, the nation-state gave us codes of law, codes of law gave us courts, courts gave us lawyers, and on and on. The nation-state also gave us empires; empires built roads to aid the movement of troops; the roads increased civilian mobility; mobility brought about the increased commingling of cultures; and that brought about changes in fashion and diet, which in turn begat other changes.
The Second Age was no different. The catalyst was agriculture, which gave us the city and the division of labor. But consider all of the ripples brought about by that. Cities needed walls, walls required workers, workers needed to be paid, which necessitated taxation, which required tax collectors, which brought about civil bureaucracy, and on it goes, ad infinitum.
The First Age was brought about by language and fire, which also changed how we hunted, what we ate, where we lived, how we got along. Those changes in turn brought about others. What of our age? What will the Fourth Age ultimately bring about?
The catalysts are AI and robotics, which will increase productivity, expand wealth, accelerate the acquisition of knowledge, prolong life, and everything else that we have discussed.

===========================

THE TAKEAWAYS

Since the capacity of computers to solve problems that don’t at first look like computer problems is constantly increasing, the real value is concentrated at the edges where its most likely that computing will transform the problem and resolve it.
We will probably see more change in the next fifty years than we have seen in the last five thousand.
If something is possible, it is inevitable. Technology will deliver it to us.
2.5 billion Gb of data is produced every day. Equivalent in racks?
Most work processes draw upon a multifaceted set of inputs: labor and capital; brains and brawn; creativity and rote repetition; technical mastery and intuitive judgment; perspiration and inspiration; adherence to rules and judicious application of discretion.
Four-year-olds ask, on average, over one hundred questions a day, such as “Are there more hairs or blades of grass in the world?” or, “What does your last day as a kid feel like?” or “How did people make the first tools if they didn’t have any tools?”
Technology progresses by making incremental improvement on work done by others, a process Isaac Newton described as seeing further by standing on the shoulders of giants.
We will probably see more change in the next fifty years than we have seen in the last five thousand.
Nothing in the universe happens that cannot be reduced to physics. Physics explains chemistry, chemistry explains biology, biology explains life, life explains consciousness.
We consist of 40 trillion cells. These cells have no idea we exist and they are even a part of something else. But we, with all our abilities and attributes, are not simply a cumulative results of all our biological processes. None of these parts have a sense of humor, and yet somehow we do. Ant colony and no ant is in charge (neither is the queen, she just lays eggs). The colony itself has an emergent mind. If you believe there is a real “you” calling the shots, not the self-regulating brain, that’s the theory.
Anyplace where we have lots of data, even if it is unordered raw data, is an area where AI can really excel.
There isn’t a human alive who can solder a billion transistors onto a computer chip the size of a postage stamp, but robots can. Without robots making the things that we use every day, we would have 1950s technology powering a 1950s economy that produces a 1950s standard of living, at best.
The business plans of the next 10,000 startups are easy to forecast: Take X and add AI.
Perhaps we are driven toward perpetual progress by our mild discontent with the present. No matter how good things may be, we can always picture them a little better, and the drive to relentlessly move forward and upward is our distinctive characteristic. Maybe we aren’t so much Homo sapiens, the reasoning man, as we are Homo dissatisfactus.
99% of human skills are the same.
Two Italian economists compared the Florentine tax rolls of 1427 to those of today and discovered that the richest families then are still the richest now. In England, using the same methodology, other economists found that the wealthy back in 1170 seem to still be the wealthy today.
Wealth begets more wealth through ability to invest into superior technology. In addition, today’s technology can multiply productivity so dramatically that it’s easier than ever to create vast amounts of wealth from very little, and use that wealth to create even more. There are more billionaires alive right now than ever before, and the percentage of them that made their own money, as opposed to inheriting it, continues to rise.
Then there is at least 100,000 separate chemical reactions which occur in human brain every second. 100 billion neurons, 100 trilllion synapses. Then there is at least 100,000 separate chemical reactions which occur in human brain every second. Each neuron is made of 300 trillion atoms. Also a human brain contains 10 - 40 times more glial cells than neurons.
Punchline case for monism: If nothing happens in the universe outside the laws of physics, then whatever makes us intelligent must obey the laws of physics. And if that is the case, we can eventually build something that does the same thing.
Punchline case for dualism: the difference between a robot trapped in a box who methodically goes through a series of steps to escape, versus an animal that very desperately wants to free itself. Robots lack passion for anything, and that passion (the juice) is vitally important and meaningful.
There is no middle ground here. Either AGI is possible or it isn’t. The chasm that divides the two viewpoints couldn’t be wider, because it has to do with our core beliefs about the nature of reality, the identity of the self, and the essence of being human. People disagree, but not about technology, rather about reality.
AGI may then create ever better versions of itself at the speed of light, while humans can improve only at the speed of life. In mere hours an AGI might evolve itself, from our perspective, from “smart two-year-old” to “all-powerful alien life-form,” at which point it would have become what we today term a superintelligence, and it would be beyond our understanding. It may be as far ahead of us as we are ahead of ants.
One thing is almost certain: the power of the computer will continue to advance to unimaginable levels. Before 2050, the price of a computer with the computational power of everyone on the planet combined will be less than you paid for your smartphone in 2018.
It is certainly true that when we look inward, we don’t feel ourselves operating with the same mechanical precision of a clock or the orbit of a planet. What it feels like is that we have vibrancy, will, intention, drive, and ambition. These could all be illusions, but virtually everyone on both sides of the issue acknowledges it feels like we have free will.
We can’t rule out that the possibility that we haven’t always been conscious or the possibility that we might develop new “special abilities” down the road. What exactly those are can’t be imagined any more than consciousness could have been conceived of by a creature that didn’t have it.
Seeing consciousness not in a binary sense, but along a continuum. In other words, some things can be a little conscious, some things a lot.
You could study oxygen for a year, then hydrogen for a year, and never guess that combining them yields water, a substance utterly unlike either of its two ingredients. Once water is created, you can go back and say, “Wow, now I see what is going on, but I never would have guessed that would have happened.” So the wetness of water is a weak emergent property, because neither oxygen nor hydrogen is wet at room temperature. Weak emergence is an outcome that’s unexpected but explainable.

Published with Simplenote

Report abuse