Tuesday, December 18, 2007

Abstraction Salesman

I'm always suspicious when there's a knocking on my door. Everyone that I would want in my house knows not to knock. So if someone is knocking on the door, it's either the Mormons, someone begging for change, or some other annoyance. I was tempted to pretend that I wasn't home, a tactic which consists of me continuing to do whatever I was doing. Instead, on the theory that it might possibly be the police, I got up and swung the door open.

Standing there was a man of just shorter than average height. I note that only because I sit at the very top of the bell curve, so I when I see that someone is taller or shorter than me, I immediately put it in the perspective of statistics. He was wearing a blue suit and had this factory fresh look about him, the look that toys have before they get played with. His left hand was holding a briefcase; his right hand was thrust out towards me, a smile on his face.

"Hi sir, my name is John, I'd just like a moment of your time!" He talked almost without breathing, and he seemed happy, like he really would like a moment of my time. I looked him over, not taking his hand. He noticed, but didn't seem to care.

"What are you selling?" I asked.

"Oh, ideas sir!" The exuberance he gave off was almost painful. "I sell ideas."

"You mean books? Action-item plans, self-help, stuff like that?" I wished that I had brought a cup of coffee with me to the door, so that I could sip it with an air of nonchalance and pretend that I had better things to be doing. I didn't.

"No sir," he said with a troubled look. "I mean I sell ideas." He gestured vaguely. "You know."

"No, I don't know."

"Like ... the idea of a chair, right? Would you like to buy chair?"

"I have chairs."

"No, I don't mean a chair, I mean the idea of a chair."

I looked him up and down. "What would I do with the idea of a chair? And why should I pay you for an idea? Ideas are free."

"Well ... yes and no. Thinking about ideas is free, but the ideas themselves are worth more than your dreams."

"How much are my dreams worth?"

"Well that depends on the quality of the dream - but look, I'm not here to buy things from you, I'm offering you the opportunity to buy quite valuable intangibles."

"But what would I do with them?"

"I don't know - that's up to you. You could share them, or combine them to make new ideas, or put them on your mantle for a conversation piece."

"I would put something intangible on my mantle?"

"Metaphorically, yes. But if you'll just have a look at what I'm offering, I'm sure that you'll find something you'd like. May I come in?"

I gave a look behind me, as though seriously considering the idea. "Eh - I've got other things going on. A lot of work to do. You know how it is."

"Alright, but here's my card - if you change your mind, I do make housecalls, ha ha." He pulled the card out of his breast pocket and held it out for me; I took it reluctantly. He headed off down the street, passing the other houses entirely. I looked down at the card - it didn't have any contact information on it, instead it simply read "John Johnson - Ideas For Sale". When I looked up, he was gone.

Free Newspapers

I just watched a web-talk given by Chris Anderson (online here) that sort of blew my mind. The last thing to really do that was Everything is Miscellaneous. Which, I'll grant you, I read about a week ago - so either my mind is easily blown, we live in mind blowing times, or I've stumbled onto these two things for related reasons. I've heard of Anderson's book The Long Tail before, but it sort of went on my list of high concept books that I should read. Having finished Everything is Miscellaneous and started on Wikinomics and The World is Flat, as well as having The Tipping Point sitting on my bookshelf, I had thought that I would have a short break. Alas, it look like The Long Tail is getting in line too, so I might be a few weeks from fluff reading.

That's not what I wanted to talk about though. There was something that he said during the talk that I've always been fascinated with, and it's this; those little boxes that have the newspapers inside them. You stick a quarter in, and the box opens up. What's to prevent someone from pulling out all of the newspapers and reselling them? Or using that whole stack for paper mache? Or peeing on them? It's a complex system of incentives at work, and one of the reason those boxes are really interesting to me.

The newspaper company isn't really selling newspapers. They're selling eyeballs to advertisers. Therefore, it's in their best interests to get as many eyeballs on their papers as possible without regard to journalistic integrity. I think they've instead found that most of the time, journalistic integrity is what sells papers. That depends on your view of how fair and accurate the new media is though. I lean towards pretty fair, most of the time. Either way, it's not in their best interests to limit their audience with high prices - that's why newspapers are disproportionately cheap when compared to the cost that goes into making them. As cost goes down, the number of readers goes up.

So why don't they just make newspapers free? Ah, well that's because the newspaper* is selling eyeballs to companies. And just as the number of readers is inversely related to the cost, the interest of the individual is proportionally related to what it cost them. In other words, if a guy were paying $10 for a newspaper, you could be pretty certain that he's going to read through the whole thing. By instituting a cost, the newspaper can weed out most of the people who would just pick up the paper and not read it. Check out this awesome chart with completely fake numbers:

Aside from the fact that it was created in Excel in about two minutes, and that it's made from data that was created by me to prove a point, it shows that we get too many people picking up newspapers and not reading them when the price is too low. And not just people not reading them either - we also get people that pick up the paper for a single story or a single section instead of going through the whole thing. The newspaper should then try to find a cost which maximizes eyeballs while minimizing the number of newspapers that aren't read. And of course there's income associated with people paying for the paper, even if it's just a fraction of the income generated by advertising.

But while the newspaper is juggling wasted papers, readership, and income, there's another big factor that isn't immediately apparent; coins. I regularly read USA Today, because I have lots of time and my college provides them free to students and staff. This reduces my actual interaction with the newspaper boxes to zero. Anyway, it normally costs 75 cents. Does it cost that because 75 cents is the amount that properly juggles the three main factors of price in a way most in-line with the needs of USA Today? Is it because 75 cents is more optimal than 74 or 76 cents? No. It's because 75 cents is three quarters. If it cost 70 cents or 80 cents, the person on the street would have to have a much more specific set of coins. So the choice comes in multiples of 25 cents, which must reduces efficiency but speeds up the decision making process.

*It's odd that "newspaper" stands in for both the company and the physical item. Unless it doesn't and I've made a grammatical faux pas.

The Rich Man's Burden

I recently gave a presentation on the American Dream, and why it's a bunch of crap. Salient points reproduced here.

As I see it, and as I often hear it, the American Dream (AD) is that any person, no matter where they start in life, can make it to the top. All it takes is hard work and a can-do attitude, two more of those quintessentially American traits. If we accept this as a fairly good model for the AD, the question then becomes, "Why are there poor people?" We can take the pro-AD answer, which is "They aren't working hard enough" or we can take the anti-AD answer, which is "A person's success or failure in life is determined by things outside of him or her self".

Let's take the first answer first, if only because it's easier to argue against something than it is to argue for it. We can see immediately that it's sort of elitist; it says that if I'm at the top, I'm there because of how awesome I am. The corollary to that is that if you're on the bottom, you're there because you just didn't try hard enough. I say "sort of" instead of "completely" because it depends on who's doing the talking. If a poor person says, "I'm poor because I just didn't try hard enough - I wasted my time doing things that I knew wouldn't get me anywhere in life", then that certainly doesn't seem so offensive. I have my doubts about how many people in poor circumstances would be willing to take the blame.

The other answer has its problems too. First, we have to realize how deterministic it is; essentially, it says that a person's life is decided by factors beyond their control. I don't think there's much argument that we're shaped by our environment, but once you start giving that as an excuse for what the politicians would call "under achievement", it both allows people slack for their failures and reinforces negative self image. But the other critique of that answer is a little more subtle - and it's not a critique of the answer so much as the response to the answer. If the rich help the poor, they're saying that the poor need them, and they're putting the poor in an inequitable position. Not only that, it becomes more about the rich than the poor; we have $1,000 dollar a plate dinners to prevent starvation in Africa, or my favorite, a chalk message drawn on the sidewalk at Harvard proudly proclaiming, "Smores for Darfur, $2".

Patterns of Information Consumption

A few centuries ago, there wasn’t as much history as there is today. That should be obvious, of course, because a few centuries of history have happened, but that’s just the tip of the iceberg. In the 14th century, prior to the invention of the printing press, there wasn’t much that a person could really know. Books were closely guarded, and they were produced at a comparatively slow rate – about a thousand were published each year. In 2006, the world put out two hundred thousand new books. To put that in perspective, that mean that in six months we matched the entire 14th century.

Before the printing press, knowledge of all kinds was confined to on of two things – a person, or a book. Since books were laboriously produced one at a time, they were almost as physically restrictive as a person. To get the information contained in either of these sources, you would have to seek it out in the real world, and hope that the book hadn’t burned, crumbled, rotted, or simply been misplaced. Because of these physical limitations on how data was accessed, knowledge and especially history tended to be geographically bound. A traveler would bring back stories, and those stories would be spread through the oral tradition, mutating into nearly unrecognizable forms. That was how people learned about other cultures, and even with the advent of the ambassador the common person would only know these things third or fourth hand.

The printing press changed things, but it took a long time, especially since literacy didn’t really catch on. With multiple copies of a book, information could be spread much faster. It was still confined, mind you, as only the most popular (or at least well funded) books would get the printing press treatment. The advent of newer and better methods of communication like the television and radio allowed one single (if still local) source to broadcast out to an entire community. Information was more free than ever, but it would take one more technology to fundamentally change human understanding one last time.

I’m speaking, of course, about the internet. Anyone who wants to say anything can say it there. Restrictions on free speech are limited only to the base and foul, and most of that gets put online anyway. Information doesn’t have to be filtered through an authority source anymore. If I want to know about the war, I can read a blog written by someone in the military, on the ground in Iraq. If I don’t trust that the information there is accurate, I can simply move on to the next blog, or follow trackbacks, comments, and criticisms. In a world where information was still constrained to physical form, this wouldn’t have been even remotely possible. Criticism of books was left to newspapers and magazines, with an interfering editorial process and more often than not an overlying capitalist agenda. There is more free information today than ever before, vast virtual archives of criticism, discussion, first hand experience, and the raw stuff of science.

Some might look at the world today and ask, “How did our ancestors get by without this stuff?” I look at the world today and think, “How are we able to process this?” Humans are eminently adaptable, true, but we weren’t built to handle the amount of information that’s coming at us. I mentioned at the beginning of this article that in six months of 2006 we matched the entire new published books output of the 14th century. Yet the individual’s capacity for reading hasn’t improved much in that time. We aren’t, as a species, that much smarter than we were then – better nutrition has helped some, but only by a few IQ points. And while the 14th century might have had information stored mostly in books, today we have movies, magazines, newspapers, television, and of course, the flood that is the internet. How are we handling it?

The initial promise of e-commerce was disintermediation – customers and businesses would be able to function more efficiently because the middle man would be cut out. In the realm of information processing, we’re seeing an opposite trend. Because so much is coming at us at once, we have to pick and choose what we’re going to pay attention to. Partly because of that, and partly because of the increase in advertising, there’s an overwhelming desire for reviewers – people who will tell us what’s important and what’s not. The nature of the internet allows us to take it a level beyond that. We can aggregate reviews, or review the reviewers, or listen to people with no qualifications or credentials. We have websites like Metacritic or Rotten Tomatoes, that can tell us what the collective thinks.

That’s the clustering trend. The other trend is a divergent one. If you have an interest, there’s a website for it. The cost for making a website is nothing, so a person with an intense interest in, say, porcelain horses can create it and maintain it with no cost other than time. For people with niche interests, time is nearly meaningless. Then at person might find another website devoted to porcelain horses, and they would link to each other. This would continue until their community had grown up to be a certain size. In this way, knowledge and information can become as specialized as individual interests dictate. So can communities.

There’s a push and pull at work here between insularity and transience. It’s more visible in the virtual world than in the physical one, but it happens to people in both places. Especially in large cities, we can and do surround ourselves only with people who fall within the narrow confines of our interests and beliefs. This is the new world of humanity; we skim the big pools but confine ourselves to the places and people we define as “ours”. There’s more of everything than there ever was, but I fear that we’re experiencing only as much as we ever did.

Sunday, December 16, 2007

Jetpacks (I want one)

Well, it turns out that the future does have jet packs. It just also turns out that they're horrifically expensive. The price? $250,000 (with YouTube clip!). And they run on jet fuel, which is also probably not to cheap. But it gets 19 minutes of flight per fueling, which is pretty impressive compared to the last generation of jet pack. If I had the money, I would buy one, just so I could go flying in places that I wasn't supposed to. Then again, there can't be many laws in place restricting the use of jet packs, can there? So I could fly over traffic and cause a ruckus, or give myself jet pack assisted leaps across the tops of buildings.

Actually, it must be pretty easy to injure yourself in one of those things, especially with such hot exhaust coming out the back. I don't think it's like unassisted flight either - you have to be strapped in somehow, and unless it's bound quite close to your body it would be likely to pinch or pull at you.

The other great advance in human flight is the wingsuit, which yes, makes you look a flying squirrel. This is another one of those things for people with a money, as it costs about $1,000 for a wingsuit (with the more significant cost of getting a plane to take you up every time you want to come down, and a parachute). Given enough time for the materials sciences to advance, it might be possible for someone to land without a parachute. Though everyone in the business says that it could theoretically be done, I'll believe it when I see it.


There's something really fascinating about robots. There are a whole lot of people working on the problem of humanoid robots, and it won't be that long before they become efficient enough to do some jobs currently done by flesh and blood people. It's already happening in millions of factories, but when we see something that's just two joints attached to a welder, it doesn't do much to spark the imagination.

Human bodies are inefficient for most of the stuff that humans do in the modern world. Legs are a great example of this - we don't really need them most of the time. Wheels are a more effecient form of locomotion, so long as you're just traveling around flat places. Given the ever more handicap-accessible world, it doesn't make real sense for a robot to have legs in an urban environment. On top of that, gyroscopic technology is now at the point where we can have the wheel equivalent of bipedal locomotion - the Segway is exhibit number one of that. This further reduces the amount of space required for wheels.

Another way that robot bodies can improve on the human design is with the hand. To pick up any object requires a maximum of three "fingers". Two fingers are needed to grasp an object like a pole, and three fingers are needed to fully enclose a small object. And of course only a single finger is needed to poke something. Besides that, the designs just look cooler than the hand. We talk about the thumb being opposable, but with a robot hand all the digits can simply swivel around a base, making them all opposable to any other digit. Besides picking up and manipulating objects, the thing we use fingers for the most is data input, not just in the typical example of hammering away at a keyboard, but more subtle forms of data input like playing the violin or doing long division. Mathematics has already been completely outsourced to computers, and the ability of computers to reproduce sound has been proven to such an extent that it seems trivial to point it out. And yet people keep trying to make a robotic hand that looks like the human one.

Here are the arguments in favor of more human robots:

The Argument from Aesthetics
People tend to be uncomfortable with new things. If a robot looks at least a little like a human, people will be more accepting of it. This is especially important in the field of healthcare, one of the primary focuses of Japanese robotics. A humanoid robot is more pleasing to the eye, which means that it's actually worth solving some of the more unique engineering problems associated with that.

The Argument from Backwards Compatibility
Almost everything in modern civilization is designed for humans. In a way, humans represent the dominant form factor; an industry standard to which it's wise to comply. Having a robot made to those specs means that it is guaranteed to be able to interface with objects and machines that humans use. In almost all cases, this won't be the best solution - but once the generalized problem is solved, suboptimal solutions become cheap enough for that to not matter all that much. Think robot chauffeurs; they actually make sense if the economic variables are right, even though that solution makes much less sense than an on-board computer with sensors.

The Argument from Augmentation
Robotics doesn't just apply to robots. One of the major areas of advancement right now is in prosthetics. By making robot arms and hooking them up to people, a robotics company can expand its market by a large amount, as well as being able to secure both healthcare and defense grants.

Traffic Lights

The traffic lights in my town aren't automated. They run on 80 second timers. For most of that time, they're red. Thirty seconds green, three seconds yellow, forty-seven seconds red. Someone timed it out that way for optimal traffic flow. When an ambulance comes through, they're surrounded by green lights, like a shaft of sunshine coming down on God's beloved. Then once they're gone all the lights reset back into their normal pattern.

The city is mostly grid shaped, but on a hill. One block is about 1/10th of a mile. Finding out the optimal speed for all green lights is just a matter of finding out how much their timing is offset by and doing a little bit of math. I'm almost tempted to step outside my house and find the nearest two traffic lights to that I can find out what the difference is. What really interests me about this is whether they're setting the lights to lower speeds to within legal limits, or whether they're doing it maximize traffic flow. Maybe both.

It's interesting to note that a staggered light system only has one speed you can consistently travel at keep hitting the greens. There's a thirty second variance in there of course, as you could go through one right after it changes from red and go through the other right before it changes to yellow. In a non-staggered system, where all of the lights in a column are the same color, there would be several speeds you could go to guarantee that you always got the green light. They would, however, be quite slow. An 80 second cycle with one light every block would have you traveling 4.5 miles per hour. You would be able to half that speed any number of times and still be hitting every green light. I think most of us would just gun it and see how many greens we could get through before it turned red on us again.

Also, when I was searching in vain for information about how the lights are programmed, I came across this lovely article about the effects of weather on street car traffic in Duluth, circa 1917. If I had $12 I would buy it.

Friday, December 14, 2007

The Implications of AuthorBot

I like the idea of AuthorBot. Once I've learned enough, I'll at least make an attempt at PoetryBot, because people are generally more forgiving when your lines don't match up well (besides which, attempts have already been made for proof of concept). But more than AuthorBot, I like all the things that it could do that a normal person couldn't do as well.
  • Do you want someone to write a sequel to your favorite book? AuthorBot could analyze the writing patterns of that book, along with other novels that were written by the same author. It would be able to study sequels in general to find out what their common characteristics are. Then, once it had all this data, it would be able to extrapolate a new sequel to the book in question. It wouldn't have to stop with one sequel either - AuthorBot could write more and turn it into a trilogy, or a seven book cycle, or more.
  • There's already more books being published in a year than a person could read in a lifetime; with AuthorBot the number of books would be infinite.
  • This is not to say that a novel writing program would put all the human writers out of work. At least for now, there's a strong human repulsion to machines in any sort of creative or social context. Why? I'm not sure - some thoughts on that later.
  • If AuthorBot did write a sequel to a book, would this be a copyright violation? It would certainly be a derivative work. But if AuthorBot were sold as a program instead of as a service, then the book would only exist as potentiae, and it would no more be illegal to sell the program than it would be to sell a photocopier.
  • AuthorBot would have no artistic integrity. It's still not technically intelligent either, it just appears intelligent even on close inspection. As a result of not having any sort of morality, you could tell it to put out horrible dreck and modern masterpieces, and it would happily do either.
  • One way to make AuthorBot a lot more powerful would be to feed it metadata. Not only author's name, title of the work, year produced, and simple stuff like that, but also reviews, customer ratings, and things like that. This is already being done with Web 2.0 - it just needs a way to be fed in. It would also allow AuthorBot to know what books are "good" and what books are "bad", though it would likely find that those synonym groups depend on who's reviewing (which would lead to yet more clustering).
  • As mentioned previously, AuthorBot would be able to write in the style of another author - it would also be able to write when given certain criteria, such as a setting, time, theme, etc. This allows custom novels to be made to individual tastes.
  • On top of that, AuthorBot could scan a list in of authors that a person likes, and synthesize a novel that would appeal to their tastes (though it would have to fit the book into a cluster - while their are people who like Westerns and spy thrillers, combining the two wouldn't usually be the best way to go).
  • I don't really see AuthorBot being brought into the individual home. Because of how much power it would take to run it, I think that people more likely visit a website where they could order books that would either be micropublished or sent to an e-reader. The micropublishing option would cost more, of course.
  • Here's another thing; AuthorBot wouldn't have to care about copyrights, because it's dynamic instead of static. "Writing" a book would take comparatively little of its time. The real draw is its dynamic nature, so it might actually be best to let people freely share the e-book version to draw in more customers.

What Evolution Means

Evolution doesn't mean what you think it means. It's an emergent property of systems, a high level phenomenon. As stated by Darwin in The Origin of Species, if a system has the following three qualities:

1. Geometric ratio of increase
2. Mutation
3. Selective Forces

then transitions in the makeup of the population will take place. It makes no sense to say that one organism is more evolved than any other organism because organisms don't evolve. Only populations do that. At best, we can say that an organism is more adapted to its environment than another organism. We can't even say that a population is more evolved than another population, unless we also state that what we mean by "evolved" is "more adapted to a particular environment".

The other big term is "evolving". This makes sense if we view the term as "being selected for traits which the majority population does not posses", but even that's a muddy definition. What people really mean is "better in a good way". Unfortunately, there isn't a good existing word for this, so evolution was taken in. There's a strong need for that sort of word too, especially in this era of constant innovation and change. In part, the problem was exacerbated by the co-opting of biological terms by the tech industry; they refer to their technology and programs in terms of generations (especially videogames - I'm ready to strangle the person who came up with "next-gen" as a buzzword). While I might accept calling these generations, they lack the first two criteria of evolution. We could, however, refer to evolution when looking at the population of gadgets, as the population fits all three criteria (selection in this case being market forces).

Thursday, December 13, 2007

Why Privacy is a Bad Thing

Imagine a (magical) computer program that could store every action taken by every individual throughout their entire lifetime. It would record information about what they bought, what they ate, who they talked to, what they said, various bioinformation like heart rate, cholesterol levels, etc.

Now imagine that this computer stores all this information with a series of tags, and the tags have their own meta-tags, so that information can be pulled out of this database with high efficiencies and assembled into useful charts, graphs, and patterns.

From the database, we could then find hidden relationships. We could look for where things cluster, such as seeing that people who eat a specific type of canned soup have a 20% higher incidence of cancer. We could track epidemics in real-time. More importantly, we could track the epidemics that people aren't really interested in self-disclosing (STDs, obesity, addiction). There would be sociological implications too, as we could see correlations in the data that would tell us the causes (or impacts) of things like abusive husbands, suicides, or even successful marriages. And of course there would be numerous commercial applications, such as finding out how effective advertising really is, tailoring products to specific demographics, fine tuning production to be more in line with consumption.

Right now, billions of dollars are spent on analysis every year. Focus groups are formed, surveys are given out, and the real world is studied as closely as possible. This magical computer system would eliminate all of that. Here's the thing; corporations are already doing their best to study trends. So is the medical community. So is the government. The benefits of trend analysis are immense. It's only recently that computers have made dynamic trend analysis a real possibility; it used to a series of single studies was performed to determined specific things. Now? We can take reams of data and crunch it in all sorts of interesting ways.

The problem for our theoretical magic computer is twofold. The first problem is that people don't want to give up information about themselves. This is understandable really, because people sometimes do things that are illegal, stupid, or socially unacceptable (sometimes all three at once!). But so long as it can be guaranteed that this information isn't going to be available to the people around you, we've eliminated the last two concerns.

It's that first concern, the "illegal" things, that we have the biggest problem on. Here we come to a basic problem with the law; sometimes laws are made that people don't follow. Jaywalking is the classic example. We also know that there's a general rule that you can go five miles above the speed limit. And nearly everyone I know has at least dabbled in file sharing. So either these laws need to change, or people need to not be prosecuted for these minor infractions (which is what happens now).

The second problem is that most major organizations hold their information as proprietary. Technically this isn't really a problem when we have a computer run on magic, but in reality there needs to be a company (or government) behind the computer. Google already stores every search that you type in, along with all of your mail if you use Gmail, and all of your documents if you use Google Docs, and that holds true for every service that they offer. They use this information to data mine and better advertise to you. But do they share this information with any other company? Of course not.

There's no sense giving your competitor an edge. Unless, of course, you can trade the advantages you have so that both you and your competitor increase profits, which is exactly what would happen if data sharing went on.

But let's track back to you, the consumer, the citizen, the person who cares about privacy. There are reasons for privacy beyond those of "getting caught". For some, it's a matter of not trusting that any big organization is going to be looking out for your interests. That's a valid concern, but it's my belief that the goals of the government and the people are in alignment most of the time. If not, the cost of fucking with the individual needs to be high (in the form of boycotting, protest, or homegrown terrorism if it comes to that). For others, it's a matter of some vague philosophical notion of a private space, to which I say that the benefits to humanity are too great to ignore. Besides that, already happening.

Of course, my opinion might have its basis in my love of data analysis, which borders on fanatical.

Monday, December 10, 2007


So let's say that you take the sum of all human English literature, and feed it into a computer program. Would that computer program, if properly written, be able to make judgments about the nature of literature? My guess is yes. The tricky part is the "if properly written" clause.

When we talk about emergent properties, we're talking about properties that arise from a system, rather than properties that are encoded in the system. Some people would call this "high level phenomenon". What I want to talk about isn't emergent properties, but something that's sort of the opposite; deriving basic system rules from an examination of relevant data. This is something that we do all the time in the sciences. We conduct studies to try to figure out what's really going on in the world. This is the basis for the scientific method.

Literature isn't something that we really need to figure out the basic rules on. As humans, we naturally know language, and our understanding of it doesn't come from the basic rules but from patterns that get built up in our brains. Humans are naturally suited to language from birth. That's why it takes babies so little time to go from gibberish to forming words and sentences; they do this without knowing what a word or sentence is. It makes a lot of sense really, because how could someone tell them what a word or sentence is without using language in the first place? Language is an emergent property of the way the human brain is structured. The structure of the human brain is an emergent property of our genetic code: your DNA doesn't code for every single neuron in your brain. This dynamic system that doesn't depend on hard-coding is why the human brain (and body) can handle all sorts of different experiences and environments. And on top of that, what DNA codes for is in the long run determined by another emergent property called evolution.

I've digressed. Let me bring it back; in the first generation of artificial intelligence, programmers attempted to hard code hundreds and thousands of rules, hoping to make something intelligent. They failed, for the simple reason that intelligence is an emergent property. Perhaps the logic was that by piling so many rules and their exceptions on top of each other, something could be produced that resembled intelligence. This is the main argument that philosopher John Searle puts forth against the possibility of so-called "strong AI" (read: human level). He says that a mere system of rules can't produce intelligence, because it wouldn't be anything more than what was put into it. I would agree, so long as we confine our definitions of artificial intelligence to programs which can't create their own rules.

So then, to create intelligence, we need a system that can develop its own rules. What I want to make is a program that can understand literature. The rules (other than a few bootstrap and admin rules) will be developed from a vast amount of data rather than us designing rules that create emergence. The idea is that the bootstrap rules will be able to look at repeating sequences and statistical correlations to make new rules.

So we feed the program 88,000 books. I use this number only for example, and because it's the number of e-books currently supported by the Amazon Kindle. The program runs through a book, notes statistically significant repetitions, and analyzes for structure. For example, if we assume that the computer starts without knowing words or grammar, then running through the book the program would make a note of the ways in which characters are arranged. It would see that certain letters appear in certain ways. For example: Q is usually followed by U; X is usually preceded by E or A; one space is rarely followed by another; quote symbol usually has another quote symbol close by; strings of characters are ended with a period. These are basic rules, rules that the program can figure out without knowing anything about the system itself. The program still wouldn't know what a word is. But if, through one of our hard-coded admin rules, we ask it whether a set of characters is likely to have a space or punctuation on either side of it, it would be able to give us a pretty judgment of that (which is how we would define a word in a pure character way).

The next step, once the program has written new rules into itself about what it believes to be valid constructions in our system, is for the program to go up a level. It would have to look at sequences of letters that are appearing and determine both their frequency and location within the data set. For example, it would have to see that "cat" appears both as a word (separated by spaces/punctuation), set in among other characters (as an unrelated thing), or as part of another word (in compounds). In the example of "cat", it would see "cat", "catty", "categorically", "scat", etc. But it would also see that the sequence "categorically" appears almost exclusively as a word. So from that, the program would derive rules about word frequency.

From a study of which words appear where, it would have to be able to derive other properties. It would find out that punctuation marks are significant to determining word order. And when looking at word order, it would eventually (with the application of statistics) find that certain types of words tend to follow other types of words. Nouns are usually followed by verbs, and vice versa (with adjectives sometimes separating them). It would be able to see that many words share character sequences within them and at certain location within the sequence - we call these prefixes and suffixes. Both of those, along with word locations in relation to other word locations, are essential to knowing whether a thing is a verb, noun, adjective, etc.

So far this is all stuff that we could hardcode, and to be sure, scientists have laboriously done it. Those programs could make words without spelling errors (usually pulled from a dictionary database) and sentences with proper grammar (those being emergent). Intelligence it was not. The next stumbling block for our theoretical program is the big one; meaning. That's what separates a clever trick like making sentences from something really astounding.

How would the program determine meaning? This is why we need thousands of books. By analyzing where words show up in relation to each other, and figuring out that some words with the same root are statistically correlated despite prefixes and suffixes (such as like and likely), the program can come up with a new rule that postulates those derivations to be synonymous. By looking at words that show up near each other, and the ways they do so, the program would be able to postulate both synonyms and antonyms. It would make new rules for tenses and new rules for view. With a large enough data set, the muddy rules of English could be mapped automatically. It still wouldn't really know what anything meant, but it would be able to see make rules that describe patterns. It would know that dog can be preceded by fat or skinny, and that, once preceded by that, it will often be preceded by it again (or one of it's synonyms). And with enough of these associations, it would see when one synonym set tends to be associated with another synonym set. In that way, it would get to "know" (make new predictive rules) about things. For example, cats are agile, or rocks are hard.

So we have grammar rules, and we have a rudimentary form of meaning, so it's likely that this program could make a sentence like "The rock is hard." But the real challenge is to produce multiple sentences that maintain a thought. Again, we need statistical analysis to give our program this rule. This is analysis at the same level as earlier levels, just bigger. It would have to know types of sentences. And once it got those, it would have to see (again, through statistical analysis), that sentences vary within a paragraph, rarely repeating their types. Of course, it would also need to define "paragraph", but that wouldn't be too hard if we've come this far; a paragraph is any group of sentences that are separated by the carriage return symbol.

So once it knows both what a paragraph is, and how they are typically structured, it needs to look at chapters, and section breaks, and entire books, and compare and contrast those both within and without their containing structures. Patterns would emerge, and those would be codified into further rules (ones that are much more muddy than the lower level rules). Here comes another tricky part; once it knows all of these rules, that might be enough. It might be able to write a halfway decent novel from that. But if it isn't enough, then it needs to be able to figure out (again, through pattern analysis) that words talk about each other, both explicitly and implicitly. And once it understood that ... well, the world would be its oyster.

Imagine the following; "AuthorBot, write me a three hundred page novel in the style of Dickens, set in 1970s New York." A few seconds later, you're printing it out for easy reading. That's the end goal, but as you see there are many layers that need to be built up for it to happen. This is all under the assumption that meaning can be derived solely by looking at a language, without any knowledge of the outside world. And someday, when my brain is big and strong, and the computers can handle the data flow, I'll program it (unless someone does it first). Consider; Moore's law predicts that in about ten years, computers will be 100 times more powerful than they are today.

If it's possible, it will be done.

Thursday, December 6, 2007

Future Shock

I've been rereading Alvin Toffler's Future Shock, and it seems odd how much of it is still relevant and how much falls flat. This won't be a critique of the predictions made by a book that's now 37 years old*, don't worry about that. What interests me more is how little of the future it was actually possible to see from the past. Of course, my real interest lies in finding out how to mitigate those problems when looking at our own future. There are all sorts of predictions which come out every year that are just plain wrong, and I think we can pretty safely ignore them, but the ones that come close to revealing the truth might offer some real insight.

Toffler talks a lot about the throwaway society; most of the things we have get replaced at a rapid rate. He's not just talking about material goods, although he makes that case too. He's talking about friendships, jobs, and culture. Starting with the automobile making its entrance into the mass market, humans became more geographically independent. We no longer had to stay in one place if we didn't want to. This meant a lot not only for tourism, but for moving; if you were tired of the city that you were living in, or ostracized by your community, you could simply leave with a minimum of hassle.

On top of this geographical freedom came occupational freedom; the trends started moving towards jobs that lasted years instead of a lifetime. The average time for a programmer to work at any given place in Silicon Valley is now eighteen months. People started to go back to school, or change their careers in the middle of their lives. We were no longer defined by what work we did. Of course, we often choose to define ourselves by what we do, so long as we enjoy our occupation. And in the year 2007, education defines most occupations, because new technology comes down the line that improves efficiency at the cost of training and implementation. This is true even for the most basic of jobs; farms now use multi-million dollar equipment, the service industry uses computer interfaces, and most manufacturing jobs have been revolutionized if not outright replaced.

Geographical and occupational freedom had a major impact on both family and friendship. Friendships that happen over a distance inevitably break down, so the people who aren't tied down to any given place aren't tied down to any given friendship. Friendships are more transitory, with more friends at the cost individual friendship depth. This is common criticism of American culture, a society which has perhaps the greatest geographical and occupational freedom of any (mostly due to our size, rather any superior form of government).

Viewing these arguments in light of the Internet is confusing to say the least. Here we have a system of friendship which is completely independent of geographical location, on top of institutions and possessions which are also independent of geography. My Gmail account, for example, can be used from any computer which has an internet connection. It's outlasted my first computer and will probably outlast my second one too. I still agree that friendships depend on proximity to an extent, but not nearly as much as they used to. As audio, video, and later haptics come onto the scene, physical proximity will mean less and less as virtual proximity comes closer and closer to "the real scene".

Lastly, my favorite chapter on rereading Future Shock was the one on the "adhocracy". This is closely related to the concept of Web 2.0, and it's such a catchy phrase that I'm not sure why it hasn't caught on. People organize into organic structures of control, from each according to their abilities, to each according to their needs. While I have my criticisms of Wikipedia, that's generally how it works over there. The amount of volunteer work that goes into the internet in general in terms of moderation, tagging, and free content is astounding. On top of that, the internet is itself ad hoc organized with an organic structure; while an individual website might have hierarchical page structure, the system of links creates a very messy structure.

Messy, but it gets you to the right place most of the time.

*Future Shock was published in 1970 for those without strong math.