.CSV

Thanks to popular culture, we have a good idea of what to expect when “strong” AI arrives. Machines attain consciousness? Prepare to be harvested as food. Detroit introduces talking cars? “Hello, Kit“.

What to expect in the near-term is less clear. While strong AI still lies safely beyond the Maes-Garreau horizon¹ (a vanishing point, perpetually fifty years ahead) a host of important new developments in weak AI are poised to be commercialized in the next few years. But because these developments are a paradoxical mix of intelligence and stupidity, they defy simple forecasts, they resist hype. They are not unambiguously better, cheaper, or faster. They are something new.

What are the implications of a car that adjusts its speed to avoid collisions … but occasionally mistakes the guardrail along a sharp curve as an oncoming obstacle and slams on the brakes? What will it mean when our computers know everything — every single fact, the entirety of human knowledge — but can only reason at the level of a cockroach?

I mention these specific examples — smart cars and massive knowledge-bases — because they came up repeatedly in my recent conversations with AI researchers. These experts expressed little doubt that both technologies will reach the market far sooner, and penetrate it more pervasively, than most people realize.

But confidence to the point of arrogance is practically a degree requirement for computer scientists. Which, actually, is another reason why these particular developments caught my interest: for all their confidence about the technologies per se, every researcher I spoke to admitted they had no clue - but were intensely curious - how these developments will affect society.

Taking that as a signal these technologies are worth understanding I started to do some digging. While I am still a long way from any answers, I think I’ve honed in on some of the critical questions.

< More... >

There is no shortage of anecdotal evidence testifying to the impact of hip-hop lyrics, from sales surges in Courvoisier cognac to the rise and fall of Girbaud jeans. (Indeed, companies are even emerging that promise - threaten? - to bring paid placement to pop songs.) With these examples in mind, I got to work: I grabbed the lyrics to almost 40,000 hip-hop songs, covering just about everything you can think of from 1996 until today. I then matched the artist and album name against Amazon’s music database in order to get the album’s release date and sales rank (a distinctly non-trivial task, given hip hop’s creative and ever changing relationship with grammar and spelling).

To verify that there is real signal in this information, I first took a look at all mentions of Tommy Hilfiger (including variants like “Tommy Hil”). Hilfiger is a well-known example of the dangers of being adopted by ‘urban’ trendsetters: the quicker the youth market embraces a new brand, the faster it falls. Hilfiger hit peak popularity in 1999, with sales doubling that year from $847m to $1.63b. Then, just one year later, everything fell apart: newer urban brands like Fubu became more popular with teenagers, while Hilfiger’s traditional customer felt the brand was now too edgy for them. Hilfiger has spent the last eight years extricating themselves from this mess.

You can see the sordid tale in the figure above, with the number of songs that mentioned ‘Tommy Hilfiger’ peaking in ‘99 and ‘00, plummeting to a new low in 2002, and now slowly slogging its way back.

Of course, Hilfiger was a small enough brand, offering an affordable enough product, that suddenly coming into favor with an urban audience, courtesy of hip-hop, was enough to really move the needle. Other brand names are more symbolic: mentions of them in songs tells you more about the overall economy, and perhaps the state of materialism, than it does about the brand’s future sales. Take that bling-standby, Cartier, for example. The chart showing how many times Cartier was called out in song over last ten years looks remarkably like the S&P 500 ten-year chart.

I think this very rough metric offers a worthwhile clue to how our relationship with brands, and consumerism in general, is changing. To be sure, it is a metric easy to abuse. Smaller brands, for example, are rarely mentioned enough times to draw any statistically significant inferences. Still, I did recently look at which brands have suffered the largest declines in hop-hop popularity. To address the data shortage problem, I used one of the published mathematical models to map Amazon’s sales rank to number of albums sold. (Actually, log number of albums.) Seemingly appropriately, Burberry was at the top of the list.

s.

I find the best writing almost impossible to read. Partly because I want to savor it; I want to deny it ends. Or, sometimes, it’s so provocative it forces me to reconsider everything before I can continue. But, mostly, great writing pisses me off because I didn’t — and I never will — write it first.

Spring is here, and with spring comes:  sunshine, fresh flowers, … and a slew of navel-gazing, big-idea technology conferences. I recently returned from two such trend-spotting confabs: CI Foo Camp and ETech.

In format, the events were completely different. CI Foo Camp, organized by Google’s Chief Economist Hal Varian and held on the Google campus, brought together 60 or so researchers all loosely connected to the idea of “collective intelligence” for a wide-ranging discussion with no set schedule. For ETech, O’Reilly’s flagship annual conference, several hundred hackers, academics, and online gadflies converged on San Diego for four days of presentations about anything deemed an emerging technology.

But what I distilled from the two conferences was very similar—the same topic kept coming up, over and over.  This emerging area doesn’t have a catchy moniker yet, but you can think of it as an amalgamation of crowd theory, human terrain mapping, and social simulation. It is the science of groups; it is a new kind of quantitative political science.

The tools and theories needed to analyze social interactions are just now reaching the level of sophistication — in accuracy, in robustness – necessary to leave the lab and enter commercial duty. We are in a period analogous to the early 1970s, when developments like the Capital Asset Pricing Model and the Black-Scholes equation transformed finance, changing it from an art to a science, and opening enormous new markets in the process. Now, new equations describing “crowd dynamics” are about to change our lives. And not always for the better. This is one of the most significant technology trends I have seen in years; it may also be one of the most pernicious.

< More... >

Lack of sleep makes me stupid. I was reminded of this last week, right around 6 am, when I was on a conference call and was asked my opinion on making wireless networks more open. As my brain struggled to right itself, my mouth moved its way through, “It’s a big positive because it will speed-up innovation in the industry.”

I would like to retract that little inanity. Not just because it’s vacuous (what’s new) but because it’s flat-out wrong, and misses the real, very important story that is currently unfolding.

< More... >