redundancy and unexpected interdependency

A couple weeks ago, the office network went down. Not surprisingly, we couldn't print or get to any electronically filed documents. Maybe a bit more surprisingly, the phones also went out because we've moved to fancy new Cisco network-based phones, and they took voice mail--which is now e-mail-based--with them. And so did the fax machines, because when faxes come in they get scanned and e-mailed to us. The net result was that we were almost dead in the water. Fortunately, the Blackberries stayed up, so we could use cell phones and could still send e-mail through the Blackberry server.

These sorts of unexpected interdependencies are cropping up more and more often as we begin to link one network to another, and as traditional network technologies shift around. I've already written about how my (now former) cell phone's alarm clock stopped working when the phone couldn't reach the cell network. There was a radio discussion today about an issue with digital TV: many people put battery-powered analog TV sets in survival kits so they can get news if there's a natural disaster, but (1) the set-top converter boxes that we'll need to use once HDTV becomes standard require wall power to run, which would make those analog battery powered TVs useless, and (2) many radio stations no longer have a news department, acting more as satellite feeds for a central office, so your transistor radio may not be able to pull in anything useful, either. Oh yeah, and many folks are moving away from conventional land-lines at home and going to VOIP phones or cell phones exclusively.

The classic way to deal with system failure is through redundancy: if the TV network goes out, you still have radio, which can get you much of what you need, or maybe you still have telephone. And so forth. The problem is that redundancy only works where you can stop a failure in one system from propagating to other systems. That's one reason most software doesn't have a redundant design: it's very hard to predict how far the effects of a failure will propagate through software, so it's generally not cost-effective to make it redundant. As our networking capabilities become more sophisticated, we're getting to the point where it's becoming harder to figure out what effect a failure of one network will have on the other networked systems we use.

The right way to do this, from an engineering perspective, is to take these sorts of problems into account when designing the systems, and then to periodically test them to make sure we got it right. That's not going to happen, at least right now, for social and economic reasons. The second-best approach would be to invest time and money into developing the design and evaluation tools necessary to either constrain the effects of a failure, so they don't propagate to other systems, or to at least be able to predict the likely failure modes and how far they'll propagate. We might eventually do that, but it'll take time. What's most likely to happen is the same thing that happened in structural engineering a hundred years ago: we may well see a series of unexpected, unforecasted failures, and they'll likely continue until we implement one of the first two approaches above.

still noisy creatures

Yes, indeedy, humans are noisy creatures. Tonight, one of our neighbors in one of the apartments across the street (I can't tell which apartment) has decided to play Night Club. He/she/they have the music cranked full volume. Loud enough that it's coming right through the closed windows and doors in the front of our house and reflecting off the apartment behind us through the back windows, too. Coppertop's had to put on a headset designed to block out aircraft noise so she can concentrate. And it's not one of those cute little ones for being a passenger in a jet, mind you. These are a set of David Clark's designed as hearing protection for pilots of propeller aircraft. I'm not normally one to call the cops on a noisy neighbor, but this is the closest I've come in a while.

e-mail culture and the company observer

I've been working at a law firm just long enough, now, to notice some of the odd cultural differences. One of the bigger ones is e-mail. Every place I've worked before, e-mail was something you used for lower priority messages, things that didn't need to get handled right away. You generally knew when the person on the other end would get into the office, so you could send them e-mail, say, on a Saturday when you were thinking about it and figure they'd get to it Monday morning.

Not here, though. You see, they issue everyone Blackberries. And that means everyone's plugged into e-mail constantly. And that, in turn, means that they treat e-mail just like a phone call. If you e-mail someone at some unfortunate hour of the night or day, they tend to get back to you right away. And that means it's poor form to dash off a message on Saturday just because that's when you happened to think of it.

This will take some getting used to. I guess I'll have to start e-mailing those Saturday inspirations to myself, instead.

where I disagree with Joel Spolsky

Joel Spolsky is a very bright guy who has some strong views on how to design forum software to build communities. Scientific American's Blog has been revamping its comment system. I don't know whether they're consciously trying to follow the Spolsky design or if the current setup is just a temporary stop on the way to something else, but the current system incorporates enough Spolsky-esque features that it's become an interesting place to observe how the design works. And I think I've seen a few problems.

The first issue is not having a preview button. Here's what Joel writes:

Q. Why don't you show people their posts to confirm them before you post them? Then people wouldn't make mistakes and typos.

A. Empirically, that is not true. Not only is it not true, it's the opposite of true.

Part one: when you have a confirmation step, most people just click past it. Very few people reread their post carefully. . . .

Part two: the lack of the confirmation step actually makes people more cautious. It's like those studies they did that showed that it's safer, on twisty mountain roads, to remove the crash barrier . . . . You're better off, statistically, just scaring the bejesus out of drivers so they creep along at 2 miles per hour around the hairpins.

The first problem here is that it assumes that people use the confirmation feature (or an optional preview feature) to fix typos, but with the software that supports blog comments, that's often not the case. Different blogs support different feature sets. Some, like this one, will let you put hyperlinks in comments, add italics for emphasis, and properly support paragraph breaks. Some require you to manually insert paragraph breaks using the appropriate HTML tag. Some let you use block quotes, like the text above, while others don't. And most won't tell you what feature set they support. As a result, without at least the ability to preview your comment, you're writing blind: you take your best guess at what features that blog supports and hope it comes out OK.

The second problem is that some people misinterpret this comment to say you shouldn't have a "preview" button. Notice that Joe's writing about a "confirm" button, something everyone would have to click. A preview button's not the same: it's optional. Some people do proof their comments, and for them, being able to see the comment in preview form is useful because it rearranges the text. It's like printing out your document and proofing it with a red pen: because it's been rearranged, you'll see things you might not have seen the first time around. Not giving them a preview button makes it harder for them to write well even when they're willing to take the time to do it.

Another place where I'm not so sure the forum design translates well to blog comments, at least where the blog has a registration system, is in showing the existing comment:

Q. Why don't you show me the post I'm replying to, while I compose my reply?

A. Because that will tempt you to quote a part of it in your own reply. Anything I can do to reduce the amount of quoting will increase the fluidity of the conversation, making topics interesting to read. Whenever someone quotes something from above, the person who reads the topic has to read the same thing twice in a row, which is pointless and automatically guaranteed to be boring.

The problem here is that blogs generally sit on the open Internet, and text on the Internet is effectively permanent. How can something so ephemeral be permanent? Because disk drive inventors are busily doubling the size of storage every year and a half, so people archive everything. Some year in the near future, you'll be able to fit the entire contents of today's world wide web in a device the size of a USB flash drive. (Don't believe it? Doubling every 1.5 years means a 100x increase in storage capacity every ten years. So today's 2GB thumb drive may be a 200GB unit in ten years, and a 20,000GB unit ten years after that, and so on.) And, thanks to archiving, that copy of the web will contain each and every one of your blog comments from X years ago.

Archiving means conversations are not like that fluid conversation you have in the corner coffee shop where everything you say vanishes into the air. A better model might be a debate between two people writing letters to the editor in a national newspaper: that's the kind of potential those comments have to come back and haunt you, at least in today's society. Hopefully one day society will shift and people will lighten up about holding others to their written word, but it's not a sure thing. So if you're going to have a registration system that encourages people to use their real names, you should also give them the tools they need to write in a way that will withstand the test of time.

Now, I'm not saying Joel is wrong. Like I said, he's a smart guy and knows a lot about software design. But at least in these areas, I think programmers need to give some careful thought when they're putting together a blog's comment system.

but what about the science?

I wound up responding to a comment on another blog that asked about the science behind global warming. It's true: there's a lot of hype out there, making it hard to find the solid information. So, partly as a public service, partly so I can find these links more easily in the future, and maybe a teensy bit because finals are making me a touch grouchy, here are some useful starting points for solid information.

• "I am totally convinced that there is something unusual happening with Global Climate. . . . What I am not convinced about is the science behind the 'causality'."

The IPCC's Working Group 1 reported on this issue. Their Summary for Policymakers (pdf) targets policymakers (as you might expect) rather than scientists. As a result, it has some technical jargon but not too much and is pretty clearly written.

If you want more details and really want to get in to the nitty-gritty, Working Group 1 has now released most of its full report. The summaries come out first, then the report rolls out, with revisions as they go. The report's pretty large, but it includes citations to the scientific literature. That means if you want the details, you can go through the report, find the citation, then go to the nearest university library and track down the papers in their journals and continue your research from there. In some cases, you can find the papers through Google Scholar, but getting the full text tends to be hit-or-miss. On the other hand, Google Scholar does a pretty good job of showing you which other papers cite the paper you're searching for, so you can see how others have commented on the paper.

• What is that I can do personally that will at least stop the situation from becoming worse if not reverse it?

The IPCC's Working Group 3 is reporting on mitigating greenhouse gas effects. They have released a preliminary version of their Summary for Policymakers (pdf), but neither the final version nor the full report is out, yet. Keep an eye on the IPCC's main web page or the WG3's page to see the updates come out. Eventually, the full report will probably be available at the IPCC's publications page, where you can currently find the 2001 assessment. (Note to self: it might be very interesting to compare the 2001 assessment against the statements in the reports out of the White House Council on Environmental Quality.)

The second half of this question comes in two parts. First, the IPCC report shows most of the greenhouse gas emissions come from the energy sector, which suggests energy conservation is something someone can do personally. Transportation is another area to consider (in other words, think twice before buying the gas guzzler.) Second, the U.S. is entering a presidential election cycle, which means We the People might have a shot at steering the debate or changing policy.

• What do scientists think the impact of global warming will be? What does it mean to me?

The IPCC's Working Group 2 is tackling this question. You can find their Summary for Policymakers here (pdf--everything's pdf, I guess because of the pretty color graphs). This one is probably also still early and will see some revisions. Watch the IPCC front page for the rest of their report to come out.

more on military commissions act and citizens

Back in October, I wrote a short post about the Military Commissions Act of 2006 and its impact on citizens. I had a chance last night to ask my Representative about it, but it's hard to give a good answer in a group setting, so I sent a follow-up e-mail. We'll see what comes of it. In the meantime, I wanted to expand a bit on the reason why I raised the issue in the first place.

• §§ 949a(1)(A) and 948d(c) of the act say one way to be an Unlawful Enemy Combatant is for a Combatant Status Review Tribunal or other competent tribunal under the President's authority to determine that you're one. The sections do not limit their scope to alien unlawful enemy combatants.
• §§ 948c and 948d(a) limit the jurisdiction of a military commission to alien unlawful enemy combatants.
• § 948a has separate definitions for "unlawful enemy combatant" and "alien," suggesting they're separable concepts.
• I could not find anything in the act that specifies whether or not the jurisdiction of a Combatant Status Review Tribunal, as opposed to a military commission, extends to non-aliens.

So the act talks about two different judicial bodies, a Combatant Status Review Tribunal and a military commission. How do we know they're different? For one thing, the act spends a lot of time defining a military commission but very little defining a Combatant Status Review Tribunal. For another, tribunals usually have three people (hence the name) but the act says commissions have at least five. § 948m(a). Combatant Status Review Tribunals decide whether or not you're an unlawful enemy combatant, and their answer is "dispositive." § 948d(C).

The act says they decide whether you're an "unlawful enemy combatant," not whether you're an "alien unlawful enemy combatant," but the military commissions only have jurisdiction over alien unlawful enemy combatants. So we have this odd situation where a Combatant Status Review Tribunal can find you're an unlawful enemy combatant, but if you're a citizen you're still outside the jurisdiction of the military commissions. Why?

Now the kicker. In Hamdi v. Rumsfeld, the Supreme Court's plurality said the executive branch can hold a citizen indefinitely only if the citizen has a chance to rebut the charge of being an "enemy combatant" before a neutral decisionmaker. So my question to the Representative is whether Congress, in the Military Commissions Act, handed the executive that neutral decisionmaker in the form of the Combatant Status Review Tribunal.

It'll be interesting to see how she responds.

net neutrality economics reading list

Some time ago, Tim Lee was asking for a reading list on the economics of net neutrality (among other things.) Over the past week, I wound up doing a lot of reading on that subject in the context of the AT&T/BellSouth merger. In the spirit of blazing a trail for others, here's an annotated bibliography.

None of these papers gets into heavy economics. If you're seeking quantitative models, confidence intervals, and empirical data, you won't find it among these articles, though you might find it in their citations. The lack of empirical data sometimes causes problems for the articles, such as when the author tries to compare the magnitude of two separate effects.

• Tim Wu, The Broadband Debate, A User's Guide, 3 Journal on Telecommunications & High Technology Law 69 (2004), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=557330. This paper is a good starting point. It does a reasonable job of laying out the various positions and identifying the main points of contention.
• Christopher S. Yoo, Beyond Net Neutrality, 19 Harvard Journal of Law & Technology 1 (2005). A good second article. Lays out many of the economic objections to net neutrality.
• Christopher S. Yoo, Would Mandating Broadband Network Neutrality Help or Hurt Competition, 3 Journal on Telecommunications & High Technology Law 23 (2004), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=495502. Yoo's later Beyond Net Neutrality restates and expands most of the arguments in this paper--including paraphrasing the opening sentences--so I would suggest reading it, instead. But I couldn't find a copy of Beyond Net Neutrality on the Internet. So if you can't get to a law library to pick up an actual, paper copy of a journal, you can get many of the arguments here.
• Rob Frieden, Network Neutrality or Bias?--Handicapping the Odds for a Tiered and Branded Internet, 29 Hastings Communications and Entertainment Law Journal 171 (2007), draft available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=893649. Does a good job of discussing peering and transit agreements and the billing concerns behind them.
• Barbara van Schewick, Torwards and Economic Framework for Network Neutrality Regulation, 5 Journal on Telecommunications & High Technonology Law 329 (2007), available at http://ssrn.com/abstract=812991. Contains an interesting discussion of effects a network provider with market power can have on complimentary markets and the associated incentives.
• Christopher S. Yoo, Network Neutrality and the Economics of Congestion, 94 Georgetown Law Journal 1847 (2006). I would have liked deeper analysis. The article's thesis relies heavily on the economic effects of congestion, but it provides very little evidence that congestion in the network (as opposed to throttling at the endpoints) occurs often enough to be significant. I lost patience about the time the author began claiming that content delivery systems (i.e. akamizing) constitute putting functionality into the core of the network. I think you can safely skip this one unless you're seriously into congestive effects and economics.
  
• Adam Thierer, Are "Dumb Pipe" Mandates Smart Public Policy? Vertical Integration, Net Neutrality, and the Network Layers Model, 3 Journal on Telecommunications & High Technology Law 275 (2005). Strong on advocacy, short on economic analysis. The author is both Senior Fellow and Director of the Center for Media Freedom at the Progress & Freedom Foundation, which bills itself as a think tank. If it's economics you seek, this is probably not the article for you.

While looking for web sources for some of these articles, I also came across the Net Neutrality Reading Room at the Center for Democracy and Technology which also links to a number of sources and annotates many of them.

strange goings-on in the neighborhood of late

This neighborhood is getting odd. Not the 'net neighborhood, the physical one. Events so far this week:

• A paramedic and fire truck pull up in front of the neighbors late at night. No evidence of smoke. After a while, they leave.
• There's a high-speed chase down the street, complete with squealing tires as the pursue-ee makes a hard right turn three houses down, closely followed by at least one pursuing cop car and a helicopter.
• Three cops show up at the neighbor's place (same neighboring apartment complex as for the paramedics, not sure if it's the same apartment). One knocks on the door, two others stand low on the stairs to be out of view from the peephole. They go inside and, after a while, come back out and leave.

Someone with Partners for Safe Neighborhoods came by yesterday, going door to door to promote safety. He mentioned possibly forming a neighborhood watch. It's beginning to look like that might be a good idea. This whole area is transitory renters, so anything to increase the sense of community would be welcome. Does anyone know how much of a time commitment organizing a watch takes?

skateboarding bulldog

This video's been making the rounds. A skateboarding dog! Too cool!

(Yeah, that's about as insightful as it's gonna get right now. I'm just not up to tackling Massachusetts v. EPA till the head and stomach settle down. Lujan and Chevron and statutes, oh my!)

power pole microturbines

Made it to campus briefly to drop off paperwork. Now I'm back on the couch.

Through the screen door, I can see a utility pole, carrying the neighborhood's above-ground electric lines. I noticed that it's taller than most of the trees. And that the trees are blowing in the breeze. Hmm. There's wind up there.

I was wondering if it'd be useful to build a microturbine that could bolt to the top of a utility pole. It'd need to be auto-feathering to avoid blowing the pole over in high winds and would need an inverter and some other fancy electronics to match the phase of the power line, but if you had all that it could pump electricity directly into the grid whenever the wind blew, supplementing the main power plants.

After some back-of-the-envelope calculations, it turns out it's probably not worth the trouble. Let's use Del Mar, California as an example. They have (or maybe had) 184 utility poles and a population of 4389. That's 24 people per pole. The average Californian uses 7,000 kwh/yr of electricity (or 798 watts if you translate to civilized units), while the average American uses 12,000 kwh/yr, or 1368 watts. So the demand is 19,035 watts per pole in California, 32,631 watts/pole outside it. Windside's cute little micro-turbine, model 4C, produces 240 watts at a wind speed of 15 m/s (33 mph). But it looks like Del Mar's in a "marginal" wind power area with average speeds of around 13 mph at altitude. Assuming the turbine's output varies linearly with wind speed, bolting one to the pole should give you about 95 watts/pole. That's 0.5% of consumption per pole in California or 0.3% for the rest of the country. So you'd need a bigger or more efficient turbine, less energy consumption, or more poles to make a significant difference.

To bad, too--they'd look cute spinning up there.

not dead yet

This stuff still has me on the couch, but I'm slowly recovering. The fever dropped yesterday. Today, my brain's still pretty mushy and I feel like crap, but it's at least a somewhat better grade of crap.

I had to leave the house yesterday for, of all things, jury duty. You see, they originally wanted me to serve during the semester, which would've been quite nasty--imagine missing two weeks worth of classes, or four if it's a long trial. Finals won't wait, being a student isn't an excuse from jury duty, and unlike businesses I don't believe schools need to make accommodations. (What would they do, anyway? Write a special final for me and somehow try to wedge it back into the curve?) So I postponed jury service. But the voice menu system gave me only a six month range, which didn't help much--even pushing it back the full six months would've put it right in the middle of studying for the bar exam, definitely a career limiting move. So I did what I could to mitigate the damage and scheduled it to run over spring break.

However, you only get to do the "postpone for any reason" thing once. Then the flu came and knocked me flat on my keester. There's a two-week leeway built into the system, but that'd land me right back on top of classes, so it was no help. And the last time I tried the voice line it connected directly to the automatic system with no option to talk to a human, and I wasn't optimistic it'd be any better in the morning before a 7:45 a.m. reporting time. So yesterday I was pretty much trapped and, by operation of the rules of jury service, I had to risk infecting the jury pool. That's the danger of bright line rules, I guess.

I was pretty careful to minimize interacting with people--mostly just sat in my chair like a lump listening to the Hitchhiker's Guide to the Galaxy on the iPod--and would've explained it to the judge if I'd gotten called, but fortunately God was looking out for all involved and I wasn't called for anything. It did set back my recovery a bit, though.

pen names and blogging

A blogger receives death threats. Those threats cause her to cancel speaking engagements and to barricade herself in her house. The incident is causing quite a stir across the blogosphere, with people leaping into the discussion on all sides.

First of all, s0 that there is no ambiguity, I think some--maybe even many--of the comments people posted about Kathy Sierra crossed the line. Two lines, in fact. The first was when they went from criticism to ad hominem attacks. That's the line that separates useful discussion from useless noise. The second was when they went from insults to threats and intimidation. That's the line that can, and ought to, put you in jail.

But one post that I thought raised some difficult issues is Brad Feld's, entitled Anonymous Bullies, in which he writes this:

Until recently I didn’t think much about the difference between an anonymous comment and one where the person identified themselves. I’ve been spending a lot of time recently thinking about reputation and trust, especially given the geometric growth in user-generated content.

Kathy’s story sealed it for me – reputation and trust are at a tipping point and are an issue that is going to have to be dealt with in 2007.

As someone who writes under a pen name ("pseudanonymously," in the security jargon), reputation is an issue that strikes home with me.

Pseudonyms serve a valuable function. I chose to write under a pen name for a number of reasons, including privacy considerations, the fact that this blog is an experiment, and that it allows my online persona to differ from my real life one. I agree that reputation is an important check on conduct within the blogosphere, so I use this name consistently.

There are some problems, though. Some of them technical. For example, if you look around the Internet, you'll see comments labeled "False Data" with a link back to this site. Because the software and protocols aren't there, though, there's no way to know whether or not it was me who posted those comments. However, this exact same problem would occur no matter what name I used. For example, if you saw a comment from someone named "Brad Feld" which didn't appear on the Feld Thoughts blog, it would be hard to be confident that the person who wrote it was, in fact, Brad Feld.

We can solve half this problem pretty easily. I could, for instance, put a copy of every comment on this blog. Then you could check the comment on someone else's blog against the copy on mine and be fairly confident I wrote it.

But the other half of the problem is harder: suppose I decided to take a swipe at someone but I wanted to be able to deny it. I might post a comment on their blog but not put a copy on my blog and then claim it was someone else who wrote the comment. There are ways of solving this part of the problem, too. For instance, your software, which shows you the comments, could automatically check each comment it shows you to see if a copy appears on the blog of whoever claimed to write it and mark a red X through any comment that doesn't have a copy on some blog somewhere. Then you might not trust the source of a comment that has a red X through it. (This isn't a complete design--there are ways to defeat it and ways to implement it more efficiently--but you get the idea of how it works.)

There is also software that claims to solve this problem, like the OpenID stuff LiveJournal uses, but it doesn't seem to be widely supported.

The idea here, and the problem we'll need to solve, is that, if you're going to rely on reputation, you need to have a way to be reasonably sure you're holding the proper person in (dis)repute.

attorneygate

Back in January, I wrote about the developing U.S. Attorney scandal. Now we're starting to see more details, such a story in the LA Times that says this:

On Tuesday, Iglesias, 49, told the Senate Judiciary Committee that two prominent Republican politicians had called him to ask whether indictments would be filed before the November election against Democratic politicians in an ongoing criminal investigation. In the weeks that followed, Iglesias and seven other federal prosecutors were forced to resign.
. . .
. . . He earned the ire of the state GOP by refusing to prosecute anyone for voter fraud after the 2004 elections, despite some Republicans' contention that 15-year-olds voted. Iglesias said he could find no federal crimes.

The story notes that the second call was from Senator Pete Domenici.

As far as I know, one of the functions of a federal prosecutor is to not bring charges where the prosecutor can't find a crime: the prosecutor has an ethical duty to seek justice. If Congress wants to hold hearings, that's a different matter. As a deliberative body, Congress is set up to perform those sorts of investigations.

Something about this whole situation just feels wrong. I can't put my finger on exactly what it is just yet, but something is rotten in the state of Denmark.

nerdcore

I haven't been paying much attention to nerdcore, hardcore rap for computer geeks, but a former co worker just forwarded a YouTube link to a very cool piece.

For those not familiar with the Unix references in the rap, sometimes you need to stop a runaway program. On a Windows machine, you'd use the process manager (the thing that pops up when you hit ctrl-alt-del on newer Windows versions). On Unix, you use a command called "kill." There are various forms of kill that have different levels of severity.

• The normal "kill" will send a "terminate" signal asking the program to stop running. The program may ignore the signal.
  
• "kill -1" sends a "hang up" signal. Some programs will respond to this signal by reconfiguring or restarting themselves.
• "kill -9" tells the operating system "do whatever it takes to shut this program down now." It does not give the program time to react. It simply takes the program down immediately.

In that case, why not use "kill -9" all the time? Many programs, as part of shutting down, will do things like record configuration changes you might have made or data they might have received. If you take the program out without warning, it won't be able to write that information and could lose data. So normally, you'll escalate to "kill -9" only in the most severe situations.

(too much?) more on the Cathedral and the Blogosphere

Apologies in advance for a long post. While my column space may be unlimited, I'm very much aware that the reader's attention isn't. On the other hand, I wanted to give this topic the space it deserves because good quality information distribution is essential to the proper operation of a democracy.

Tim Lee's interesting response points out two difficulties with my critique about bloggers and journalists. First, he feels that I give newspapers too much credit, and blogs too little, when it comes to depth, and second, he is skeptical that newspapers have the advantage in integrity.

Laying the Groundwork

To respond to these points, I need to lay some groundwork. I'd like to look for structural advantages and disadvantages on both sides, so let's also expand the focus to professional media outlets, including newspapers, radio, and TV, on the one hand and, on the other, blogs, podcasts, and vlogs. Since we need names for these groups to be able to talk about them, I'm going to call the first group "centralized media" and the second "distributed media." The question I'm trying to answer is what, if any, advantages does centralized media offer that distributed media doesn't.

Defining Centralized and Distributed Media More Closely

To define them a little more closely, here are the main traits I see of each group. First, centralized media:

• a larger number of people operating within a business entity
• the people have formal training in journalism
• there's usually a hierarchical organization in which a small editorial staff must edit, review, and approve anything a larger staff of reporters writes
• the business entity generates revenue and attempts to operate for a profit
  

For the moment, I'm ignoring the functions outside reporting and editing. They may be significant, such as (as Lee points out) advertising potentially distorting integrity, but I'll deal with them in the context of talking about profit motives. I'm also leaving ownership of a printing press or transmitter out of the equation.

Next, distributed media:

• each producer typically has a small number of people, often just one
• the people may have strong expertise, but it is usually in a field other than journalism
• most of the organizational focus crosses producer lines, is informal, and is fluid, typically in the form of a web of trust
• profit is usually not a strong motive
  

Again, I'm leaving out the actual medium, the web server and associated software, because there's little to stop a newspaper or TV station from running a corporate blog, and I've already agreed that easy linking and comment feedback are good things for both forms of media. On the other hand, if the fact that a lot of people are involved in a centralized media producer made it less responsive to criticism, that would be a more fundamental issue that's independent of the distribution technology it's using.

Clearly there's a continuum between centralized and distributed media. A one-person community newspaper published as a labor of love starts to look a lot like a blog, for example. Ars Technica, on the other hand, seems to have many of the traits I've assigned to centralized media (though unfortunately their "about us" page doesn't talk about their editorial structure.)

The Long Tail Phenomenon

Finally, Lee invokes the long tail phenomenon, asserting

The average blog post is almost certainly of lower quality than the average newspaper article. But because there are so many more blogs than newspapers, and because there is such a great diversity of bloggers, the best blog posts are often better than the best newspaper articles.

I agree that the number of blogs is an advantage we shouldn't ignore. However, there is also a small number of very good centralized media producers among a lot of marginal ones, so let's focus on comparing the best with the best on both sides in order to focus on the structural issues. I will pull in averages only where they illustrate the influence of the structure. For example, it might be necessary to talk about averages when discussing the influence of the profit motive on journalistic integrity or the influence of "link love" on fact gathering through a web of trust.

Analyzing News Versus Gathering It

I would expect distributed media to be very good at analyzing news, but less good at gathering it. By "gathering news," I mean interviewing people, conducting (or commissioning) and collating research, and digging up facts. It includes everything from sending a war correspondent to Iraq, to putting a reporter on Air Force One, to interviewing people, to flying a helicopter over the latest traffic mess or forest fire.

News Analysis

The main strength of distributed media in news analysis comes from the fact that the good producers often have a strong background in the fields they analyze. For example, many authors of academic blogs are world-class experts in their particular specialties.

One influence that works against distributed media in news analysis is a lack of review coupled with the large number of producers. Most academic journals (with law reviews a notable exception) are peer reviewed, but most academic blogs are not. Centralized media outlets also have a layer of editorial review, though I've suggested that editorial staff really works more towards giving the entity a consistent outlook than one that's necessarily always balanced. For example, Fox News is consistent in how it presents the news. Whether it's balanced seems to be debatable. Note, though, that there are fewer centralized media producers than distributed media producers, and each centralized producer pumps out more stories, making it easier to evaluate each centralized producer's point of view. In contrast, distributed media relies on the web of trust. Unless the person doing the evaluation happens to personally be an expert in , say, California Civil Procedure, Venture Finance, or Security, that person will have to rely on someone else's evaluation. I've suggested the web of trust makes it harder to properly evaluate the analysis the further in the web you are from its source.

On the whole, though, I think the level of expertise probably outweighs any web of trust issues. I would expect the web of trust to be able to flag egregious problems, such as a supposed environmental expert quietly being on Exxon's payroll, and I'd rather hear directly from the expert than to hear the expert's views clipped and distorted through a reporter.

News Gathering

When it comes to gathering information, I think centralized media has the advantage for two reasons: money and access. It is the rare blogger who could afford a helicopter to fly over a forest fire location, and even if the blogger could do so, getting the FAA to grant access to the airspace would likely present a problem. Similarly, the Whitehouse restricts access to press conferences, if for no other reason than security. We may see more distributed media in these roles over time, but for the present having the money and contacts to put a reporter on location gives centralized media the upper hand in this area.

Every generalization has its exceptions. For example, distributed media would probably do quite well gathering information about software or appliances, things that are relatively accessible and inexpensive. A prime example would be breaking the Sony DRM Rootkit story. Similarly, reporting on the latest court decisions using public filings as primary sources is an area where distributed media should do well, particularly when those filings are public records available from web sites. On the other hand, I would not expect a distributed media producer to spend a few hundred hours compiling an empirical study unless it had an outside source of revenue, such as a research grant. To be fair, the number of times centralized media is willing to commit a few hundred hours of reporter time to a project is pretty limited, too, but remember that we're comparing best to best here.

Journalistic Integrity

The fundamental issue with journalistic integrity is whether the media producer recognizes and discloses conflicts and produces fair and balanced reporting. (I am using the term "balanced" to mean the reporting is not too slanted. I doubt it's possible to eliminate all slant, but it is possible to make a conscious effort to limit it.) Let's look at the motivations in both cases.

Their Main Motivations

For centralized media, profit is an important motive, both for the media outlet as a whole, and for individual participants in terms of their salaries. Another factor is reputation, again both for the outlet and for the individual: if you plagiarize or fabricate stories, you might find yourself unemployable as a journalist. Journalists receive training in recognizing conflicts of interest and the importance of full disclosure, though Lee has expressed skepticism about how effective that training is at changing the conduct of the average journalist in the normal case.

For distributed media, currently fame and recognition, what Richard Bennett called "link love," seem to be important. Others might produce because of a strong commitment to a particular set of policies, but I'd lump that in with fame and recognition, not for the author but for the policies.* Reputation remains an important factor. Distributed media producers do not generally have training in recognizing conflicts of interest or full disclosure unless they seek it out.

Profit/Recognition Motive Versus Reputation

The profit motive for centralized media and the fame/recognition motive for distributed media seem to be a wash. For example, whether a newspaper has a reputation for being balanced or for being best used to wrap fish does not seem likely to affect revenue from its classifieds. A newspaper that's known for stellar reporting may get wider readership, but so might a newspaper that regularly prints a full-page pinup in each issue. On the other hand, one notable effect money often has is to get the individual participant to take the job more seriously. For instance, as I mentioned earlier, I can budget more time and attention to an activity that puts food on the table.

It's not clear to me what effects a decrease in reputation would have. This is an area where I'd love to see some empirical studies comparing reputation for balance with number of readers, listeners, or viewers. What I would expect for centralized media is that declining reputation would shift the demographics but would not be closely correlated with a shift in total audience size, so the impact on revenue would largely depend on what advertisers are willing to pay for a given demographic. For distributed media, a similar phenomenon might occur, where who's linking to the site might change but the total number of links might not. What effect that might have on fame and recognition seems would depend on what audience the producer is trying to reach. Note that distributed media's audience is part of the web of trust, so rearranging the web means reaching different people but doesn't necessarily mean reaching more or fewer people.

Training in Journalistic Ethics

Finally, education in recognizing conflicts of interest might shift the total balance for two reasons. The first is that it equips the reporter to recognize those problems ahead of time rather than learning about them by falling into them. The second is that the reporter might have a stronger commitment to fair reporting as a result of the training. Lee gives these influences less weight than I do, but I suspect we won't be able to resolve this disagreement without more information.

----

*There's a third group: people who blog to stay in touch with friends and family. I've left them out of the analysis because they don't seem likely to be the sorts of news gathering and analysis producers we're talking about here. Omitting them might be wrong, though, because if you believe they are an important part of the web of trust for distributed news gathering, it may be necessary to look more closely at how their motivation affects the way they filter information.

more on bloggers and journalists

Tim Lee, who is obviously better at checking the links to his articles than I am, has a couple thoughtful responses to my critique.

The second article tackles my criticism of the difficulty blogs have in making sure their reporting is accurate. Lee describes the process this way:

[A]ny given reader doesn’t read “the blogosphere.” They read 10-100 specific blogs. And they tend to read the same blogs, day in and day out, for months or even years. So every blog with a non-trivial readership does have a significant number of people tracking its reputation.

Moreover, bloggers read each other, creating a web of trust that’s an important part of vetting information on the blogosphere.

I don't know the original source of the web of trust concept, but I first encountered it in cryptography. Let's suppose you want to make sure that the person I am in real life is the same person who posts on this blog. If you know me, it would be easy enough to know for sure. You could simply ask me ahead of time to put something distinctive in a blog post and then check to make sure it showed up. But suppose you don't know me, but you know one of my friends. You could trust that friend to make sure it's me. And so on through friends of friends.

Of course, you might not trust that friend 100%. Or you might think there's a chance that friend, even acting in good faith, will simply make a mistake. So if you're a goodly distance away, you might check with two or three friends and hope they know me through independent paths, so that a mistake up-stream--a screw-up by a common friend somewhere close to me--doesn't throw off your results. But that increases the effort you have to expend to find out if the information is good or not. In other words, it increases your transaction costs. And checking the facts of a news story is more involved than the simple yes/no question of whether X is me.

Now, that's not to say reporters don't have the same problem. They definitely do, especially in an age where they seem to insist on interviewing other reporters rather than going to primary sources. However, one thing a newspaper is able to bring to the mix is that it shortens the web of trust. The reporters who work for a newspaper have a strong financial incentive to follow whatever tone the newspaper's leadership sets for fact checking. And the newspaper puts out a lot of news articles. So even if an individual reporter writes rarely, if you assume that reporter is generally following the newspaper's lead, then you have a lot of data with which to build a fairly complete picture of the newspaper's overall tendency to report accurately and carefully (its quality) and the distortions it routinely introduces (its slant). It's the same process that happens with blogs, but you're playing the game of telephone with fewer links in the chain. Note that these are essentially the same ideas Lee expresses:

[M]ost of the contributors to a newspaper are not widely known by their readers. In those cases, all of the vetting is put on the shoulders of the paper’s editorial staff. You have to take it on faith that the editor of the paper has good judgment in choosing reporters.

We're just interpreting the impact differently. Lee rightly points out that if you conclude that certain reporters at the paper (or at Reuters or AP, for that matter) are not trustworthy, you don't have many options but dropping the entire paper. I agree, but I don't think that's a problem as long as you have enough papers and other news sources to turn to.

I agree with Lee that the comments section of a blog is a major improvement over a newspaper's editorial page. So is the fact that one blog can so easily link to another, and that the search engines will index all those links. And based on Richard Kuttner's piece, I think he would agree as well. I certainly hope to see either more news organizations allowing reader comments, or the spread of technologies that allow readers to comment on web pages regardless of whether or not the page itself has a comment feature.

My Monday schedule is, as usual, a total mess, and I want to give Lee's other article some careful thought before I respond. I'll hopefully have something up about it later today or tomorrow.

bloggers and journalists

Tim Lee, at the Technology Liberation Front, posted an article discussing the similarities between blogging and newspapers, on the one hand, and open source and the traditional software companies, on the other (think Eric Steven Raymon's The Cathedral and the Bazaar). His article, in turn, links to a Richard Kuttner piece on the Columbia Journalism Review. I've been thinking about a similar topic a lot, recently, and the conjunction sparked a few ideas.

So, leaving opinion writing aside, what do newspapers have that blogs don't?

Lee, a libertarian blogger, seems to say "not much," at least long term, outside of specialized areas like war correspondence. He points out that pervasive technology means there's most likely to be a blogger somewhere near where things are happening, and the information will get picked up by other blogs. He also points to investigative journalism that's now happening in blogs, and that newsmakers can now communicate directly with the public.

In contrast, Kuttner, a long-timer journalist and columnist, says that "the best material on the Internet consistently comes from Web sites run by print organizations." But I'm having trouble finding why he thinks that's so--not surprising since his focus is really on how the newspapers can survive. However, if we don't know why it's so, we can't justify faith that it will continue to be so.

One point Kuttner seems to make, with which I agree, is that depth is expensive. Most of what I write here is opinion, because opinion is cheap. You won't find many really in-depth articles because I don't have the time to research them. If you paid me a salary to do it full time, that might change, but as it is I--like many bloggers--am sharply limited in how much time I can devote to this project. While Lee bridles at the term "amateur," I am exactly that: blogging doesn't put food on my dinner table, so I have to budget my time accordingly.

Two other points Kuttner raises are civic-mindedness and, implicitly, journalistic integrity. Journalists take classes on journalistic integrity, though there are some cases where I have to wonder whether they're universally effective. Blogging is a new enough medium, run by people who generally lack journalist training, that it's still grappling with issues like disclosure. For example, Joel Spolsky writes about being approached to do product reviews and being allowed to keep the product, raising a potential conflict of interest. It's possible a popular blog could support its author full-time, but that author may quickly find herself at sea and losing credibility without a commitment to integrity, full disclosure, and balance. Similarly, without that commitment, civic-mindedness can quickly become punditry and nothing more. The other side of the problem can happen with the bloggers doing fact gathering that Lee mentions: their main incentive to be fair and balanced is reputation, but how do you track the reputations of millions of amateur reporters in the field around the world?

So I think that's what newspapers can really offer to the mix: ideally, they can concentrate enough people with the proper training and resources to bring consistent depth and balance to their reporting. To the extent a newspaper just reports the facts or echoes a press release, it contributes very little beyond the convenience of gathering that information to a central place. Its real value comes from context, fact checking, and balance. If you leave out the actual printing press, I see blogs on a continuum with newspapers--most blogs are like one-person papers, usually opinion-heavy, often run on a shoestring, much like 21st Century pamphleteers. As you start adding people, expertise, money, and resources to a team blog it begins looking more and more like a newspaper.

Correction: the author of the CJR piece is Robert Kuttner, not Richard Kuttner. My apologies to Mr. Kuttner.

extremism: a follow-up

This is just a quick follow-up to last night's post on extremism. There is, of course, more going on than just intolerance run rampant. Jared Diamond's books Collapse and Guns, Germs, and Steel make a pretty good case that one of the things we're seeing is resource constraints, such as the effects of intermittent electricity, extremely high fuel and food prices, and rampant unemployment. Essentially, he makes a pretty good case that, when a human population's resource base shrinks below the level that will sustain that population, one of the things that tends to happen is enough warfare to reduce the population down to a sustainable level. Unfortunately, I've lent out my copies of both books so don't have them here to get into more depth. The scary implication, though, is if Diamond's right, then there may be no good reason to think it couldn't happen here.

extremism

I recently saw "The Bitter Truth" text. It's that list of reasons a good Muslim can't be a good American. I couldn't find the original source of the text, but you can read essentially the whole thing here. I'm not going to reproduce it in this post. You can also read a pretty good point-by-point response here.

I get concerned when I see text that describes a people as fundamentally different and incorrigible. Too many wrongs have followed painting with that kind of broad brush. What's more, this description of Muslims simply doesn't fit my personal experience. The Muslims I've known personally have been as varied as the Christians, the Jews, the agnostics, the atheists, and those following pretty much every other belief system. In fact, none of the Muslims I've known has been as intolerant as, say, the followers of Fred Phelps, but that's probably just because I'm more likely to run into a Christian than a Muslim in this country, so I also have a better chance of bumping into an extremist Christian than an extremist Muslim.

And it seems to me, when you get right down to it, the real problem is extremism. It's people not willing to tolerate other people who don't look like them, or talk like them, or worship like them, or think like them. It's xenophobia. And it's bigotry. It's been with us for longer than recorded history. And neither a Muslim suicide bomber in Iraq nor a white supremacist American with a Ryder truck in Oklahoma City has a monopoly on it.

you know it's the 21st Century when . . .

You know it's the 21st Century when the neighbor kids are playing Red Light / Green Light . . . while riding around in circles in a two seat electric toy car.

Sure wish I'd had a two-seat electric car when I was playing that game.

OK, back to work. The air is thick with "Red light!" "Green light!"