February 6, 2024. This essay reflects on what made Bell Labs so successful as an “institute of creative technology”, why that success has been so hard to replicate, and how one might attempt to replicate it in a distributed, self-organized, and rebellious fashion.

Bell Labs was a cool place. Over the course of a century, its researchers invented/discovered lasers, the transistor, information theory, modern cryptography, error-correcting codes, the Cosmic Microwave Background, electron diffraction, photovoltaic cells, quality control, CCDs, optical fibres, UNIX, C, support vector machines, the Fast Fourier Transform, Grover search, and Shor’s algorithm, among other innovations too numerous to name. What made it special?

Money, to begin with. Bell Labs operated under the auspices of a state monopoly, so no one had to hustle for grants, venture capital, or quarterly revenue. But most steadily funded researchers don’t win a Nobel Prize or a Turing Award. By itself money doesn’t make good research; it’s something you worry about when there isn’t enough! Of course, good industrial research is expensive, and more money means more talent, more equipment, and more time. But outcomes are not linear in these, and from a quick survey of well-heeled Ivies or the moonshot factories of Big Tech, its clear that money does not buy Bell-level success.

I think the special sauce was not material—the cash flow or the personnel or the supply of oscilloscopes—but philosophical. This essay explores that philosophy in a little more detail.

Contents

1. The Kelly gang
2. Fine-tuned freedom
3. Recipe for successipe
4. More cowbell
5. Scaling down
6. Rebel labs

1. The Kelly gang

I stole “institute of creative technology” from Mervin Kelly, director of research and later president of Bell Labs. See “The Bell Telephone Laboratories-An Example of an Institute of Creative Technology” (1950), Mervin Kelly. All quotes of Kelly are from this source. A physicist by trade, Kelly played a pivotal (though not quite Nobel-worthy) role in the invention of the transistor, and as president, helped develop the laser and the solar cell. He was the scientist-manager par excellence; to paraphrase Falstaff, not only the cause of research in himself, but the cause that research is in others. At the same time, his views on scientific management would terrify a modern CEO. Fundamental research was a “non-scheduled area of work”, so the scientists did not have to jump through the corporate hoops of OKRs, deadlines, or progress reports. No hustle, no crunch; just research.

The buildings were designed to facilitate cross-talk, stimulation, and focus. Kelly oversaw that geomancy as well. As he put it: “We give much attention to the maintenance of an atmosphere of freedom and an environment stimulating to scholarship and scientific research interest. It is most important to limit their work to that of research.” An ill-framed objective might distract from their real job.

Of course, to meet the technical needs of a giant like Bell, someone had to keep track of what was going on, and do something with the research. Once a fundamental research project reached maturity, it would be handed over to a development team. In Kelly’s words: “This pattern repeats itself again and again. A research programme is initiated. Then as new knowledge that gives promise of worthwhile application is obtained, a fundamental development team is activated. It builds a background of basic technology under the watchful eye and with the consultative aid of the men of research.” Researchers consulted, but were strictly forbidden from helping to develop their research, lest they “lose contact with the forefront of their field of scientific interest”.

Development was only one of a number of concentric layers of deployment, including systems engineering, facilities development, and operations. It’s clear that Bell put a tremendous amount of effort into structural policy and effective divisions of labour. But these divisions really only imposed boundary conditions; what happened within a division would be governed by differential equations set by management. Kelly could have treated his researchers as boffins in need of practical guidance, or drones requiring coercive instruction. It happens. Instead, as the transistor episode shows, he saw them as collaborators, and directed their efforts with superb managerial and technical nous. He lead from the inside.

One might be inclined to view this “institute of creative technology” as a lucky and unrepeatable confluence of factors. But Kelly directed Bell Labs for less than 10 years; there were many Kellys. The only way to sustain success at that level for a century, I would argue, is to marry material stability and good structure with a philosophy of creative applied research that works. Kelly gives us hints about this philosophy, but to help fill out the negative space, we can look at institutions which do not give rise to exceptional applied research.

2. Fine-tuned freedom

The Institute for Advanced Study has more material security, more on-paper freedom, more vertiginously soaring ivory tower, than one could poke a tenured chair at. But its eminent faculty and promising young postdocs are there, respectively, by postselection (what they did before they arrived) and preselection (what they will likely do after they leave); not much happens at the Institute itself. As Feynman memorably put it: “Nothing happens because there’s not enough real activity and challenge: you’re not in contact with the experimental guys. You don’t have to think how to answer questions from the students. Nothing!” From “Surely You’re Joking, Mr. Feynman!” Adventures of a Curious Character (1985), Richard Feynman. The IAS is too far away to have an impact on reality.

At Bell Labs, Kelly made sure the theorists were in contact with the experimentalists by literally building their offices next to each other. People would talk, share problems, and create interest groups across disciplinary lines. The solid-state group that invented the transistor started as an informal study session lead by William Shockley. Claude Elwood Shannon might ride his unicycle down the hallway to talk to Tukey, Bardeen or Richard Hamming, before retreating into solitude; the metallurgists taught the chemists a cool trick about polymer synthesis, and later that day, a chemist nerdsniped a materials physicist. Magic would happen.

But it wasn’t just a critical mass of talent and cross-fertilizing expertise: the research was ultimately grounded in the ever-shifting operational requirements of the world’s largest telecommunications enterprise. This did not literally mean solving Bell’s operational conundra; rather, through good management and institutional structure, researchers would use their freedom to obey Paul Graham’s dictum: “live in the future, then build what’s missing”. Or, as Kelly put it, the idea was to provide “the coupling between the ever-advancing forefront of pure science and the forward march of our communications technology. The closer the coupling, the more completely will we keep in step with the progress of science.”

This is worlds away from the enfeebled freedom that Feynman lampooned. The freedom to think, sure, but about what? In contrast, if you’re put on a team with a provocatively hard domain of inquiry, told to get darn good at it, talk to your colleagues, think big, ask stupid questions, and go test things out in the lab down the hall, you end up with an institute of creative technology and not of advanced study.

This sounds great and all, but by the metric of domain complexity, Big Tech should be the new Bell Labs. Epitomized by MAMAA (Meta, Amazon, Microsoft, Apple and Alphabet), Big Tech does indeed have patents, papers and products aplenty, with a track record we can read off Nasdaq. But despite the billions of dollars they pour into applied computer science, these companies share only seven Turing Award recipients between them, And a few of these recipients, e.g. Yann LeCun, did their most highly cited work at Bell Labs. while Bell Labs had nine. Arguably, this a problem of timescale; like the Nobel, Turing Awards tend to arrive long after the fact. In twenty years, I bet we will see more MAMAA employees on the honour rolls.

But I wouldn’t bet on too many more. The simple reason is that Turing awards are not a good investment, and in the cutthroat landscape of technoprogressive disruption, you “move fast and break things” or you die. You don’t have time to sit around doing fundamental algorithms research. This capitalist realpolitik explains why, for instance, Alphabet is making massive cuts to its “moonshot factory” X. It also explain why research groups in MAMAA and beyond have aggressively pivoted into generative AI. Transformers are interesting, but a solved problem from a fundamental science perspective; the rest is engineering, albeit lucrative and difficult. I’m exaggerating a little, especially because much of the basic science was done in the Big Tech setting. See §2.4 for more on this.

With a state-backed monopoly, this realpolitik didn’t apply to Bell. The product ensured a healthy supply of capital and research questions, a “problem-rich environment” as one employee said. But the research was decoupled from the profitability of the system it served. Employees could afford to fail, could afford to dream, could afford to talk to people outside their comfort zone and explore low-entropy connections; the product didn’t govern the outcome, dictate the process, or provide the yardstick for success and failure. In Big Tech, the product is the outcome, the process, and the metric.

3. Recipe for successipe

So, to state it crudely, the IAS is all freedom and no application, and Big Tech is all application and no freedom. Creative applied research needs both. Collecting our observations so far, we can list a few key ingredients in Bell’s recipe for successipe:

• Academic freedom
• Material security
• Problem-rich domain
• Division of labour
• Enforced focus
• Enforced interaction
• Collaborative management

This seems pretty good, but I’ll admit, there are some major questions it doesn’t answer:

• What motivated researchers?
• How did managers intervene in or guide research?
• How was research coupled to the realities of the communications systems?

I suspect the answers lie somewhere between the operational realm and the mysteries of human behaviour. Hire smart people, give them money, equipment, and fun problems to work on, and your output will probably depend sensitively on who you choose. Like scouting and talent development in professional sports, building a winning team requires more than deep pockets; you need sharp eyes and a gut instinct for how the different parts of the team will gel. As Brattain once said of the transistor research group, “It was probably one of the greatest research teams ever pulled together on a problem.” It’s hard to overstate the importance of collaborative synergy in big science.

Then again, Claude Elwood Shannon, the magician of Bell, invented information theory and modern cryptography single-handed; what gives? Another Bell luminary, Richard Hamming, viewed openness as the key criterion rather than collaborativeness per se: “there is a pretty good correlation between those who work with the doors open and those who ultimately do important things, although people who work with doors closed often work harder. Somehow they seem to work on slightly the wrong thing—not much, but enough that they miss fame.” See “You and Your Research” (1986), Richard Hamming. That openness is how collaboration and discovery happens. Shannon may have been a lone wolf, but he was modest, curious, and intellectually receptive in other ways.

The last two questions probably have the same answer. To couple the state of the art in communications systems to the state of the art in pure science, somebody had to have a foot in both worlds. That would be the director of research. As Jon Gertner puts it in his wonderful book, The Idea Factory: Bell Labs and the Great Age of American Innovation (2012), Jon Gertner. “the managers themselves had to keep track of how the technology and politics and finances of their endeavour meshed together.” The researchers, on the other hand, were tasked with being domain experts in the latter. So the coupling arose via interactions between the director, lab heads, and the constituent scientists. This was, incidentally, another set of concentric layers that ensured proper focus and divorced research from profit. Putting it all together, we have a preliminary recipe for creative applied research.

4. More cowbell

Of course, this raises the question: if Bell Labs made innovation systematic, why haven’t its results been systematically reproduced elsewhere? IBM and Microsoft have tried, with some luck. I don’t think Kelly’s management was a lucky confluence, but maybe there was a lucky confluence, namely Bell Labs itself; I mean, you don’t get a state-sponsored mandate to a develop a whole new realm of technology every day! Big Tech tries to develop new realms, but without mandate in a prohibitively competitive environment. The IAS gives a mandate, but an empty one. Money and freedom aren’t enough; you need a mission.

You might conclude pessimistically that Bell Labs’ mission—to help build a nation-wide telecommunications infrastructure—and the freedom and money that came with it, are basically unrepeatable. But I’m not sure I buy it. Or rather: I’m not sure anyone else wants to. Big Tech doesn’t like the ROI on Nobel Prizes, and for that matter, neither does Small Tech, whose heterogenous landscape of startups relies on ready revenue and ROI-haunted venture capital to stay afloat. The moonshot factories like X are perhaps too disconnected from the domain of application; that’s why you get quirky projects like the hyperloop or Loon, cute, brilliant, and totally unnecessary. The perfect fodder for xkcd!

Plenty of universities and state labs do creative applied research, but in isolation. Part of the magic of Bell was in the scale and connectivity. We can do a little back-of-the-envelope calculation to see why this is important. Suppose you have $n$ research groups, most of which talk to most of the others. Then the number of connections scales as $O(n^2)$. If profitable connections are distributed at random along edges, e.g. there is a chance that a good idea lies along an edge between two groups, then you have an $O(1)$ chance of finding it. If all groups speak to some $O(1)$ number of other groups, or some group speaks to most others, you have an $O(1/n)$ chance of finding it. Finally, if only a few groups speak to a few others, the chance is $O(1/n^2)$. I would wager that most universities live in $O(1/n^2)$ regime, and state labs in the $O(1/n)$ regime. The odds of finding those creative overlaps are not good.

An interesting exception is OpenAI, which (in my view) does seem to have a “culture of repeatable innovation” to unironically quote Sam Altman. I’m not sure how things work there, but I suspect that starting as a non-profit baked some kind of decoupling between product and research into the structure. It also had a Lawful Good mandate for a provocatively difficult problem domain. In a field like AI, the benefits of cross-fertilization between adjacents domains are probably pretty obvious, particularly as multimodal AI comes into its own. So, I suspect that the different teams of OpenAI are in the $O(1)$ regime for probability of discovering something new. I’m not sure about the other ingredients in our successipe—these may be neither nor sufficient—but it seems like OpenAI’s structural quirks are part of what makes it special. I guess we’ll see how that culture of innovation holds up under the pressures of Big Tech rockstardom.

5. Scaling down

In lieu of a state monopoly or an idiosyncratic Silicon Valley non-profit, will these guidelines be applicable? I think so, and at smaller sizes than you might expect. Armed with the insights above, let’s revisit the ingredients in our successipe, and see how it scales down:

1. Freedom. Academic freedom means that researchers and not managers initiate research. As in a university setting, this isn’t a license to do anything; it’s trust and permission within scope. This can be implemented even in a small team.

2. Security. This one is trickier. As we’ve discussed above, in tech, stable funding is hard to come by, and in most universities, even tenure is no guarantee of money. But this is where separation of concerns becomes important; the researchers should not be the ones hustling for grants, or developing and selling the product. They should get paid to research. Professors tend to be miserable project managers, and the same is true in industry.

3. Separation. The benefits of a robust division of labour should, by this point, be clear. Don’t saddle researchers with business problems, just pay them to think and give them what they need. For that to feasible, you need to have good managers to couple science to commerce, and teams to perform development, integration, and so on. Keep them separate, but with well-defined and performant interfaces. An early-stage startup of, say, five people is probably too small to separate this way, but $100$ is plenty.

4. Focus. The goal is clearly separating research from its application is focus. Researchers work best when their bellies (and wallets) are full, and they get to have fun with their problems. But, as Kelly realized, to keep up with the frontier you need to move fast; you can’t afford to be distracted. This is a subtle business, because what looks like distraction to the manager can be unconscious insight for the scientist. This is where scope and domain become important.

5. Domain. A stimulating problem domain guarantees a supply of good research questions, and that coupling to the scientific state-of-the-art Kelly was so concerned with. In fact, most industrial domains harbour unsolved scientific problems; we know much less than we think! In addition to stimulating scientific innovation, a good domain also helps draw the line between distraction and freedom judiciously employed. You think homotopy type theory might shed some light on how to generalize that algorithm for herding sheep? By all means. Managers shouldn’t try to control the outcome, but to nurture and guide the process.

6. Mandate. Of course, even when a domain is stimulating, if approached without urgency and a sense of mission, it is likely to yield lacklustre results. That is why mandate is important. It gives you a problem to solve, and the responsibility to solve it; a collective and personal stake in something that will make a difference. This contrasts with splendid ivory-tower isolation, where (if you’re lucky!) your results only matter to a handful of other experts.

7. Interaction. Speaking of other experts, the hardest ingredient to replicate at a small scale is the effect of cross-disciplinary chatter and all-to-all connections. With six groups, you have fifteen edges to explore; between two groups, only one. In smaller teams, I think the natural answer is in-house specialization, where different researchers play the same role as different teams. Another solution is to build rapport with external research groups; although better than nothing, like university collaborations it tends to suffer from suboptimal scaling in $n$.

8. Management. If researchers are encouraged to focus and become domain experts, some of the cross-fertilization has to be coordinated from above. In turns, this requires technical maturity and vision, the insight of scientist-managers like Kelly. Then again, as grumpy professorial PIs show us, not every scientist is cut out for this role; nor should they be. Some people are better at research, some at management, and some in between. Industry has the unexpected luxury of choosing people for the roles they are best at.

9. Talent. Kelly emphasized the importance of quality hires, of recruiting scientists of “the same high quality as required for distinguished pure research in our universities”. But there is as much managerial knack to scouting, building, and managing teams for scientific research as there is for professional sports, even if conventional ideas about genius (and conventional institutional arrangements) obscure this fact. Successful teams of any size are based on having the right people, with the right management, for the right task.

10. Coupling. The impact of applied research is related to how effectively coupled the research is to the problem domain. That, in turn, bubbles out of an alchemy of multiple elements. I can’t put it better than Kelly: “Years of experience have taught us that the type and quality of men selected for our research, the environment that we provide, and the distance in their work that we ask them to penetrate beyond the forefront of creative technology are the most important factors in determining the closeness of coupling—the effectiveness.” It evades mechanization because it seems that, after all, creative technology is an art and not a science.

All things considered, I see no reason why the recipe should not be attempted at a smaller scale and in less propitious circumstances. Maybe it is or has been, with indifferent results, and the experiment abandoned. But I wonder if it’s simply unpopular because it usually means large capital outlays on unpredictable outcomes like lasers that benefit everyone, including your competitors. I guess you pays your money and you takes your choice.

6. Rebel labs

So, let’s agree that this approach is due for a revival. How do we manufacture a monopoly, or conjure up a steady stream of coin? We can’t. Maybe sprawling, shiny facilities for $O(n)$ labs, with robust all-to-all chatter? Nope. In lieu of top-down technocapital benefaction, I wonder if it’s possible to implement a partial feature set from the bottom up. What might this look like? Let’s try following the recipe we worked so hard to reverse engineer.

A good first step is to gather a bunch of smart people(talent) who work well together, like to share ideas, and have a wide range of expertise (interaction). Give them a hard applied problem to work on (domain), induce a sense of mission Perhaps because the problem is urgent and unsolved. Or might lead to a cool startup. Or fame and glory. Or cookies. People are motivated by different things. (mandate), and give them the permission (or perhaps the obligation!) to creatively explore the domain as their antennae dictate (freedom). Distribute the effort via the magic of the internet, with tools for collaboration, feedback, and a high but not prohibitive threshold of participation (focus); people need to stay engaged, but not so engaged that they can’t afford to remain involved (security). Doing this solves the problem of security because people stay without having to pay them!

In this self-organized, downscale setting, having a managerial hierarchy doesn’t make much sense. But somebody should take responsibility for coordinating different strands of research to ensure that the different voices harmonize, like an orchestra conductor (management). Ideally, this individual—we’ll use the non-hierarchical term “convener”—is good with people, has a global sense of the domain so the research folks don’t have to (separation), and knows how to work the channel between the abstrusities of research and the real-world use case (coupling). Maybe the coupler and the convener could be different people, if the project gets large enough, but at first will probably both be whoever kickstarted the whole shebang.

All this may seem quixotic, but it’s analogous to the common model of open-source software. Coders around the world come together to create beautiful things, for free, in a distributed, self-organized way. The biggest difference between open-source software and what I’m describing is probably (a) the important of who is in the team, and (b) the sustained mental effort. I hope I’m being chauvinistic about both of these, because if I am, it means “open-source science” is even more feasible than I thought! I think the modern scientist has an unfortunate habit of waiting for institutional blessing before thinking. Another very encouraging example is the Polymath Project in pure mathematics.

So, this essay has really been a lengthy preface to a sales pitch: why not try the Bell Labs model in an open-source setting? I call this Rebel Labs for obvious reasons. I’ve put some example projects on the lab page; these are all things I am tentatively working on, and will add more details soon. But the point is, the projects will be a lot better if smart folks take interest and get involved! I don’t think we’ll invent the laser, but then again, Bell Labs shows that collaborative research synergy is at least as powerful as coherent light.