The Fork in the Road: AI Horseless Carriages or Collective Intelligence for Science

These early experiments swapped engines instead of horses, rather than rethinking vehicle design from the ground up. The AI horseless carriage story is at its core a warning: when a revolutionary new technology like AI is invented, the first tools built with it risk failure because they mimic the old way of doing things.

What are the “horse-carriage” risks in AI for science? And what is the “car” that we could be designing for instead?

I would argue that the AI horseless carriage is plugging AI into the existing publishing system. The result, taken to its somewhat absurd but not unlikely endpoint, looks like a loop of AI peer review on AI generated papers. I call this scenario a horseless carriage since papers and peer review in their current form are old ways of doing science that we already know are obsolete.

To be clear, AI automation is highly valuable for well-scoped tasks such as screening literature, extracting data, or running standardized protocols. The risk isn't automation itself, but automating broken systems. Such wholesale automation risks devolving science into a meaningless “doom loop”, with AI acceleration creating a kind of centrifugal force that deskills humans and pushes them out of the research loop (see figure below). We're seeing similar doom loops emerge in education, where AI-generated assignments meet AI grading systems, progressively removing students from meaningful learning while appearing to increase efficiency. This kind of automation is pernicious since in many respects it is the path of least resistance: it requires minimal institutional change and aligns with big AI companies' push to sell automation at scale. It also makes certain myopic economic sense, as cuts to science funding render human labor more scarce and incentivize institutions to look to automation as an imperfect but cheaper substitute.

But there is a promising alternative path, and what I would argue is the "car" we should be building: collective intelligence (CI) for science. In its broadest form, CI can be seen as “a collaborative approach to problem-solving, typically supported by technological tools, which allows for real-time co-ordination and mobilization of knowledge that is distributed among many individuals.”

The key difference between these paths lies in how AI relates to humans. In the horseless carriage scenario, AI automates tasks, deskilling researchers and pushing them to the periphery. In collective intelligence systems, AI instead amplifies and connects human thinking: facilitating collaboration, aggregating distributed knowledge, and helping groups coordinate at scales previously impossible. The focus shifts from automation to augmentation, and from substitution to synergy. In the horseless carriage scenario, each advance in AI capability tends to further marginalize human participation. In CI systems, improvements benefit both: better AI tools enable richer human collaboration, while more human participation generates better training data and clearer signals for AI. A particularly promising approach called Hybrid Collective Intelligence specifically designs for these virtuous cycles, where AI and humans improve each other recursively.

Achieving this kind of human-AI synergy would be transformative, and, it could be argued, even necessary for science. It would be a car, not a horseless carriage. But it will also be challenging, requiring us to rethink science communication infrastructure from the ground up.

Rethinking science communication is long overdue, and as Prachee Avasthi recently wrote, we should be treating it as a research problem in and of itself. The good news is that CI can help us there too! The key insight is that we can view science, and in particular science communication, as a CI system, albeit one using very outdated technology and practices. Perhaps we can apply insights from collective intelligence theory to improve the practice of science itself? This question has received surprisingly limited attention to date, but one notable work discussing it is the excellent Science Communication as a Collective Intelligence Endeavor. The piece identifies several key features that distinguish collective intelligence systems and explores how these could transform science communication. Four features are particularly relevant to our discussion:

1. Structured aggregation of distributed knowledge, data and discourse: enabling synthesis at scales impossible for individual researchers.

2. Increased participation by both researchers and the broader public: democratizing contribution while maintaining epistemic standards.

3. Increased diversity of contributions and participants: bringing multiple perspectives to bear on complex problems.

4. Increased responsiveness through rapid feedback cycles and the ability to respond to updates and new information in real-time.

In the time since the paper was published, new publishing tools and recent AI developments are combining to drive renewed interest in this problem space. Three metascience trends are now converging to open a unique window of opportunity for translating CI theory into practice.

Already in a 1963 talk, Nobel laureate Peter Medawar argued that "the scientific paper is a fraud”, since "it misrepresents the processes of thought that […] gave rise to the work that is described in the paper." Modular research is an exciting trend in science towards publishing those processes of thought more faithfully. New tools like Nanopublications, Discourse Graphs and our Semble enable publishing precise and machine readable representations of atomic knowledge units such as hypotheses, observations, questions, reviews or even reading recommendations. Unbundling research into these connected and remixable units unlocks larger scale distributed research, similar to how the structured and modular nature of code facilitates the coordination of large scale open source software projects.

CI dimensions supported: structured aggregation, increased diversity and participation (1,2,3)

The second circle is AI. Medawar’s insight has aged well - more than 60 years later, in the age of AI, we’re realizing that we can’t train AI models to think properly if we don’t have faithful representations of our actual thought processes. As Astera co-founder Seemay Chou recently wrote - “Automated AI agents can’t learn scientific reasoning based on a journal record that presents only the glossy parts of the intellectual process, and not actual human reasoning. We are shooting ourselves in the foot.” The success of AI coding assistants is a hugely impressive demonstration of the potential at the intersection of AI and modular research - what is code if not modular, structured reasoning? Training on large scale code data has unlocked powerful new forms of AI reasoning. Now “all” we need is Github, not just for code, but for modular, structured knowledge.

CI dimensions supported: scalable knowledge aggregation, increased responsiveness (1,4)

So how do we go beyond the glossy thinking of papers to find actual human reasoning in the wild? How do we get people to write less papers and author more nanopublications? This brings us to the third and final circle, social media. Science social media is one of the most interesting new frontiers of scientific discourse. Michael Nielsen wrote about it in Reinventing discovery: the new era of networked science: “online tools are transforming the way scientists make discoveries. These tools are cognitive tools, actively amplifying our collective intelligence, making us smarter and so better able to solve the toughest scientific problems”. Science Twitter was indispensable to me during my PhD, especially since everything was online due to COVID. It was where I could get in the heads of experts in a way that would have been impossible just by reading their papers. Yes, science social media is clouded by hype and distractions and self promotion. But beyond those clouds, new forms of research are taking shape. As we’ve previously written, social media provides a key piece of the incentive puzzle for broader adoption of modular publishing practices: researchers are literally already nanopublishing there already, they just don’t know it yet. While social networks still lack the structure provided by modular research tools and the scalable aggregation capacities provided by AI, new open social protocols like ATProto are widely used by researchers and offer an exciting space for experimenting with new kinds of science social media (drop in and say hi here!).

CI dimensions supported: increased participation, responsiveness and diversity (2,3,4)

Note: These mappings are approximate and are meant to highlight each trend's primary contributions to CI. Also, each trend supports multiple CI dimensions, often in complementary ways.

The synergy between these circles is why we think the stars are aligning for work on collective intelligence for science. While each trend has primary strengths—modular research for structured knowledge, AI for scalability, social media for participation—their real power lies in combination. This, we propose, is the car. Imagine social networks that incentivize and elicit authentic knowledge sharing, tools that structure this thinking into modular units, and AI systems powered by that data to support next generation discovery and synthesis. A cooperative human-AI Github for knowledge.

Instead of the AI horseless carriage where science drowns in AI slop papers, let’s build systems that keep humans firmly in the loop as essential participants.

And instead of the centrifugal force that pushes humans out of the loop, let’s create centripetal forces that draw humans together, using modular research, AI, and social networks to enable new forms of collaboration and discovery.

These early experiments swapped engines instead of horses, rather than rethinking vehicle design from the ground up. The AI horseless carriage story is at its core a warning: when a revolutionary new technology like AI is invented, the first tools built with it risk failure because they mimic the old way of doing things.

What are the “horse-carriage” risks in AI for science? And what is the “car” that we could be designing for instead?

I would argue that the AI horseless carriage is plugging AI into the existing publishing system. The result, taken to its somewhat absurd but not unlikely endpoint, looks like a loop of AI peer review on AI generated papers. I call this scenario a horseless carriage since papers and peer review in their current form are old ways of doing science that we already know are obsolete.

To be clear, AI automation is highly valuable for well-scoped tasks such as screening literature, extracting data, or running standardized protocols. The risk isn't automation itself, but automating broken systems. Such wholesale automation risks devolving science into a meaningless “doom loop”, with AI acceleration creating a kind of centrifugal force that deskills humans and pushes them out of the research loop (see figure below). We're seeing similar doom loops emerge in education, where AI-generated assignments meet AI grading systems, progressively removing students from meaningful learning while appearing to increase efficiency. This kind of automation is pernicious since in many respects it is the path of least resistance: it requires minimal institutional change and aligns with big AI companies' push to sell automation at scale. It also makes certain myopic economic sense, as cuts to science funding render human labor more scarce and incentivize institutions to look to automation as an imperfect but cheaper substitute.

But there is a promising alternative path, and what I would argue is the "car" we should be building: collective intelligence (CI) for science. In its broadest form, CI can be seen as “a collaborative approach to problem-solving, typically supported by technological tools, which allows for real-time co-ordination and mobilization of knowledge that is distributed among many individuals.”

The key difference between these paths lies in how AI relates to humans. In the horseless carriage scenario, AI automates tasks, deskilling researchers and pushing them to the periphery. In collective intelligence systems, AI instead amplifies and connects human thinking: facilitating collaboration, aggregating distributed knowledge, and helping groups coordinate at scales previously impossible. The focus shifts from automation to augmentation, and from substitution to synergy. In the horseless carriage scenario, each advance in AI capability tends to further marginalize human participation. In CI systems, improvements benefit both: better AI tools enable richer human collaboration, while more human participation generates better training data and clearer signals for AI. A particularly promising approach called Hybrid Collective Intelligence specifically designs for these virtuous cycles, where AI and humans improve each other recursively.

Achieving this kind of human-AI synergy would be transformative, and, it could be argued, even necessary for science. It would be a car, not a horseless carriage. But it will also be challenging, requiring us to rethink science communication infrastructure from the ground up.

Rethinking science communication is long overdue, and as Prachee Avasthi recently wrote, we should be treating it as a research problem in and of itself. The good news is that CI can help us there too! The key insight is that we can view science, and in particular science communication, as a CI system, albeit one using very outdated technology and practices. Perhaps we can apply insights from collective intelligence theory to improve the practice of science itself? This question has received surprisingly limited attention to date, but one notable work discussing it is the excellent Science Communication as a Collective Intelligence Endeavor. The piece identifies several key features that distinguish collective intelligence systems and explores how these could transform science communication. Four features are particularly relevant to our discussion:

1. Structured aggregation of distributed knowledge, data and discourse: enabling synthesis at scales impossible for individual researchers.

2. Increased participation by both researchers and the broader public: democratizing contribution while maintaining epistemic standards.

3. Increased diversity of contributions and participants: bringing multiple perspectives to bear on complex problems.

4. Increased responsiveness through rapid feedback cycles and the ability to respond to updates and new information in real-time.

In the time since the paper was published, new publishing tools and recent AI developments are combining to drive renewed interest in this problem space. Three metascience trends are now converging to open a unique window of opportunity for translating CI theory into practice.

Already in a 1963 talk, Nobel laureate Peter Medawar argued that "the scientific paper is a fraud”, since "it misrepresents the processes of thought that […] gave rise to the work that is described in the paper." Modular research is an exciting trend in science towards publishing those processes of thought more faithfully. New tools like Nanopublications, Discourse Graphs and our Semble enable publishing precise and machine readable representations of atomic knowledge units such as hypotheses, observations, questions, reviews or even reading recommendations. Unbundling research into these connected and remixable units unlocks larger scale distributed research, similar to how the structured and modular nature of code facilitates the coordination of large scale open source software projects.

CI dimensions supported: structured aggregation, increased diversity and participation (1,2,3)

The second circle is AI. Medawar’s insight has aged well - more than 60 years later, in the age of AI, we’re realizing that we can’t train AI models to think properly if we don’t have faithful representations of our actual thought processes. As Astera co-founder Seemay Chou recently wrote - “Automated AI agents can’t learn scientific reasoning based on a journal record that presents only the glossy parts of the intellectual process, and not actual human reasoning. We are shooting ourselves in the foot.” The success of AI coding assistants is a hugely impressive demonstration of the potential at the intersection of AI and modular research - what is code if not modular, structured reasoning? Training on large scale code data has unlocked powerful new forms of AI reasoning. Now “all” we need is Github, not just for code, but for modular, structured knowledge.

CI dimensions supported: scalable knowledge aggregation, increased responsiveness (1,4)

So how do we go beyond the glossy thinking of papers to find actual human reasoning in the wild? How do we get people to write less papers and author more nanopublications? This brings us to the third and final circle, social media. Science social media is one of the most interesting new frontiers of scientific discourse. Michael Nielsen wrote about it in Reinventing discovery: the new era of networked science: “online tools are transforming the way scientists make discoveries. These tools are cognitive tools, actively amplifying our collective intelligence, making us smarter and so better able to solve the toughest scientific problems”. Science Twitter was indispensable to me during my PhD, especially since everything was online due to COVID. It was where I could get in the heads of experts in a way that would have been impossible just by reading their papers. Yes, science social media is clouded by hype and distractions and self promotion. But beyond those clouds, new forms of research are taking shape. As we’ve previously written, social media provides a key piece of the incentive puzzle for broader adoption of modular publishing practices: researchers are literally already nanopublishing there already, they just don’t know it yet. While social networks still lack the structure provided by modular research tools and the scalable aggregation capacities provided by AI, new open social protocols like ATProto are widely used by researchers and offer an exciting space for experimenting with new kinds of science social media (drop in and say hi here!).

CI dimensions supported: increased participation, responsiveness and diversity (2,3,4)

Note: These mappings are approximate and are meant to highlight each trend's primary contributions to CI. Also, each trend supports multiple CI dimensions, often in complementary ways.

The synergy between these circles is why we think the stars are aligning for work on collective intelligence for science. While each trend has primary strengths—modular research for structured knowledge, AI for scalability, social media for participation—their real power lies in combination. This, we propose, is the car. Imagine social networks that incentivize and elicit authentic knowledge sharing, tools that structure this thinking into modular units, and AI systems powered by that data to support next generation discovery and synthesis. A cooperative human-AI Github for knowledge.

Instead of the AI horseless carriage where science drowns in AI slop papers, let’s build systems that keep humans firmly in the loop as essential participants.

And instead of the centrifugal force that pushes humans out of the loop, let’s create centripetal forces that draw humans together, using modular research, AI, and social networks to enable new forms of collaboration and discovery.