Back to Writing

The Comet Problem: We're All Building Search Engines Right Now

In Halt and Catch Fire, the Comet team builds something real and good while the ground shifts under them. That's happening to a lot of product teams right now. And it's the most exciting thing I've seen since the early internet.

This is part 2 of a three-part series on how AI is reshaping product management. Part 1 covers why agile needs to evolve, and Part 3 covers what teams will look like on the other side.

The Scene That Stuck With Me

I don’t watch a lot of TV. So when a show lands, I think about it for a long time.

There’s a stretch in the final season of Halt and Catch Fire that I haven’t been able to shake. Joe and Gordon are building Comet, a human-curated web directory. They’re indexing the internet by hand. Every day, new sites come in and the team categorizes them, organizes them, makes them findable. The work is real. The product is good. Users like it. The team is grinding.

And then they see a beta of Netscape Navigator with Yahoo! baked in as the default portal. Search is winning. The curated directory is dead. Comet’s entire strategy collapses. Not because the product was bad. Not because the team wasn’t talented. The ground shifted under them while they were busy doing good work.

That’s the part that sticks with me. They weren’t lazy. They weren’t making mistakes. They were heads-down, shipping, iterating, making progress every single day. They loved the work. And the paradigm changed anyway.

This Time Is Different

I’ve been in tech since the early 2000s. I’ve lived through a lot of shifts. Static web to dynamic web. Monoliths to microservices. Waterfall to agile. And every one of those transitions was gradual. You had years to see it coming. You could retool slowly. The industry moved, but it moved at a pace where you could keep up by paying attention.

This shift is not gradual.

AI is compressing what used to take a team of engineers weeks into what one person can do in an afternoon. And the second-order effects are going to hit fast. Price pressure, because when building is cheap, someone will undercut you. Hyper-custom workflows, because when software is cheap to build, customers will expect solutions tailored to their specific needs rather than settling for generic tools. The companies that adopt this first won’t just have a competitive advantage. They’ll make everyone else look like dinosaurs.

That’s a different speed of change than anything I’ve experienced in twenty years.

You Might Be in the Path Right Now

I keep thinking about that scene because I see the same pattern playing out across the industry right now. Teams are heads-down. Roadmaps are full. Sprints are planned. Everyone’s busy. The work feels productive because it is productive, in the context of today’s assumptions.

But the assumptions are shifting.

Here’s a test I keep coming back to: when a customer comes to you with an atypical request, something outside your standard workflow, can you actually satisfy it? Because with software getting cheaper to build every month, someone can. Maybe not a big competitor. Maybe a solo builder with AI who can spin up a custom solution in a weekend. The bar for “good enough” is dropping fast, and the bar for “exactly what I need” is becoming reachable for the first time.

If your product only works for the typical use case, and you can’t flex for the edge cases, you’re vulnerable. Not because your product is bad. Because the economics of building custom solutions just changed.

Think about where this lands:

Workflow optimization products. If your product exists to manage a complex manual process, and AI can collapse that entire process into a single step, you’re not being disrupted by a better workflow tool. You’re being disrupted by the workflow disappearing.

Aggregation and curation products. If humans are organizing, tagging, summarizing, or categorizing information, and an AI can do it instantly, the hand-built index is dead. It doesn’t matter how good your taxonomy is.

Middleware and integration products. If your product exists because two systems don’t talk to each other and you sit in between, AI agents that can navigate both systems directly threaten your entire value proposition.

Process management tools. This connects directly to Part 1. If your product helps teams manage sprints, track velocity, or coordinate handoffs, and AI collapses the team size and build cycle, you’re managing a process that’s shrinking.

None of these are bad products. The teams building them aren’t doing anything wrong. That’s what makes it so hard to see.

Why It’s Hard to See the Wave

Most teams in the path of a paradigm shift have every reason to believe they’re on the right track. Users are growing. The product is improving. The team is shipping. Every metric says “keep going.”

That’s exactly the trap.

When you’re heads-down in execution mode, you evaluate progress against your current plan. Are we hitting our sprint goals? Are users growing? Is the product getting better? And the answer can be yes to all of those while the entire market shifts around you.

The problem isn’t that teams aren’t paying attention. It’s that the signals of a paradigm shift look different from the signals of normal competition. A competitor launching a similar product is a signal you know how to read. A completely new approach that makes your category irrelevant is much harder to see, because it doesn’t show up on your competitive radar.

Right now, most product teams are evaluating AI as a feature to add or a tool to adopt. “Should we add AI to our product?” is the wrong question. The right question is: “Does AI change whether our product needs to exist?”

That’s a harder question. But it’s the right one.

Building on the Shift vs. Building in Its Path

Not every product is in the path. Some are positioned to benefit from the paradigm shift rather than be consumed by it.

The difference comes down to where your value actually lives.

If your value is in the process (organizing, coordinating, managing complexity), you’re probably in the path of the shift. AI is very good at collapsing process.

If your value is in the outcome (the decision made, the relationship built, the insight generated), you’re more likely building on top of the shift. AI makes outcomes cheaper and faster to reach, which makes your product more valuable, not less.

A product that helps PMs write better user stories is in the path of the shift. AI can write user stories. A product that helps PMs understand what their customers actually need is building on top of the shift. AI makes it easier to synthesize research, but the judgment about what to do with it is still human.

A product that manages the complexity of deploying software is in the path of the shift. AI is simplifying deployment. A product that helps teams decide what to deploy and whether it worked is building on top of the shift. Speed without direction is just chaos.

The question for every PM: is your product reducing complexity that AI will eliminate, or enabling decisions that AI makes more important?

What PMs Should Be Asking Right Now

If any of this resonates, here are the questions I’d be sitting with:

“What assumption does my product depend on that AI might invalidate?” Every product is built on assumptions about how work gets done. If AI changes how that work gets done, your product might still solve a real problem in a way that no longer makes sense.

“Can I satisfy the atypical request?” When a customer asks for something outside your standard workflow, what happens? If the answer is “that’s not on the roadmap,” someone with AI and a weekend might beat you to it. The ability to flex, to customize, to move fast on non-standard needs is becoming a real competitive advantage.

“If I were starting this product today, would I build it the same way?” This is the most honest test. If the answer is no, you need to understand what changed and whether your current product can evolve toward what you’d build now, or whether you’re optimizing a local maximum.

“Am I measuring velocity or value?” Sprint velocity, feature output, story points completed. All of those measure how busy you are. None of them measure whether what you shipped actually mattered to a customer. In a world where building is cheap, the teams that win aren’t the ones shipping the most. They’re the ones shipping the right things.

Everyone Has to Move

This isn’t just a PM problem. It’s everyone.

PMs need to get closer to customers and further from process. The ticket-writing, backlog-grooming, sprint-planning version of the PM role is getting compressed. The customer-insight, strategic-betting, problem-understanding version is getting more valuable. If you’re a PM who can sit with a client, understand their actual problem, and translate that into something buildable, you’re going to thrive. If you’re a PM whose primary skill is managing a sprint board, the ground is shifting under you.

Engineers need to move up the stack. Writing code is becoming less of the job. Understanding systems, making architectural decisions, and working directly with customers on technical problems is becoming more of it. The engineer who can talk to a client is going to be worth more than the engineer who can write the cleanest code.

Designers need to think in systems, not screens. When AI can generate interfaces, the value moves to understanding user behavior, designing for edge cases, and creating coherent experiences across increasingly personalized products.

The common thread: the people who are closest to the customer and the problem are the ones who are best positioned. The people who are closest to the process and the tooling are the ones who need to adapt fastest.

Why I’m Excited

Here’s where I break from the usual “AI is coming for your job” narrative.

I think this is the most exciting time to be in product since the early internet.

I remember what it felt like in the early 2000s when the web was opening up and suddenly you could build things that reached people everywhere. There was this energy of possibility. You didn’t know exactly what would work, but you knew the space of what was buildable had just expanded dramatically.

That’s exactly what’s happening now. The space of what’s buildable just expanded by an order of magnitude. Products that were never economically viable are suddenly possible. Customization that was never affordable is suddenly cheap. The long tail of specific, opinionated, niche solutions is about to explode.

Yes, some existing products and roles will get disrupted. That’s real and it matters. But the opportunity on the other side is enormous. If you’re a PM who genuinely understands your customers, who can make good product bets, who can move at the speed AI enables, you’re about to have the best stretch of your career.

The Comet team’s story is a cautionary tale. But the broader story of that era, the story of the internet itself, is one of massive creation. More was built than was destroyed. More opportunity was created than was lost. The people who recognized the shift and moved toward it instead of away from it did extraordinarily well.

That’s the opportunity right now. Don’t be Comet. But don’t be afraid of the shift either. Run toward it.

Next up: Part 3 looks at what product teams actually look like on the other side of this shift.