Trusted Transparency

This is part of our regular series on new directions in software engineering.

Many optimization problems can be cast as a particle randomly exploring a state space. You can think of states as particular solutions, and a step corresponds to going to a neighboring solution. A general heuristic for optimization, called “go with the winners”, runs multiple particles in parallel, and every once in a while classifies the particles as “promising” or “not promising.” Each particle that is “not promising” is then moved to a particle that is “promising.”

Quite surprisingly, by coupling independent explorations at right times, “go with the winners” algorithms can outperform multiple independent random explorations, often by an exponential amount!

The humble particles in go-with-the-winners provide a toy world for software teams. Suppose you are a team trying to find an optimal product-market fit. You and your coworkers simultaneously explore different ideas, you align your goals, and you keep building. Each of you is a particle, your team coordination marks some directions as more promising, and when you align your goals, you move resources to “promising” directions. Each member observes the status of all others, and coordination determines where to go next.

All analogies are wrong, some analogies are useful. The world of randomly moving particles throws away many nuances of software development in the real world but helps us identify a core, and important, coordination principle: trusted transparency. Let me walk you through it.

Trusted transparency is the principle that the “state” of each team member is available and queryable by all team members. The state is more than the present work artifacts—code, docs, transcripts—it is also the history of how the team member got to its current cognitive context. Shared state allows quick alignment of resources, from not-promising directions to promising ones.

Transparency is not a new idea: open source projects have done it for decades. Bell Labs ran on hallway transparency, with open-door policies and shared research context as its exploration mechanism. What is new is that AI agents collapse the cost equation that made “record everything” untenable historically: the marginal cost of recording dropped to near zero, and the marginal cost of retrieving and synthesizing insights dropped to near zero as well. (It also helps that we have gotten more comfortable with recorded online meetings!) The cost of not recording is higher than ever, because each team member generates 10-50x more artifacts per unit of time than before. Simultaneously, an agent can ingest hundreds of hours of context in seconds. In contrast, pre-AI software engineering mechanisms of coordinating context through human-driven summarization (standups, sprints, docs, JIRA tickets) have an increased marginal cost, because human cognitive resources did not scale correspondingly.

Altered Mechanisms

A few mechanisms compound when you take the transparency principle seriously.

The exploration ledger becomes the team's primary artifact. Not the product — the log of exploration. It's where hypotheses, experiments, partials, failures, and surprises accumulate. Append-only, tagged by area and epistemic status (exploratory / result / decided), queryable on demand. When someone starts a new agent session, the first thing the agent ingests is “what's already been tried in this area.” Learning compounds instead of evaporating.

Distillations of the ledger become the durable IP. When agents implement, the intent captured in the exploration ledger outlives the implementation. We care less about the line-by-line code details or current specs. We care more about the decision process: how were the product decisions made? What intent informed the agents? Code becomes regeneratable; distillations of the ledger are the team's accumulated capital.

Manage coordination while artifact volume explodes. Amazon is famous for its two-pizza teams: small groups of closely-coordinating developers who maintain coordination. As agents become first-class team members, the pizza is metaphorically sliced among many more team members. This leads to scaling problems: as time for development goes to zero, the cognitive gap in the team goes to infinity. I do not have a quick answer to this scalability problem: we have to build new software development mechanisms that model attention as a resource.

Trust

So far, we focused on the transparency part of trusted transparency. Historically, transparency has an unfortunate tendency to become surveillance.

For transparency to compound, trust must be absolute. It is extremely difficult for people to forego their egos, to show their false starts and rambling detours. Ways of ensuring trust and their organizational operationalization are key. Trust can be earned through long-term collaboration, but also included as an explicit organizational mechanism. An example is the “no-blame” approach to correction-of-errors. Another is the charitable interpretation doctrine: when interpreting an idea, focus on the ways it may work rather than the ways it won’t. Remember that in exploratory mode, novelty is what you optimize. An interesting failure is a more valuable indicator for novelty than a mundane success.

Trust is necessary. If the ledger becomes a weapon—for penalizing “bad paths”, for settling old scores—people withdraw rather than expose themselves. Transparency becomes performative: people stop sharing half-baked ideas, maintain private scratch spaces and public sessions. The context becomes hidden.

Trust is necessary but not sufficient: we still have to ensure the flow of independent context. It is easy for context to converge prematurely, as desire for group coherence dominates independent exploration. More on this in a future post.

Bottom Line

The principle:

default to public, structured, queryable, distillable artifacts, paired with operational trust commitments.

Transparency makes context visible and shareable across the team. Trust commitments separate transparency from surveillance—they have to be enforced, not just stated.

The bet I'd make: the small exploratory teams that find product-market fit in the next few years will look unusually transparent and unusually organized. This is by no means obvious. Full transparency and ledger-as-source-of-truth is a large cognitive shift from the way we work in teams. There is a core technical challenge is to distill the ledger to move knowledge across the team. But the rewards that compound from the trusted transparency principle are likely to outweigh the uncomfortable downsides of rethinking software development practice.