TL;DR

The corporate use of artificial intelligence has become a commodity, offering significant value at a falling cost. But behind this apparent affordability lies an invisible and growing cost: the leakage of proprietary knowledge that occurs every time a company uses an AI model.

You are not buying artificial intelligence. You are selling your company to it, one prompt at a time. Every correction you make, every piece of context you add so the model understands your business, every time you refine a response because it came close but failed to capture how your company truly thinks — none of that disappears once the task is done.

That is knowledge, and knowledge has an owner. The uncomfortable question is: once it has passed through the model, is it still yours?

Arrow’s Paradox

In 1962, economist Kenneth Arrow described a paradox that anyone who has ever tried to sell consulting or a course knows firsthand: the value of a piece of information can only be assessed after it has been revealed, but by then it has already been given away for free. That is why patents exist — they allow an idea to be disclosed without simply giving it away. The seller of knowledge has always carried this risk.

The Reverse Information Paradox

Artificial intelligence has inverted this equation. Satya Nadella recently described what he called the reverse information paradox: now it is the buyer who risks leaking knowledge, and does so precisely in order to use what they bought. You pay for intelligence twice — once in money, and again in something more valuable: the proprietary knowledge you must reveal for the model to actually perform well.

The better you want it to perform, the more of your company you have to hand over.

Over time, this asymmetry only grows in one direction. The vendor learns more and more about you, while you learn very little about what it is learning in return.

What happens in practice?

This incentive structure produces predictable behavior, and this is where the discussion becomes more interesting than a simple complaint about data privacy. If a company suspects that feeding the model its most valuable knowledge means losing it irreversibly, it won’t stop using AI — it will withhold exactly what has the most value and feed the model only what’s left over: generic tasks, low-risk data, the operational waste that doesn’t compromise anything strategic.

This is exactly the logic George Akerlof described in 1970 in the classic “The Market for Lemons”: when the buyer cannot verify the quality of what’s on the other side of the table, high-quality sellers exit the market because they cannot signal differentiation, and only the poor-quality product remains, circulating at an average price. Adverse selection doesn’t eliminate the transaction — it impoverishes whatever continues to be transacted.

The Market for Exhaustion

Applied to the relationship between companies and AI vendors, this mechanism creates the market for exhaustion. The companies with the most valuable proprietary knowledge — the processes, the judgment calls, the corrections that exist only because someone inside truly understands the business — are precisely the ones with the most to lose, and therefore the most reason to withhold. What flows freely into the models is, proportionally, the least differentiated knowledge in the market.

The outcome benefits no one: the company underuses the tool out of fear, and the provider learns less and less of real value, even as it accumulates more and more data.

This dynamic is no longer theoretical. Standard clauses in enterprise software contracts have been quietly granting broad rights for AI vendors to train models on customers’ source code, financial records, and legal documents. The market has already begun to react: GitHub started using Copilot users’ interaction data to train models by default starting this past April, requiring an active opt-out, while GitLab went in the opposite direction, stating that it does not train on customer code on any plan and contractually prohibiting the vendor from using customer inputs or outputs for its own purposes.

The friction isn’t limited to small vendors and unwary clients — Microsoft itself has publicly pressured Anthropic over its data retention terms, showing that corporate governance still hasn’t caught up with the pace of AI providers’ data policies.

The market is already responding with product moves, adopting private models — even at a higher price — to reduce exposure of training knowledge. Companies like SAP are building their own proprietary models to keep in-house the domain knowledge that would otherwise have become free input for third parties.

What’s still missing is a mechanism as mature as the patent was for Arrow’s paradox, or as warranties and certification were for Akerlof’s paradox. Nadella proposes five pillars for this:

  • control over one’s own evaluations and over the institutional memory generated through use
  • the ability to train and fine-tune models within one’s own data boundary
  • the freedom to switch models without losing accumulated intelligence
  • cost efficiency from orchestration decoupled from a single vendor
  • and finally, the composition of all of this into a continuous learning cycle that belongs to the company, not the vendor.

This leaves a simple question for any company that has already put AI at the center of its operations: if the model you use today disappeared tomorrow, would your company still know what it learned from it?

About the author: Cairo Cananéa is a specialist in data science and AI for marketing.