Insights

Economics and outlook

• Guide

  Developing an AI strategy and the capabilities to deliver it

  Choose initiatives, develop capabilities and assign delivery responsibility through staged commitments.

• Paper

  The economics of AI-assisted work

  Compare complete costs, acceptable outcomes, residual risk and organizational learning across alternatives.

• Article

  Measuring the costs and benefits of AI-assisted work

  Work through whole-workflow accounting, distinguishing saved time, usable capacity and cash savings.

Models and decisions

• Paper

  Customer analytics: prediction, experimentation and business decisions

  Connect acquisition, retention and growth to prediction, experimentation and implementable decisions.

• Paper

  Understanding model decisions and the limits of explanations

  Choose explanations that answer a business question and understand the limits of attribution and causal interpretation.

• Resource

  Designing and evaluating customer interventions

  Design and evaluate customer interventions, including constrained allocation and an operating worksheet.

• Article

  When to use generative AI, a task-specific model, or ordinary code

  Compare generative AI, task-specific models and ordinary code on the same completion and cost basis.

• Article

  What AI decision models do, and when to use them

  Understand computational judgments, confidence and decision policies without mistaking them for statistical independence.

System design and operation

• Paper

  Implementing AI in business workflows

  Translate written procedures, completed work and practitioner judgment into requirements, permissions and tested behavior.

• Paper

  Building AI systems for business documents and knowledge

  Build document and knowledge workflows with source authority, extraction, retrieval and evidence for review.

• Guide

  Operating AI systems and deciding when to reevaluate them

  Monitor accepted work, respond to changes and incidents, and decide what evidence a new operating decision needs.

• Resource

  Designing document retrieval and access for AI systems

  Choose retrieval methods and test source currency, access, revocation and answer adequacy through a complete example.

• Article

  Redesigning workflows for AI: tasks, exceptions and responsibility

  Redesign tasks, exceptions and responsibility together, with explicit acceptance of the completed work.

• Article

  Using coding agents: from a written task to a tested change

  Follow a constructed coding task from written requirements through implementation, independent tests and continuation.

Evaluation and reliability

• Paper

  Evaluating AI-assisted systems

  Define outcomes and reference judgments, validate evaluators, and interpret samples and uncertainty.

• Resource

  Decision records and responsibilities in AI workflows

  Connect a consequential decision to its evidence, authorized actions, accountable owner and unresolved work.

• Article

  Evaluating an AI-assisted workflow

  Evaluate a complete workflow rather than accepting a successful-looking output as the business result.

• Article

  What LLM and agent benchmark scores actually tell you

  Interpret benchmark populations, scoring and uncertainty before transferring a ranking into a deployment choice.

Software

Open source tools for coding workflows, document processing and token measurement.

• Coding Agent Toolkit →
• PDFsmith →
• PDF-Bench →
• mdtoken →