Theory 01
Principal-Agent Theory in AI
Origin: Dowding & Taylor (2024); Eisenhardt (1989) in Session 09 slides
Dowding & Taylor (2024) apply the Principal-Agent (P-A) framework to algorithmic decision-making (ADM). The key point is that AI is not only a technical tool. Once an organization lets an algorithm recommend, rank, screen, price, diagnose, or decide, it has delegated decision authority from a human principal to an algorithmic agent.
In ordinary agency theory, the principal hires an agent because the agent has specialized capacity. That same specialization creates a governance problem: if the principal could perfectly observe and evaluate everything the agent does, delegation would be unnecessary. With machine learning, this problem becomes sharper because the agent can process more data than humans, but the reasons for a specific output may be hard or impossible to inspect directly.
Key Agency Problems (Dowding & Taylor, 2024)
- Information asymmetry: The algorithmic agent possesses complex decision logic that the principal cannot easily verify.
- Goal misalignment: The system may optimize a measurable proxy, such as prediction accuracy, engagement, or cost reduction, while the principal cares about broader strategic, ethical, or scientific objectives.
- Agency costs: The costs of monitoring, auditing, validating, benchmarking, and accepting residual loss from imperfect delegation.
- Residual loss: Even after governance, some gap remains between the algorithm's behavior and the principal's ideal outcome.
Theory 02
Agency Problems: Adverse Selection, Moral Hazard, and Contracts
Origin: Session 09 slides; Eisenhardt (1989); Dowding & Taylor (2024)
The session slides connect ADM to classic agency theory. The useful exam move is to distinguish when the information problem occurs: before choosing the agent, after delegating to the agent, or during evaluation of outcomes.
| Concept | Classic Agency Theory | Algorithmic Decision-Making |
|---|---|---|
| Adverse selection | The principal cannot know whether the agent is a "peach" or a "lemon" before contracting. | A firm may not know whether a model, vendor, dataset, or benchmark is genuinely suitable before adoption. |
| Moral hazard | The agent changes behavior after contracting because monitoring is incomplete. | A vendor may underinvest in maintenance, data quality, documentation, or safety once the client is locked in. |
| Agency cost | The cost of reducing information asymmetry and limiting opportunism. | Audits, validation studies, red-teaming, explainability work, human review, legal review, and compliance reporting. |
| Contract specificity | The degree to which success can be measured unambiguously. | High for simple classification accuracy; lower for strategic decisions, fairness, safety, scientific novelty, or patient welfare. |
Contract Design: Behavior-Based vs. Outcome-Based
Behavior-based contracts specify processes: documentation, model cards, audit rights, data governance, human review, and validation routines. They are useful when outcomes are uncertain or hard to measure.
Outcome-based contracts specify results: accuracy, uptime, bias thresholds, false positive rates, cost savings, or validated predictions. They are useful when the output can be measured clearly and the supplier has control over the result.
The slides list factors influencing this choice: information systems, outcome uncertainty, risk aversion, goal conflict, task programmability, outcome measurability, and relationship length.
Signaling
Agents try to show quality before selection: certifications, audit reports, benchmark performance, transparent documentation, or credible references.
Screening
Principals actively test alternatives: pilot projects, vendor due diligence, challenge datasets, sandbox trials, and independent model evaluation.
Human Involvement
Dowding & Taylor distinguish humans in the loop, on the loop, and out of the loop. The higher the stakes, the stronger the case for human oversight.
Theory 03
Trust in Algorithmic Systems
Origin: Dowding & Taylor (2024); Pavlou & Gefen (2004) via article discussion
Trust is a prerequisite for delegation, but Dowding & Taylor's central argument is that algorithmic trust should not rest only on a direct relationship between one user and one model. Since most users cannot inspect deep learning systems themselves, trust must be supported by institutions that make algorithmic choice meaningful.
Institution-Based Trust
Trusting the system because it is embedded in rules, audits, standards, liability regimes, disclosure requirements, and third-party evaluation.
Explainability (XAI)
The technical ability to make model behavior understandable. It helps, but Dowding & Taylor warn that explainability alone cannot solve Type 3 opacity.
Relational Trust
Trust built over time through consistent, reliable performance of the algorithm in practice.
Why Institutions Matter
Individual users face high monitoring costs. Institutions reduce those costs by producing reusable information: standards, model cards, external audits, public benchmarks, professional norms, regulatory classifications, and reputational signals. This lets users choose among algorithmic alternatives without becoming machine-learning experts themselves.
Theory 04
The Opacity Problem
Origin: Burrell (2016) in Dowding & Taylor (2024); Session 09 slides
Algorithmic opacity is not a single problem. The slides distinguish three types, and each requires a different response. This matters because managers often treat opacity as if it can always be solved by opening the code or adding an explanation layer.
| Type of Opacity | Meaning | Governance Response |
|---|---|---|
| Type 1: Secrecy | Firms hide models, data, or processes for commercial or legal reasons. | Audit rights, disclosure duties, procurement requirements, and trusted third-party access. |
| Type 2: Technical illiteracy | Information is available, but users lack the expertise to understand it. | Documentation, training, model cards, user-centered explanations, and professional intermediaries. |
| Type 3: Intrinsic complexity | The model is so complex that even designers cannot straightforwardly explain each internal step. | Testing, benchmarking, monitoring outcomes, robust validation, and institutional trust rather than full direct inspection. |
Limits of Technical Fixes
Explainability can improve transparency, but it can also create false reassurance, simplify away the real model, or reduce predictive performance. In high-stakes decisions, the exam point is not "always use XAI"; it is to ask whether the decision context requires an interpretable model, a black-box model with strong external validation, or no automation at all.
Theory 05
Algorithmic Internalities
Origin: Dowding & Taylor (2024)
Dowding & Taylor (2024) introduce algorithmic internalities: harms caused by algorithms that are borne by the users or principals who chose to rely on them. This contrasts with algorithmic externalities, where harms fall on third parties who did not choose the system.
Internalities vs. Externalities
Many ethical debates focus on externalities, such as biased policing or discrimination against applicants. Dowding & Taylor add that the firm also needs to worry about internalities:
- Loss of Human Capability: De-skilling of workers who become over-reliant on AI.
- Strategic Narrowing: The algorithm might optimize for past patterns, missing radical future innovations.
- Financial Loss: Bad predictions leading to failed R&D or poor resource allocation.
- Preference Misrepresentation: Recommender systems may infer short-term revealed preferences while ignoring higher-order preferences.
Capability Approach
Dowding & Taylor use Sen's capability approach to ask whether ADM expands substantive human freedom. Algorithms can reduce decision costs, help users navigate too many choices, and support better decisions. But they can also reduce autonomy if users lose the ability to evaluate, contest, or override algorithmic choices.
Exam Framework
How to Evaluate and Choose Among Algorithmic Alternatives
Origin: Session 09 learning objective; Dowding & Taylor (2024)
The session's learning objective is practical: how should managers evaluate algorithmic alternatives? A strong answer combines agency theory, opacity, and institution-based trust.
| Question | What to Look For |
|---|---|
| What decision is delegated? | Recommendation, ranking, prediction, diagnosis, resource allocation, or fully autonomous action. |
| How high are the stakes? | Low-stakes personalization can tolerate more automation; healthcare, hiring, finance, and legal decisions need stronger governance. |
| What is the human role? | Human in the loop, on the loop, or out of the loop. Oversight should match risk and task complexity. |
| How measurable is success? | If outcomes are measurable, use outcome contracts and benchmarks. If not, emphasize behavior contracts, audits, and process controls. |
| Which institutions support trust? | Regulation, audit ecosystems, standards, model cards, third-party evaluations, and disclosure rules such as AI Act or DSA-style transparency obligations. |
Case Application
Novo Nordisk — Governing the Agent
Applying Session 09 logic to the OpenAI partnership:
- Agency Costs at Novo: The cost isn't just the OpenAI subscription. It's the cost of Novo's scientists verifying every lead the AI generates. If they don't verify, the Agency Risk (bad drug leads) is too high.
- Institutional Trust: Novo must rely on Institutional Trust. They need clear governance frameworks for how clinical data is used and how AI results are validated against biological ground truth.
- Black Box Risk: In pharmaceutical research, a "Black Box" prediction isn't enough for the FDA. Novo needs Explainability to transform AI-output into scientific knowledge.
- Contract Specificity: A contract can specify data security, audit access, uptime, and documentation more easily than it can specify "scientific discovery." That pushes Novo toward behavior-based governance plus milestone-based validation.
- Human-in-the-Loop Design: Drug discovery is high stakes, uncertain, and scientifically complex. Algorithms can prioritize hypotheses, but expert scientists must remain in or on the loop for validation.
Exam Preparation
Likely Oral Exam Questions
-
Core Why is the Principal-Agent framework useful for studying AI in organizations? ▶
- It highlights that AI is not just a tool, but a delegation of authority.
- It focuses attention on information asymmetry: the firm may observe outputs without understanding the full decision process.
- It helps identify agency costs: monitoring, auditing, validation, contracting, and residual loss.
-
Apply What is "Institution-Based Trust" and why is it relevant for Novo Nordisk? ▶
- It's trust based on the "rules of the game" (regulations, external audits).
- Relevant because Novo cannot monitor OpenAI's trillions of parameters directly. They must trust that the institution of the partnership (contracts, data privacy laws) will protect them.
- A stronger answer adds third-party audits, model documentation, validation protocols, and regulatory compliance as trust-producing institutions.
-
Synthesis Can a firm ever fully eliminate Agency Costs when using AI? ▶
- No. There is always a residual loss.
- You can increase monitoring (which costs money) or accept more risk (which can cost money).
- The goal is to find the optimal point where the marginal cost of monitoring equals the marginal benefit of reduced error.
-
Synthesis How would you evaluate competing AI tools for a high-stakes organizational decision? ▶
- Define the delegated decision and the human role: in, on, or out of the loop.
- Assess adverse selection risk through screening: pilots, benchmark tests, documentation, and vendor due diligence.
- Choose governance based on measurability: outcome contracts for measurable tasks, behavior contracts for uncertain tasks.
- Use institution-based trust: audits, model cards, standards, regulatory compliance, and independent validation.