Mathematical & Free Energy Foundations
How the Adaptive Complementarity Framework derives allocation from the Free Energy Principle
Under the Free Energy Principle (FEP), any system that maintains a boundary with its environment must, on average, minimize variational free energy — a tractable upper bound on surprise, the negative log-evidence of an observation under a generative model. Surprise itself requires marginalizing over hidden causes and is generally intractable, so systems minimize free energy instead, thereby approximating Bayesian inference while increasing model evidence.
Variational free energy:
\[ F = \mathbb{E}_{q(\mu\mid s)}\!\big[\ln q(\mu \mid s) - \ln p(s,\mu \mid \theta)\big] = D_{\mathrm{KL}}\!\big(q \,\|\, p\big) - \ln p(s \mid \theta) \]Because \(D_{\mathrm{KL}}(q\,\|\,p) \ge 0\), it follows that \(F \ge -\ln p(s \mid \theta)\): free energy upper-bounds surprise, and minimizing \(F\) tightens the bound. In this framework, task allocation is treated as selecting the Human–AI configuration that minimizes task-relevant free energy under context constraints.
A task is summarized by a normalized state vector \(\mathbf{x} = (U, N, W, \tilde{\varpi}, \tilde{\varepsilon}) \in [0,1]^5\). Each dimension operationalizes a distinct aspect of variational free energy, and each has a defined effect on the recommended regime.
| Dimension | Free-energy correlate | Effect on allocation |
|---|---|---|
| Uncertainty \(U\) | High posterior entropy \(H[q(s)]\) — weak beliefs about hidden states | ↑ human involvement |
| Novelty \(N\) | Low model evidence \(\log p(o\mid m)\) — observations violate model assumptions | ↑ human involvement |
| Value conflict \(W\) | Divergent preference models \(p(o\mid C_i)\) — no single model resolves the trade-off | ↑ human involvement |
| Precision \(\tilde{\varpi}\) | Inverse-variance confidence \(\varpi_{\mathrm{raw}} = 1/\sigma^2\) in predictions | ↑ automation |
| Prediction error \(\tilde{\varepsilon}\) | Realized mismatch / surprise proxy \(\varepsilon_{\mathrm{raw}} = o - \mathbb{E}_q[g(s)]\) | ↑ human involvement |
Active inference extends the FEP from passive prediction to action. Candidate policies are scored by their expected free energy, which decomposes into complementary drives — anticipated information gain (epistemic value) and anticipated deviation from preferred outcomes (pragmatic value):
\[ G(\pi) = \underbrace{-\,\mathbb{E}_{q(o\mid\pi)}\big[D_{\mathrm{KL}}\!\big(q(s\mid o)\,\|\,q(s)\big)\big]}_{\text{epistemic}} + \underbrace{D_{\mathrm{KL}}\!\big(q(o\mid\pi)\,\|\,p(o)\big)}_{\text{pragmatic}} \]Policies are then selected by a precision-weighted softmax, where \(\gamma\) controls the exploration–exploitation trade-off:
\[ P(\pi) \propto \exp\!\big(-\gamma\, G(\pi)\big) \]Tasks with high epistemic value — model revision, hypothesis generation, value synthesis — favor human judgment; tasks amenable to pragmatic optimization in well-specified domains favor algorithmic execution.
Evaluating \(G(\pi)\) over full policy horizons is intractable in organizational settings. The framework therefore introduces a tractable surrogate defined directly on the state vector, preserving the qualitative sign structure of the epistemic/pragmatic decomposition:
\[ \mathcal{G}(\mathbf{x};\boldsymbol{\theta}) = \beta_0 + \beta_U U + \beta_N N + \beta_W W - \beta_{\tilde{\varpi}}\,\tilde{\varpi} + \beta_{\tilde{\varepsilon}}\,\tilde{\varepsilon} + \sum_{i \lt j} \gamma_{ij}\, x_i x_j \]The interaction terms \(\gamma_{ij}\,x_i x_j\) capture non-additive effects (e.g. uncertainty–novelty amplification, precision–error compounding). The sign constraints follow directly from the EFE interpretation:
\[ \beta_U,\ \beta_N,\ \beta_W,\ \beta_{\tilde{\varepsilon}} \ge 0, \qquad \beta_{\tilde{\varpi}} \ge 0 \]A monotone link maps the surrogate to a bounded score, which a discrete selector partitions into operational regimes:
\[ r_t = \sigma\!\big(\mathcal{G}(\mathbf{x}_t;\boldsymbol{\theta})\big) \in (0,1), \qquad a_t = f(r_t; b_1, b_2) \in \{\text{AI}, \text{Hybrid}, \text{Human}\} \]The composed allocation rule is \(\phi(\mathbf{x}_t) = f\big(r(\mathbf{x}_t;\boldsymbol{\theta}); b_1, b_2\big)\), with default breakpoints \((b_1, b_2) = (0.3, 0.7)\):
r ∈ [0, b₁]r ∈ (b₁, b₂)r ∈ [b₂, 1]The score \(r_t\) is not an ethical objective; it is constrained to preserve accountable choice and must not collapse the human alternative set to a singleton where responsibility must be located.
Each regime is a different information-processing policy with a distinct free-energy profile. Under the framework's modeling commitments, the per-mode expected free energy admits a linear-quadratic approximation whose coefficient signs follow from the EFE decomposition:
AI — low when precision is high, novelty and value conflict low:
\[ G(\pi_{\text{AI}}) \approx \beta_0^{\text{AI}} + \beta_N N + \beta_W W - \beta_{\tilde{\varpi}}\,\tilde{\varpi} \]Hybrid — optimal at moderate uncertainty, enabling shared information gain:
\[ G(\pi_{\text{Hybrid}}) \approx \beta_0^{\text{Hyb}} + \alpha_U U + \alpha_N N + \alpha_{\tilde{\varepsilon}}\,\tilde{\varepsilon} \]Human — low precisely when value conflict is high or novelty demands model revision:
\[ G(\pi_{\text{Human}}) \approx \beta_0^{\text{Hum}} - \alpha_W W - \alpha_N N \]Coefficient signs are the binding theoretical constraint; magnitudes are calibrated through organizational learning rather than treated as discovered in a strong model-identification sense.
The score function \(r(\mathbf{x};\boldsymbol{\theta})\) satisfies three properties that make allocation stable and interpretable:
- Continuity. \(r(\cdot)\) is Lipschitz continuous in \(\mathbf{x}\): small perturbations in dimensional assessments produce proportionate score adjustments, so measurement noise cannot trigger score oscillations. The discrete selector may jump at a threshold, but the underlying score varies smoothly.
- Differentiability. \(r(\cdot)\) is continuously differentiable (\(C^1\)) everywhere, enabling gradient-based sensitivity analysis. Partial derivatives \(\partial r / \partial x_i\) quantify how each dimension drives the recommendation, supporting interpretable explanations.
- Monotonicity. Increasing uncertainty, novelty, value conflict, or prediction error systematically increases the score (toward human involvement); increasing precision decreases it (toward automation). This rules out counterintuitive allocation patterns.
The same formalism defines the boundary between collaborating agents. Partitioning states into internal \(\mu\), external \(\eta\), and blanket states (sensory \(s\), active \(a\)) yields conditional independence:
\[ p(\mu, \eta \mid s, a) = p(\mu \mid s, a)\, p(\eta \mid s, a) \quad\Longleftrightarrow\quad \mu \perp\!\!\!\perp \eta \mid \{s, a\} \]Human internal states (beliefs, goals) and AI internal states (parameters, activations) stay statistically separated yet coupled through their blanket variables. Precision \(\tilde{\varpi}\) modulates the coupling strength while preserving operational closure — the basis for preserving human autonomy. See the boundary visualization →
Parameters are not fixed. When an actual outcome differs from the prediction, the score parameters update by precision-weighted gradient descent on free energy:
\[ \Delta\boldsymbol{\theta} = \eta \cdot \tilde{\varpi} \cdot \tilde{\varepsilon} \cdot \nabla_{\boldsymbol{\theta}}\, \mu(\mathbf{x};\boldsymbol{\theta}) \]where \(\eta\) is the learning rate. Updates scale with precision and with the size of the prediction error, so reliable, surprising outcomes carry the most weight — the organizational-scale analogue of precision-weighted belief updating.