Models
Six families, one at a time.
No single class of model is adequate for a changing world. We treat models as components of one decision architecture: each contributes something specific, and each is explicit about what it represents, what it assumes and when it breaks. The figures are small working instances of each method, not SI's production models.
Model families
Statistical
Explicit probability distributions over what is observed.
- Represents
- Uncertainty as a distribution with interpretable parameters — location, scale, dependence.
- Assumes
- The process generating the data is stable enough for its parameters to mean something.
- Breaks when
- When the distribution shifts, or its tails are heavier than the model allows.
A bivariate Gaussian density with its one- and two-sigma contours. The covariance rotates; samples follow it.
Time-series
Structure in how observations depend on their own past.
- Represents
- Temporal dependence, and forecasts whose intervals widen with the horizon.
- Assumes
- The dependence structure seen in the past persists into the forecast window.
- Breaks when
- When the regime changes and the stated intervals stop covering what actually happens.
An autoregressive process with 50% and 90% forecast bands. The readout tracks how often new observations land inside the 90% band.
Clustering
Grouping observations without labels.
- Represents
- Latent types or regimes as groups in a feature space.
- Assumes
- Distance in that space is meaningful and the number of groups is roughly right.
- Breaks when
- When the groups are artefacts of scaling, initialisation or an arbitrary choice of k.
Lloyd's algorithm for k-means: points are assigned to the nearest centroid, centroids move to the mean of their points, until nothing changes.
Attention
Learning which parts of the context matter for the current step.
- Represents
- Context-dependent weighting across long sequences.
- Assumes
- Enough representative data to learn which relationships are real.
- Breaks when
- When weights latch onto spurious correlations, or inputs fall outside the training distribution.
Softmax attention from one query position to every key in a short sequence. Arc weight and the bars below show the attention distribution.
Feedback
Systems whose outputs become their own inputs.
- Represents
- Loops, stability, equilibria and oscillation.
- Assumes
- The dynamics have been identified correctly.
- Breaks when
- When acting on the system changes the system — the model is part of what it models.
The Van der Pol oscillator's phase plane. Trajectories that start inside and outside all converge to the same limit cycle.
Reinforcement
Learning to act from consequences.
- Represents
- Sequential decisions, and the long-run value of states and actions.
- Assumes
- The reward captures the actual objective and the environment can be explored safely.
- Breaks when
- When the reward is misspecified, or the environment differs from the one used for learning.
Value iteration on a small grid world. Value propagates outward from the goal; once it settles, an agent follows the greedy policy.
One architecture
In practice the families work as parts of a single decision process. Statistical models supply calibrated uncertainty; time-series models carry it forward; clustering proposes the regimes a forecast must respect; attention selects which parts of the history matter now; feedback models keep the system stable while it acts; and a reinforcement learner chooses actions within the limits the others set.
The questions stay the same whichever model is in front of us: what does it need to be true, how will we notice when it is not, and what should the system do then?