Skip to content

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

Fintech Iterative Cap Redistribution — Index Engineering Algorithm

A canonical, well-specified, cross-language (Python + TypeScript) reference implementation of the capping loop behind every constrained index: pin the members over the cap, share what is left among the rest, and look again — because the redistribution can push somebody else over. The loop is really a search for one number, and this module finds that number directly and checks the loop agrees.

Python TypeScript License Tests

📖 Full article (canonical): Iterative Cap Redistribution — The Fintech Builder

This repository is the runnable, production-oriented companion to that article. The article teaches the concept; this repo is the code you install and build on.

🧭 Browse all algorithms: Awesome FinTech Algorithms — the full index of the library. 🗂️ This algorithm's domain: Index and Benchmark Engineering › Weighting and Capping 📥 Just want to call it? It also ships in the fintech-algorithms npm package — see Two ways to use this.

Catalog topic D03-F02-A07
Domain D03 — Index and Benchmark Engineering
Family D03-F02 — Weighting and Capping
Difficulty 3 / 5
Languages Python, TypeScript

Table of contents


Why it takes more than one pass

weights = raw weights / their total
repeat: every breaching member is fixed at the cap;
        the remaining weight is split pro rata by RAW weight among the members still free
until: no member is above the cap

Cap this basket at 0.25 and the obvious one-pass answer is wrong, because capping A hands weight to everybody else — and that pushes B over the same cap:

pass 1   over the cap: A       pinned A       free members scaled by 1.293103448276
pass 2   over the cap: B       pinned A, B    free members scaled by 1.388888888889
pass 3   over the cap: nobody  pinned A, B    free members scaled by -
ending at the cap: A, B

One multiplier moves the whole free side

Each pass gives every free member the same treatment: its raw weight times a single factor. That is why the uncapped members keep their relative sizes exactly — capping the top of the basket does not reshuffle the bottom of it.

C:D:E:F stay 0.14:0.1:0.07:0.05
published as 0.194444, 0.138889, 0.097222, 0.069444

verify_redistribution checks that property directly (freeMembersKeepTheirRawRatios), which is what a wrong implementation breaks first.


The loop is a search for one number

The only thing the iteration discovers is how many members end up at the cap. Once that count k is known, the answer is a formula: the rest share 1 - k × cap in proportion to their raw weights. Order the members by raw weight and the right k is the smallest one for which the largest remaining member lands at or below the cap — no iteration required.

that count is 2 (A, B), found here without iterating
anything above a raw weight of 0.18 is capped; the rest are scaled by 1.388888888889
one-shot answer matches the loop: true
and the answer is a fixed point -- run it again and nothing moves: true

fixed_point computes it both ways and reports whether they agree. On 300 random baskets in each language they always do — and feeding the answer back in as raw weights returns it unchanged, which is what "converged" actually means.


What a cap costs

The cap is the design decision, so every candidate should be priced:

six names cannot go below a cap of 0.166666666667
cap 0.05  infeasible
cap 0.1   infeasible
cap 0.15  infeasible
cap 0.2   pins 3   turnover 0.24        effective names 5.633147
cap 0.25  pins 2   turnover 0.17        effective names 5.092338
cap 0.3   pins 1   turnover 0.12        effective names 4.664319
cap 0.4   pins 1   turnover 0.02        effective names 3.977864
cap 0.5   pins 0   turnover 0           effective names 3.81971
uncapped, this basket is worth 3.81971 effective names of 6

Tightening the cap buys breadth and pays for it in turnover, monotonically. A cap below 1/n cannot be met by any weights that sum to 1, and is refused up front rather than discovered after the loop fails to converge.


Two ways to use this

This repo is the production home: the full implementation, the analysis surface below, and 154 tests across two languages.

The fintech-algorithms npm package ships the same topic as one import among several hundred.

fintech-algorithms/index-and-benchmark-engineering/weighting-and-capping/iterative-cap-redistribution

Install

Python

cd python
pip install -e ".[dev]"

TypeScript

cd typescript
npm install
npm run build

Quickstart

from fintech_iterative_cap import calculate, fixed_point

basket = {"ids": ["A", "B", "C", "D", "E", "F"],
          "rawWeights": [0.42, 0.22, 0.14, 0.1, 0.07, 0.05],
          "cap": 0.25}

calculate(basket)["weights"]            # [0.25, 0.25, 0.194444, 0.138889, 0.097222, 0.069444]
calculate(basket)["iterations"]         # 3
fixed_point(basket)["cappedCount"]      # 2

TypeScript is the same call:

import { calculate, fixedPoint } from 'fintech-iterative-cap';

calculate(basket).iterations;   // 3

Run the tour in either language:

cd python && python examples/quickstart.py
cd typescript && npm run example

Both print byte-identical output.


Views: the analysis surface

redistribution_trace — the loop, pass by pass

Who breached, who is pinned, the weight left to share, and the single scale every free member is multiplied by — the mechanism in one number per pass.

fixed_point — the same answer without the loop

The capped count, the raw weight at which capping starts (thresholdRawWeight), the scale for the rest, the one-shot weights beside the loop's, matchesTheLoop, and isFixedPoint.

cap_schedule — price every candidate cap

Shown above: feasibility, names pinned, iterations, turnover from the uncapped basket, largest weight, Herfindahl index and effective constituents. Pass your own list of caps, or take the defaults.

verify_redistribution — seven checks on somebody else's weights

theBasketIsEchoedInOrder, theCapIsEchoed, everyWeightIsAtMostTheCap, cappedMembersSitAtTheCap, freeMembersKeepTheirRawRatios, theWeightsSumToOne and theWeightsAreTheOnesTheLoopProduces — all written to publication precision, because a weight rounded to six decimals may sit half a unit above the cap and the sum may miss 1 by half a unit per member. Checking those as exact equalities would fail the correct answer.


Input shape

{
  "ids":        ["A", "B", "C", "D", "E", "F"],  // non-blank, unique, aligned with rawWeights
  "rawWeights": [0.42, 0.22, 0.14, 0.1, 0.07, 0.05],  // positive; any scale, they are normalised
  "cap":        0.25,            // in (0, 1]; cap x members must be at least 1
  "tolerance":  0,               // optional slack on the cap; default 0, must be below the cap
  "maxIterations": 200           // optional, default 200
}

API reference

Function Returns
calculate(data) / iterative_cap_redistribution(data) {ids, weights, cap, iterations, maxWeight, weightSum}
validate_request(data) the parsed request, exact, with inputWeightSum and startingWeights
cap_weights(request) {weights, iterations, passes} in exact fractions
redistribution_trace(data) the loop pass by pass
fixed_point(data) the closed-form answer and the agreement checks
cap_schedule(data, caps?) one row per candidate cap
verify_redistribution(data, result?) {checks, failedChecks, ok, published…}

TypeScript exports the same surface in camelCase, plus Rational, number, render, scaled, toFraction and trim from ./exact.ts.


Edge cases & limitations

This port is stricter than the reference engine, on purpose.

  • Raw weights must be positive, and the sum they arrive with is reported rather than silently normalised away. The reference validated nothing: a negative weight became zero, and a zero one could raise a confusing "no redistribution capacity".
  • The cap is exact. The reference used the numeric tolerance as slack on the cap itself, so a large tolerance stopped the cap binding at all. Here the default slack is zero and the tolerance must be smaller than the cap.
  • Feasibility is exact. cap × members ≥ 1 is checked before any work; the reference allowed it to fall a tolerance below 1.
  • A loop that has not converged refuses. It never publishes a half-capped basket.

Verified, not assumed. Differential against the monorepo reference on 12,000 generated baskets: 0 unexplained divergences, every one classified by name — ids validation 275, non-finite 285, coercion 260, non-positive value 221, tolerance slack 90, infeasible cap 9. An exact-rational oracle written from the article, not from the port, reran the whole loop on all 7,679 port results and agreed on every one. Python vs TypeScript on 1,200 scenarios × 5 calls: byte-identical canonical JSON (1.8 MB). Examples byte-identical and ASCII-only; tsc --strict checked in an isolated staging directory.


Testing

cd python && pytest          # 99 tests
cd typescript && npm test    # 55 tests

Related algorithms


License

MIT © The Fintech Builder