ranking

type GainFunction = 'linear' | 'exponential'

How a graded relevance value converts into a discounted gain.

linear uses the grade as-is. exponential uses 2^grade - 1, which is the web-search convention and pulls highly relevant results further ahead of merely acceptable ones.

interface GainOptions {

Options for the discounted-cumulative-gain family.

Properties

function rankedRelevance<T>(ranked: ArrayLike<T>, relevant: Iterable<T>): number[]

Align a ranked list of ids with a set of relevant ids.

import { rankedRelevance, precisionAtK } from 'fino:ml/metrics';

const relevance = rankedRelevance(['d3', 'd7', 'd1'], ['d1', 'd3']);
console.log(relevance); // [1, 0, 1]
console.log(precisionAtK(relevance, 2)); // 0.5

function precisionAtK(relevance: ArrayLike<number>, k: number): number

Fraction of the top k results that are relevant.

When fewer than k results were returned the denominator stays k, so a short list is penalized rather than flattered.

import { precisionAtK } from 'fino:ml/metrics';

console.log(precisionAtK([1, 0, 1, 0], 2)); // 0.5

function recallAtK( relevance: ArrayLike<number>, k: number, options: { totalRelevant?: number } = {}, ): number

Fraction of all relevant items that appear in the top k.

import { recallAtK } from 'fino:ml/metrics';

console.log(recallAtK([1, 0, 1, 0], 2)); // 0.5
console.log(recallAtK([1, 0, 1, 0], 2, { totalRelevant: 4 })); // 0.25

function hitRateAtK(relevance: ArrayLike<number>, k: number): number

Whether any relevant item made the top k, as 1 or 0.

Averaged over queries this is the "did we show them anything useful at all" rate, which is often the metric a product actually cares about.

import { hitRateAtK } from 'fino:ml/metrics';

console.log(hitRateAtK([0, 0, 1], 2)); // 0

function reciprocalRank(relevance: ArrayLike<number>): number

Reciprocal of the rank of the first relevant result, or 0 if there is none.

import { reciprocalRank } from 'fino:ml/metrics';

console.log(reciprocalRank([0, 1, 1])); // 0.5

function meanReciprocalRank(relevances: ReadonlyArray<ArrayLike<number>>): number

Mean reciprocal rank across queries.

import { meanReciprocalRank } from 'fino:ml/metrics';

console.log(meanReciprocalRank([[0, 1], [1, 0]])); // 0.75

function dcgAtK(gains: ArrayLike<number>, k: number, options: GainOptions = {}): number

Discounted cumulative gain over the top k.

Each grade is discounted by log2(rank + 1), so a relevant result found at position 1 is worth more than the same result at position 10.

import { dcgAtK } from 'fino:ml/metrics';

console.log(dcgAtK([3, 2, 3, 0], 4).toFixed(4)); // 5.7619

function ndcgAtK( gains: ArrayLike<number>, k: number, options: GainOptions & { idealGains?: ArrayLike<number> } = {}, ): number

Discounted cumulative gain normalized by the best achievable ordering.

The result is in [0, 1] regardless of how many results a query has or how generous its grades are, which is what makes it comparable across queries. Reports 0 when no ordering could score above zero.

import { ndcgAtK } from 'fino:ml/metrics';

console.log(ndcgAtK([3, 2, 3, 0], 4).toFixed(4)); // 0.9778

function averagePrecisionAtK( relevance: ArrayLike<number>, k: number, options: { totalRelevant?: number } = {}, ): number

Average precision over the top k: precision measured at every hit.

Rewards putting relevant results early rather than merely including them, which precisionAtK alone cannot distinguish.

import { averagePrecisionAtK } from 'fino:ml/metrics';

console.log(averagePrecisionAtK([1, 0, 1, 0], 4)); // 0.8333333333333333

function meanAveragePrecisionAtK( relevances: ReadonlyArray<ArrayLike<number>>, k: number, ): number

Mean average precision across queries.

import { meanAveragePrecisionAtK } from 'fino:ml/metrics';

console.log(meanAveragePrecisionAtK([[1, 0], [0, 1]], 2)); // 0.75