Turn a collection of p-values into defensible discoveries

Start here

Adjust once, keep the original order

import jdistlib.disttest.MultipleTesting;
import jdistlib.disttest.MultipleTesting.Method;

double[] p = {0.01, 0.04, 0.03, 0.002, 0.50};
double[] adjusted = MultipleTesting.adjust(
    p, Method.BENJAMINI_HOCHBERG);

// adjusted: 0.025, 0.05, 0.05, 0.01, 0.50

Each adjusted value belongs to the hypothesis at the same array position. The input is never sorted or mutated.

Choose deliberately

The method encodes the error target and assumptions

If the hypothesis family contained tests whose p-values were not supplied, use adjust(p, method, totalNumberOfTests). The default counts only non-missing values.

Decision helpers

Ask for flags, a count, or the raw cutoff

boolean[] rejected = MultipleTesting.reject(
    p, 0.05, Method.BENJAMINI_HOCHBERG);
int count = MultipleTesting.countRejected(
    p, 0.05, Method.BENJAMINI_HOCHBERG);
double rawCutoff = MultipleTesting.threshold(
    p, 0.05, Method.BENJAMINI_HOCHBERG);

threshold returns the largest observed raw p-value among the rejected hypotheses, or NaN when none is rejected. Reporting the adjusted values is usually more informative than reporting only this cutoff.

Adaptive FDR

Use two-stage BKY when its assumptions fit

MultipleTesting.AdaptiveFdrResult bky =
    MultipleTesting.benjaminiKriegerYekutieli(p, 0.05);

boolean[] rejected = bky.getRejected();
int firstStage = bky.getStageOneRejections();
int estimatedNulls = bky.getEstimatedTrueNulls();
double finalBhLevel = bky.getFinalLevel();

BKY first runs BH at q / (1 + q), estimates the number of true nulls as m - r1, and runs BH again at the resulting adaptive level. It can improve power, but its proven FDR guarantee is for independent test statistics.

Prior information · JDistlib 0.7.1+

Give prespecified hypotheses more or less priority

// Larger weights give greater priority. The API rescales them
// to mean one, so only their ratios matter.
double[] weights = {2.0, 1.0, 0.5, 0.5, 1.0};
double[] weightedAdjusted =
    MultipleTesting.adjustWeightedBenjaminiHochberg(
        p, weights);
boolean[] weightedRejected =
    MultipleTesting.rejectWeightedBenjaminiHochberg(
        p, weights, 0.05);

double[] weightedHolm =
    MultipleTesting.adjustWeightedHolm(p, weights);
double[] weightedBy =
    MultipleTesting.adjustWeightedBenjaminiYekutieli(
        p, weights);

Weighted BH and BY order p[i] / weight[i]; weighted Holm uses that ordering and reallocates the remaining weight after each step. Weighted Bonferroni is also available. BH is for independence or suitable positive dependence, BY retains FDR control under arbitrary dependence, and Bonferroni/Holm control family-wise error under arbitrary dependence. Every weight must be finite and positive. Multiplying all weights by the same constant changes nothing because JDistlib normalizes their mean to one over the non-missing family.

Adaptive step-down · JDistlib 0.7.1+

Use GBS when independence is defensible

MultipleTesting.StepDownFdrResult gbs =
    MultipleTesting.gavrilovBenjaminiSarkar(p, 0.05);

boolean[] rejected = gbs.getRejected();
int count = gbs.getRejectedCount();
double rawCutoff = gbs.getThreshold();
double rankCutoff = gbs.getCriticalValue();

GBS walks upward through sorted p-values and stops at the first failure, using the rank-i boundary i*q / (m + 1 - i*(1-q)). The result exposes both the largest rejected p-value and its critical boundary. Its proven finite-sample FDR guarantee is under independence.

Structured families · JDistlib 0.7.1+

Select groups, then test within the selected groups

int[] group = {10, 10, 20, 20, 30, 30};
MultipleTesting.GroupedFdrResult grouped =
    MultipleTesting.selectiveGroupedBenjaminiHochberg(
        p, group, 0.05, 0.05);

boolean[] selectedGroups = grouped.getSelectedGroups();
boolean[] rejected = grouped.getRejected();

The first stage computes a Simes p-value for every labeled group and selects groups with BH. If R of G groups are selected, the second stage runs BH inside each selected group at qWithin * R / G. The advertised target is the expected average FDR over selected families, not a pooled FDR over every leaf hypothesis. Group labels may be any integers; all p-values must be present.

Growing streams · JDistlib 0.7.1+

Use a stateful controller when hypotheses arrive over time

double[] gamma = OnlineFdr.polynomialGamma(10_000, 1.6);
LordPlusPlus lord = new LordPlusPlus(0.05, 0.025, gamma);
OnlineFdrDecision decision = lord.test(nextPValue);

Saffron saffron = new Saffron(
    0.05, 0.01, 0.5, gamma);
OnlineFdrDecision adaptive = saffron.test(nextPValue);

test chooses its level from past decisions before seeing the new p-value, returns that level and decision, and advances the controller once. LORD++ and SAFFRON require a prespecified order and independent null p-values for their standard FDR guarantees. SAFFRON additionally treats p-values at or below lambda as candidates. The finite gamma array is zero-padded after its horizon; choose a horizon long enough for the stream or deliberately accept that later spending can reach zero.

Exact tests · JDistlib 0.7.1+

Exploit known discrete null distributions without using the Heyse shortcut

DiscretePValueDistribution[] nulls = {
    DiscretePValueDistribution.exact(
        new double[] {0.01, 0.05, 0.20, 1.0}),
    new DiscretePValueDistribution(
        support2, leastFavorableCdf2)
};
DiscreteFdr.Result result = DiscreteFdr.dbhStepDown(
    p, nulls, 0.05); // dbhStepUp is also available

Each hypothesis supplies its finite attainable support and least-favorable null CDF. The constructor verifies a nondecreasing, super-uniform CDF ending at one. DBH computes level-dependent critical values from all heterogeneous CDFs and controls FDR under independence. exact(support) is a convenience only when the exact-test CDF equals the attainable p-value at every support point.

Numerical range

Keep extremely small probabilities in log space

double[] logP = {-1000.0, -720.0, Math.log(0.03)};
double[] logAdjusted = MultipleTesting.adjustLog(
    logP, Method.BENJAMINI_HOCHBERG);
boolean[] rejected = MultipleTesting.rejectLog(
    logP, 0.05, Method.BENJAMINI_HOCHBERG);

adjustLog accepts natural logarithms, including negative infinity for an exact zero, and returns natural-log adjusted p-values. The calculations stay in the log domain, including Šidák and Hommel, so values such as exp(-1000) never have to underflow to zero.

Incomplete recording

Right-censored p-values need a known family size

Suppose a pipeline records only p-values at or below 0.05. This is not the same as ordinary missing data: every omitted value is known to be greater than 0.05. Supply both that limit and the total number of tests.

double[] recorded = {0.001, 0.004, 0.01, 0.03};
MultipleTesting.CensoredTestResult result =
    MultipleTesting.testRightCensored(
        recorded, 0.05, 10_000, 0.05,
        Method.BENJAMINI_HOCHBERG);

double[] conservativeAdjusted = result.getAdjustedPValues();
boolean exactAtThisLevel = result.areDecisionsExact();

The implementation completes the unrecorded tail with p=1. The returned adjusted values are therefore conservative. Decision flags are exact when the censoring limit is at least the method's largest possible rejection boundary—for BH at level q, for example, when the limit is at least q. Otherwise, reported discoveries remain valid but more discoveries may be unknowable from the retained data.

Storey q-values

Estimate how many hypotheses are truly null

// Estimate from the complete study-wide collection of p-values.
double pi0 = MultipleTesting.estimateNullProportion(studyPValues);
double[] q = MultipleTesting.qValues(studyPValues, pi0);

// Robust alternative inherited from the QGeneric workflow.
double quantilePi0 =
    MultipleTesting.estimateNullProportionQuantile(
        studyPValues, 0.10);
double[] quantileQ = MultipleTesting.qValues(
    studyPValues, quantilePi0);

The default lambda grid runs from 0.05 through 0.95. Advanced callers can supply the grid and smoothing degrees of freedom, or call qValues(p, pi0) when π₀ comes from an external analysis. Setting π₀ to one gives the Benjamini–Hochberg adjusted values. A zero estimate is rejected with guidance to inspect the p-value distribution or choose a suitable lambda grid.

Input contract

Use NaN for missing p-values

NaN is preserved at the same position and excluded from the default test count. Every other input must be finite and between zero and one. This replaces QGeneric's private undefined-value sentinel and prevents silent ordering errors.

The implementation also corrects the legacy Šidák direction: adjusted values use 1 - (1 - p)^m, evaluated through stable logarithmic functions for very small probabilities.