How to Calculate Federated Learning Privacy Budget
Differential privacy (DP) has become the de facto assurance framework for federated learning programmes that aggregate updates from consumer devices or sensitive enterprises. A credible privacy budget—expressed as the ordered pair (ε, δ)—is more than a badge of compliance. It signals how aggressively you sample clients, how noisy you make their contributions, and how long you run the training job. Privacy engineers must therefore move beyond folklore toward a structured calculation that finance, legal, and safety teams can audit.
This walkthrough codifies the DP-SGD moments accountant approximation tailored to cross-device federated learning. We explain how to define the analytical boundary, identify the variables you must measure, derive the equations, and validate the output against telemetry. The approach complements sample-size reasoning in the LLM validation guide and risk coverage analytics discussed in the generative AI red team coverage walkthrough so governance teams can triangulate capability, risk, and privacy.
Define the privacy budget boundary
Start by articulating what portion of the federated workflow you will subject to DP guarantees. The canonical DP-SGD setup clips each client’s gradient update to a norm bound, adds Gaussian noise, and aggregates the result. The privacy budget then covers the release of model updates to the coordinating server. Explicitly exclude post-processing steps such as evaluation, thresholding, or retention policies because DP guarantees are preserved under post-processing. Likewise, if you generate audit trails or incident reports, ensure they only rely on the privatised model state; otherwise you must analyse additional disclosures separately.
Your boundary should specify whether participation is without replacement within a round, how many rounds execute between model releases, and whether adaptive clipping is in scope. Documenting these conditions keeps stakeholders aligned when you later revisit privacy posture alongside incident metrics in the AI safety response coverage guide.
Catalogue variables, units, and data sources
Collect the following variables in SI-consistent units so that the privacy accountant remains auditable:
- N – Total eligible clients or devices participating in the federation (dimensionless count). Source: enrolment ledger or authentication directory.
- m – Clients sampled per training round (dimensionless count). Source: orchestration logs.
- q – Sampling ratio, defined as m ÷ N (unitless). Calculated value.
- T – Number of aggregation rounds included in the privacy guarantee (dimensionless count). Source: training scheduler.
- σ – Noise multiplier applied to clipped gradients (dimensionless). Source: DP-SGD configuration.
- C – Clipping norm (L2 magnitude). Even though the basic ε calculation does not use C explicitly, auditors expect to see the bound recorded.
- δ – Target failure probability, typically set below 1 ÷ N. Choose δ conservatively; if regulators impose a threshold (e.g., 10−6), enforce it even when N is larger.
Automate data collection where possible. Many production systems log round participation and clipping statistics; feed those logs into your accounting notebook. Doing so reduces manual transcription errors and lets you rerun the calculation whenever hyperparameters change mid-training.
Derive the DP-SGD privacy budget formula
The federated learning community widely adopts the moments accountant introduced by Abadi et al. (2016) to approximate ε. For Poisson subsampling with sampling ratio q, noise multiplier σ, T rounds, and target δ, a practical upper bound on ε is:
ε ≈ q × √(2T × ln(1/δ)) ÷ σ + (T × q²) ÷ σ²
δ < 1 ÷ N
The first term corresponds to the square-root growth predicted by advanced composition, while the second term captures higher-order effects that become relevant as participation becomes denser or noise decreases. Although more precise accountants exist (Rényi DP, privacy loss distributions), this closed form is widely accepted for programme design and matches what our embedded calculator computes. Always keep σ strictly positive; if you disable noise, ε collapses to infinity because the mechanism is no longer differentially private.
Step-by-step calculation workflow
1. Validate cohort size and participation
Begin with the enrolment ledger to confirm N, the number of registered clients. Remove devices that have opted out or have been inactive past your retention window. Next, pull orchestration logs to compute m, the number of clients actually sampled per round. Divide m by N to obtain q. If you employ importance sampling or straggler tolerance, derive an effective sampling rate by averaging across the rounds in scope.
2. Confirm clipping norms and noise multiplier
Inspect the DP-SGD configuration to record C and σ. Clipping thresholds often adapt as models stabilise; freeze the minimum observed C for conservatism. Verify that noise injectors use the stated σ by comparing logged variances of added noise to σ²C². If multiple layers or parameter groups use different clips, ensure each is at most C before applying the accountant.
3. Count rounds and enforce δ policy
Determine T, the number of rounds you will include. For rolling training programmes, teams often report privacy budgets per calendar period (e.g., monthly). Align T with that reporting cadence. Select δ according to policy: a common default is δ = 1 ÷ N, but some organisations cap δ at 10−6 to simplify compliance documentation. Record the rationale in your privacy impact assessment.
4. Compute ε and interpret the components
Plug N, m, T, σ, and δ into the formula. Calculate both the square-root term and the quadratic term separately before summing them into ε. Logging each component helps stakeholders see whether the privacy cost is dominated by long training runs (large T) or low noise (small σ). Translate q into a percentage so non-experts understand how frequently clients participate.
5. Document versioning and governance
Archive the calculation with hash-stamped references to the model version, dataset descriptors, and any mitigation playbooks triggered by privacy drift. Synchronise the privacy budget report with release governance so the figure appears alongside risk metrics such as coverage scores or escalation queues tracked with the AI agent escalation coverage calculator.
Validation and monitoring
Validation occurs on two fronts. First, reconcile the sampling ratio by comparing orchestration logs against the counts used in your accountant. Any discrepancy indicates that the effective q differs from the design assumption. Second, run sanity checks on σ by replaying a subset of rounds offline and measuring noise variance empirically. Deviations suggest instrumentation drift or misconfigured pseudorandom generators.
Monitor privacy loss over time by plotting ε per release cycle. If you operate continuous training, maintain a cumulative privacy ledger that subtracts expended budget once models are decommissioned. Combine the ledger with readiness analytics from the RAG knowledge half-life walkthrough to demonstrate ongoing diligence across model quality, privacy, and governance.
Limits and interpretation
The approximation used here assumes Poisson subsampling and constant σ across rounds. If your system performs secure aggregation with varying participation or adaptively tunes noise, you must revert to a full Rényi DP or privacy loss distribution accountant. Likewise, DP only protects against the release of model updates. Side channels such as timing information, metadata leakage, or device attestation logs require independent mitigations.
Remember that ε is not a single universally interpretable number. Industry practice often targets ε between 1 and 8 for consumer products, but regulators may request context on risk appetite, attack models, and actual data sensitivity. Provide narrative commentary alongside the numeric output so readers understand why the budget is appropriate for your deployment.
Embed: Federated learning privacy budget calculator
Input your cohort size, sampling cadence, rounds, and noise multiplier to generate an auditable ε, δ pair along with composition diagnostics.