On this page
GaussianAnalysisconvex/
convex/
Self-concordant barriers and the log-determinant barrier, minimizers of least squares objectives with a barrier, the Lindeberg replacement for barrier-regularized least squares with regular features and the removal of the barrier and of the ridge, projections onto closed convex sets and Moreau's decomposition for closed convex cones.
Guide
Written for this documentation from the sources and the README, and checked against the Rocq statements. The file headers and the statements are the reference.
This directory holds Theorem 2.9 of Koehler and Sohn, Universality and sharp thresholds for ellipsoid fitting (arXiv 2608.27372): a Lindeberg argument for least squares with a self-concordant barrier. Around it are the convex analysis the theorem needs, the removal of the barrier (Lemma 2.11), and Moreau's decomposition for closed convex cones.
The setting is a subspace V of 'rV[R]_N, a convex set D open in its affine hull z + V, and MathComp's directional derivatives 'D_h along V; matrices enter through mxvec. Self-concordant barriers, such as - ln det on trace slices of the positive definite cone, regularize least squares objectives of n features. For n independent features from two regular laws with the same covariance, the optimal values have close means and concentrate.
Reading order
self_concordant.v: self-concordant functions and barriers, the Dikin ellipsoid, Bregman and barrier bounds.logdet_barrier.v: - ln det and its two-sided form as barriers on trace slices of the positive definite cone.least_squares.v: the objectives, their values, and regular features.minimizers.v: minimizers, first-order conditions, and Taylor's formula with a Bregman remainder.ls_insertion.v: pointwise bounds on the cost of inserting one feature.feature_moments.v: moments of linear and quadratic forms under a regular feature.gauss_feature.v: Gaussian features are regular.feature_mixture.v: conditions under which a finite mixture of centered laws is a regular feature.barrier_removal.v: the cost of the barrier in the value.barrier_lindeberg.v: measurability of the value, a telescoping lemma, and Theorem 2.9.ridge_comparison.v: concentration of the value without barrier.moreau.v: projections onto closed convex sets and Moreau's decomposition.
Core definitions
rel_open_convex(self_concordant.v): D is convex, y - z is in V for y, z in D, and D is open in z + V.sc_function,sc_barrier(self_concordant.v): B is three times differentiable along V, with a gradient and a Hessian, D^2 B[h,h] >= 0, (D^3 B[h,h,h])^2 <= 4 (D^2 B[h,h])^3, and B tends to +oo along segments leaving D; a theta-barrier also has (D B[h])^2 <= theta D^2 B[h,h].pd_slice,logdet_barrier,pd_slice2,logdet_barrier2(logdet_barrier.v): the positive definite matrices of trace s with - ln det, and the matrices of trace n strictly between m I and M I with the two-sided barrier.ls_objective(least_squares.v): F_Z(theta, b; beta) = (2n)^-1 sum_i (b - <Z_i, theta>)^2 + beta (lambda |theta|^2 + b^2).ls_barrier_value,ls_value(least_squares.v): Phi_Z(D, B; beta, gamma), the infimum of F_Z(theta, b; beta) + gamma B theta over theta in D and real b, and Phi_Z(D; beta), the same without barrier.regular_feature(least_squares.v): regular_feature K tau Sigma mu means lambda = opnorm Sigma > 0, mu is centered with covariance Sigma, E <z, theta>^4 <= K^4 lambda^2 |theta|^4, and Var (z T z^T) <= tau^2 |T|_F^2 for T symmetric.proj,polar(moreau.v): the nearest point of K to z, when it exists, and the polar cone.
Main results
Throughout, D is convex and relatively open with directions V.
Self-concordance.
sc_taylor3(Lemma 2.15 (ii)): for B self-concordant on D, z in D and h in V, if the local norm r = sqrt (D^2 B(z)[h,h]) (sc_norm) is at most 1/2, then z + h is in D and |B (z + h) - B z - D B(z)[h] - r^2 / 2| <= 2 r^3 / 3.sc_barrier_contraction(Lemma 2.10): for a theta-barrier with theta >= 0, x0, x in D and 0 < eps < 1, B ((1 - eps) x + eps x0) <= B x0 + theta ln (1 / eps).logdet_sc_barrier,logdet2_sc_barrier(Lemma 2.8): with the symmetric matrices of trace 0 as directions, logdet_barrier n is an n-self-concordant barrier on pd_slice n s, and logdet_barrier2 n m M a 2n-self-concordant barrier on pd_slice2 n m M.
Minimizers and features.
sc_minimizer: if D is nonempty, B >= 0 is self-concordant on D, gamma > 0, and g is continuous, convex on D and at least kappa |z|^2 with kappa > 0, then g + gamma B attains its minimum on D.ins_cost_quadratic(Proposition 2.14): for B >= 0 self-concordant on D, lambda, beta, gamma > 0, 36 <= n gamma and a minimizer (theta_m, b_m) on D x R of the leave-one-out objective without feature i, the cost Delta(z) of inserting z as feature i (ins_cost) satisfies |Delta(z) - r(z)^2 / (2 (n + l(z)))| <= (2 + 3 r(z)^4) / (n sqrt (n gamma)), where r(z) = b_m - <z, theta_m> and l(z) = <alpha(z), H^-1 alpha(z)> (ins_ell), H the Hessian at the minimizer.gauss_regular_feature: for S != 0, the law of x S with x of law gauss k is (3^(1/4), sqrt 2 opnorm (S^T S))-regular with covariance S^T S.mixture_regular_feature: a finite mixture, with weights w_i, of centered laws with second-moment matrices Sig_i, E_i <z, theta>^4 <= A |theta|^4 and Var_i (z T z^T) <= tau^2 |T|_F^2 for T symmetric is (K, sqrt (tau^2 + c^2))-regular with covariance Sigma = sum_i w_i Sig_i, if opnorm Sigma > 0, sum_i w_i |Sig_i - Sigma|_F^2 <= c^2 and A <= K^4 opnorm Sigma^2.
Universality of the values (Theorem 2.9). Let mu and nu be (K, tau)-regular with the same covariance Sigma, lambda = opnorm Sigma, tau >= 0, D nonempty, B >= 0 self-concordant on D, n > 0, beta, gamma > 0, Phi = Phi_Z(D, B; beta, gamma), and Lref = (1 + 1/beta) (1 + inf over D of lambda |theta|^2 + gamma B theta) (ls_ref_scale).
barrier_lindeberg_mean: if 36 <= n gamma, then |E_(mu^n) Phi - E_(nu^n) Phi| <= 100 (1 + K^4)^2 Lref^2 ((n gamma)^(-1/2) + sqrt (dim V) (tau / lambda + 1) / n).barrier_lindeberg_variance,barrier_lindeberg_concentration: Var_(mu^n) Phi <= 4 (1 + K^4)^2 Lref^2 / n; for t > 0 and 36 <= n gamma, |Phi - E_(nu^n) Phi| exceeds the mean bound plus t with mu^n-probability at most 4 (1 + K^4)^2 Lref^2 / (n t^2).
Removing the barrier.
barrier_removal(Lemma 2.11): for every tuple Z, a theta-barrier B >= 0 on D with theta >= 0, theta_ref in D, b_ref real, lambda, beta >= 0, gamma > 0 and 0 < eps < 1, 0 <= Phi_Z(D, B; beta, gamma) - Phi_Z(D; beta) <= eps F_Z(theta_ref, b_ref; beta) + gamma B theta_ref + gamma theta ln (1 / eps).ridge_comparison: under the hypotheses of Theorem 2.9, with B a theta-barrier, theta >= 0, theta_0 in D, 0 < eps < 1, t, M > 0 and 36 <= n gamma, |Phi_Z(D; beta) - E_(nu^n) Phi_Z(D, B; beta, gamma)| exceeds the mean bound plus t + eps M + gamma B theta_0 + gamma theta ln (1 / eps) with mu^n-probability at most 4 (1 + K^4)^2 Lref^2 / (n t^2) + (theta_0 Sigma theta_0^T / 2 + beta lambda |theta_0|^2) / M.
Projections.
proj_variational,proj_nonexpansive: for K closed, convex and nonempty, p in K is nearest to z iff <z - p, y - p> <= 0 for y in K, and proj K is 1-Lipschitz.moreau(Lemma 5.1),cone_proj_support: for a closed convex cone C, z = proj C z + proj (polar C) z with orthogonal parts, and the supremum of <z, y> over y in C with |y| <= 1 is |proj C z|.
How the proofs go
- Bounds on the Hessian along a segment, from a comparison in one variable (
sc_ode_ge,sc_ode_le), give the Bregman bounds of Lemma 2.15 and, with the blow-up of B at the exit point, the Dikin ellipsoid (sc_dikin). logdet_linewrites - ln det (A + t U) as c - sum_i ln (1 + t l_i) by simultaneous diagonalization, andlogsum_sc_barrierchecks the barrier conditions in one variable.- Theorem 2.9 follows the paper: measurability through a countable dense subset of D (
measurable_inf_rel_open), a telescoping over the coordinates (lindeberg_telescope) withloo_compareat each step, built oninsertion_compare, Efron-Stein for the variance and Chebyshev for the concentration. The paper asks D to be bounded and B to be a barrier; the proof uses neither: the ridge term makes the objective coercive, and a self-concordant B suffices. - Lemma 2.11 contracts toward the reference point;
ridge_comparisonadds Markov's inequality for F_Z(theta_0, 0; beta); mixtures use the law of total variance.
Connections
The directory requires modules of matrix/ (euclidean.v, opnorm.v, real_spectral.v), gaussian/ and probability/, and comparison/. In the library, probability/ requires four of these files, and moreau.v is required by comparison/, comparison/, matrix/, matrix/ and matrix/. Modules of extension/ require most of these files; applications/ requires none.
Files
| File | Title | Lines | Lemmas | Defs | Requires in this directory |
|---|---|---|---|---|---|
| self_concordant.v | Self-concordant functions and barriers | 1,247 | 41 | 7 | none |
| least_squares.v | Constrained least squares with features, and regular features | 254 | 9 | 7 | none |
| logdet_barrier.v | The log-determinant barrier | 904 | 36 | 12 | self_concordant |
| minimizers.v | Minimizers of least squares objectives with a barrier | 640 | 18 | 2 | self_concordant, logdet_barrier, least_squares |
| feature_moments.v | Moments of regular features | 544 | 42 | 0 | least_squares |
| gauss_feature.v | Gaussian features are regular | 522 | 8 | 0 | least_squares, feature_moments |
| ls_insertion.v | Inserting one feature in a barrier-regularized least squares problem | 807 | 39 | 12 | self_concordant, logdet_barrier, least_squares, minimizers |
| barrier_removal.v | Removing the barrier | 177 | 5 | 0 | self_concordant, least_squares |
| barrier_lindeberg.v | Universality of barrier-regularized least squares | 1,576 | 37 | 2 | self_concordant, least_squares, minimizers, ls_insertion, feature_moments |
| ridge_comparison.v | Least squares values with a ridge and without barrier | 276 | 7 | 0 | self_concordant, least_squares, feature_moments, barrier_lindeberg, barrier_removal |
| moreau.v | Projections onto closed convex sets and Moreau's decomposition | 335 | 15 | 2 | none |
| feature_mixture.v | Mixtures of features | 291 | 12 | 0 | least_squares, feature_moments |
Table of main results35 rows of the README
| File | Results | Statement |
|---|---|---|
| self_ | Self-concordant functions and barriers (Nesterov-Nemirovskii; Koehler-Sohn, Definition 2.7) on a convex set D of 'rV[R]_N that is relatively open in z + V for a subspace V: derivatives along the directions of V (MathComp's 'D_h) up to order 3, a gradient vector and a Hessian matrix, convexity, (D^3 B[h,h,h])^2 <= 4 (D^2 B[h,h])^3, and B tends to +oo along segments that leave D; a theta-barrier also has (D B[h])^2 <= theta D^2 B[h,h]. sc_norm B z h is the local norm sqrt (D^2 B(z)[h,h]), bregman_div B y z = B y - B z - D B(z)[y - z]. | |
Along a segment z + t h in D, with r = sc_norm B z h: r^2 / (1 + t r)^2 <= D^2 B(z + t h)[h,h], and D^2 B(z + t h)[h,h] <= r^2 / (1 - t r)^2 when t r < 1. The one-dimensional comparison behind it compares 1 / sqrt (psi + e) with t for a function psi >= 0 with psi'^2 <= 4 psi^3. | ||
The Dikin ellipsoid: if sc_norm B z h < 1 then z + h is in D (Nesterov-Nemirovskii, Theorem 2.1.1); proved from the bound above and the blow-up of B at the exit point of the segment. | ||
Lemma 2.15 of Koehler and Sohn: r - ln (1 + r) <= bregman_div B (z + h) z when z + h is in D, and bregman_div B (z + h) z <= - r - ln (1 - r) when r < 1; hence (i) if the Bregman divergence is at most 1/2 - ln (3/2) then r <= 1/2 and it is at least r^2 / 3, and (ii) if r <= 1/2 then z + h is in D and |B (z + h) - B z - D B(z)[h] - r^2 / 2| <= 2 r^3 / 3. | ||
A self-concordant function is convex on D, and continuous at every point of D for the points of D. | ||
Lemma 2.10 of Koehler and Sohn, for a theta-self-concordant barrier with theta >= 0: D B(z)[y - z] <= theta for y, z in D (Nesterov, Theorem 4.2.4), and B ((1 - eps) x + eps x0) <= B x0 + theta ln (1 / eps) for x0, x in D and 0 < eps < 1. | ||
Monotonicity on [a, b] from the sign of the derivative, and a function with a nonnegative second derivative on [a, b] lies above its tangents there. | ||
If B agrees on D, along the line through z in the direction h of V, with a function of one variable that has three derivatives, these are the derivatives of B along h at the points of the line in D. A linear form on row vectors is mxdot g h and a bilinear form Q has Q h h = h M h^T, for the vector and the matrix of their values on the basis. | ||
| logdet_ | Matrices as row vectors through mxvec: pd_slice n s is the set of the positive definite n x n matrices of trace s, logdet_barrier n x = - ln det (vec_mx x), pd_slice2 n m M is the set of the matrices A of trace n with m I < A < M I, logdet_barrier2 n m M x = - ln det (A - m I) - ln det (M I - A) + n ln ((1 - m) (M - 1)), and sym_traceless n is the subspace of the symmetric matrices of trace 0. | |
For A positive definite and U symmetric there are G invertible and reals l_i with A = G^T G and U = G^T diag(l) G; hence -ln det (A + t U) = c - sum_i ln (1 + t l_i) whenever A + t U is positive definite, which happens exactly when every 1 + t l_i is positive, and tr (A^-1 U) = sum_i l_i, tr ((A^-1 U)^2) = sum_i l_i^2. | ||
A function that equals c - sum_(i < k) ln (1 + t l_i) along every line through D (with D convex and relatively open, the l_i and c depending on the line), and whose first and second derivatives at each point of D are a linear and a bilinear form, is a k-self-concordant barrier on D. | ||
Lemma 2.8 of Koehler and Sohn, one-sided (Nesterov-Nemirovskii, Proposition 5.4.5): pd_slice n s is convex and relatively open with directions sym_traceless n, and logdet_barrier n is an n-self-concordant barrier on it. On pd_slice n n it is nonnegative (det A <= (tr A / n)^n), with value 0 at the identity. | ||
Lemma 2.8 of Koehler and Sohn, two-sided: pd_slice2 n m M is convex and relatively open with directions sym_traceless n, and logdet_barrier2 n m M is a 2n-self-concordant barrier on it; if m < 1 < M it is nonnegative there, with value 0 at the identity. | ||
| least_ | The least squares objective with a selection P of the features and a weight s: s sum_(i in P) (b - <Z_i, theta>)^2 + beta (lambda |theta|^2 + b^2) + gamma B theta; with P the indices other than i and s = 1 / (2 n) it is the leave-one-out objective of the Lindeberg argument. | |
| minimizers.v | Every subspace V of 'rV[R]_N is the row space of a matrix with \dim V orthonormal rows, and is closed; for D convex and relatively open with directions V, z in D and y in the closure of D, y - z is in V and z + t (y - z) is in D for 0 <= t < 1. | |
For B >= 0 a self-concordant function on D and g continuous, convex on D and nonnegative, the sublevel sets of g + gamma B in D are closed, and if g z >= kappa |z|^2 with kappa > 0 then g + gamma B attains its minimum on D (D need not be bounded). In particular the least squares objective with a barrier attains its minimum on D x R. | ||
At a minimizer (theta, b) the derivative along every direction of V x R is 0, and F (theta + h) (b + c) - F theta b is the quadratic part at (h, c) plus gamma times the Bregman divergence of B (Koehler-Sohn, (2.17)-(2.18)). For H positive definite, the infimum over u of u H u^T / 2 + (r + <a, u>)^2 / (2 nn) is r^2 / (2 (nn + l)) with l = a H^-1 a^T ((2.19)). | ||
| ls_ | Lemmas 4.1 and 4.2 and Proposition 2.14, (2.23), pointwise: for the leave-one-out minimizer of the barrier-regularized problem, the Hessian H in coordinates of V x R is positive definite; the cost Delta(z) of inserting a feature z satisfies 0 <= Delta(z) <= r(z)^2 / (2 n), and if 36 <= n gamma then |Delta(z) - r(z)^2 / (2 (n + l(z)))| <= (2 + 3 r(z)^4) / (n sqrt (n gamma)), with r(z) the residual of z and l(z) = <alpha(z), H^-1 alpha(z)>. | |
l(z) = c0 - 2 <v, z> + z T z^T with T symmetric, 0 <= c0 <= 1 / (2 beta), |v|^2 <= k / (4 beta^2 lambda), 0 <= z T z^T <= |z|^2 / (2 beta lambda) and |T|_F^2 <= k / (4 beta^2 lambda^2), k = \dim V; and (2.21): beta (lambda |theta_m|^2 + b_m^2) is at most the leave-one-out objective at (theta_ref, 0). | ||
| feature_ | For a (K, tau)-regular feature mu with covariance Sigma (Koehler-Sohn, Definition 2.5): every measurable g with |g z| <= a + b |z|^4 is integrable, E <z, theta> = 0, E <z, theta> <z, theta'> = theta Sigma theta'^T, E z T z^T depends only on Sigma and T, and E (z T z^T - E z T z^T)^2 <= tau^2 |T|_F^2 for T symmetric. | |
E (b - <z, theta>)^2 = b^2 + theta Sigma theta^T, E (b - <z, theta>)^4 <= 8 (1 + K^4) (lambda |theta|^2 + b^2)^2 with lambda = opnorm Sigma, and for l z = c - 2 <z, v> + z T z^T with T symmetric, E l = c + E z T z^T and E (l - E l)^2 <= 2 tau^2 |T|_F^2 + 8 lambda |v|^2; v Sigma v^T <= opnorm Sigma |v|^2. | ||
| gauss_ | For x of law gauss k and A symmetric, E x A x^T = tr A and E (x A x^T)^2 = (tr A)^2 + 2 |A|_F^2 (spectral theorem and orthogonal invariance of gauss k); E <x, a> = 0, E <x, a> <x, b> = <a, b> and E <x, a>^4 = 3 |a|^4. | |
(2.10) of Koehler and Sohn: for S != 0 and a law nu on 'rV[R]_N with nu A = gauss k [set x | A (x S)], nu is a (3^(1/4), sqrt 2 opnorm (S^T S))-regular feature with covariance S^T S; opnorm S ^ 2 <= opnorm (S^T S). | ||
| barrier_ | Lemma 2.11 of Koehler and Sohn: for B >= 0 a theta-self-concordant barrier on D (convex, relatively open), theta_ref in D, lambda, beta >= 0, gamma > 0 and 0 < eps < 1, 0 <= Phi_Z(D, B; beta, gamma) - Phi_Z(D; beta) <= eps F_Z(theta_ref, b_ref; beta) + gamma B theta_ref + gamma theta ln (1 / eps) for every tuple of features Z, by contracting toward the reference point (F_Z is convex); hence (2.15), the comparison of the values without barrier for two tuples W and G. | |
| barrier_ | The value Phi_Z(D, B; beta, gamma) of the barrier-regularized problem is a measurable function of the features Z, for D convex and relatively open and B >= 0 continuous on D: an infimum over D of an upper semicontinuous family is the infimum over a countable dense subset. | |
Lindeberg's replacement for a nonnegative integrable functional F of n independent coordinates, with bounds b_i that may depend on the other coordinates: if |E_mu F(x^(i <- t)) - E_nu F(x^(i <- t))| <= b_i x for all i and x, then |E_(mu^n) F - E_(nu^n) F| <= sum_i E b_i under the hybrid law with mu before i and nu after. The integral of a function that does not depend on a coordinate does not depend on its law, and the coordinate j of tuple_prod rho n has law rho j. | ||
Propositions 2.14 and 2.16 of Koehler and Sohn: for two regular features with the same covariance, the means of the cost of inserting one feature differ by at most (4 + 48 (1 + K^4) S^2) / (n sqrt (n gamma)) + 4 sqrt (1 + K^4) sqrt (dim V) (tau / lambda + 1) S / (beta n^2), where S = lambda |theta_m|^2 + b_m^2 is the size of the leave-one-out minimizer; for the value itself the same holds with S replaced by its bound ((2 n)^-1 sum_(j <> i) <x_j, theta_ref>^2 + beta lambda |theta_ref|^2 + gamma B theta_ref) / beta, and the variance in the coordinate i is at most 2 (1 + K^4) / n^2 times the square of that bound. | ||
Theorem 2.9 of Koehler and Sohn, for mu, nu (K, tau)-regular with the same covariance, D convex, relatively open and nonempty (not necessarily bounded), B >= 0 self-concordant on D, beta, gamma > 0 and Lref = (1 + 1/beta) (1 + inf_D (lambda |theta|^2 + gamma B theta)): if 36 <= n gamma, |E_(mu^n) Phi - E_(nu^n) Phi| <= 100 (1 + K^4)^2 Lref^2 ((n gamma)^(-1/2) + sqrt (dim V) (tau / lambda + 1) / n); Var_(mu^n) Phi <= 4 (1 + K^4)^2 Lref^2 / n; and Phi is within that bound plus t of E_(nu^n) Phi outside an event of mu^n-probability at most 4 (1 + K^4)^2 Lref^2 / (n t^2). | ||
| ridge_ | The comparison of ridge-regularized least squares values of Koehler and Sohn (proposition at lines 6365-6416), as a concentration inequality with the barrier weight gamma and the contraction eps left free: for mu, nu (K, tau)-regular with the same covariance, B >= 0 a theta-self-concordant barrier on D, theta_0 in D, t, M > 0 and 36 <= n gamma, the mu^n-probability that |Phi_Z(D; beta) - E_(nu^n) Phi_Z(D, B; beta, gamma)| exceeds the bound of (2.12) plus t + eps M + gamma B theta_0 + gamma theta ln (1 / eps) is at most 4 (1 + K^4)^2 Lref^2 / (n t^2) + (theta_0 Sigma theta_0^T / 2 + beta lambda |theta_0|^2) / M (Theorem 2.9, Lemma 2.11 and Markov's inequality for F_Z(theta_0, 0; beta), whose mean is the last numerator). | |
Phi_Z(D; beta) is the value with a barrier of weight 0, hence a measurable function of the features for D convex, relatively open and nonempty; Markov's inequality P(f > M) <= E f / M for f >= 0 integrable. | ||
| moreau.v | Projections in 'rV[R]_N with the Frobenius inner product: a nonempty closed set has a nearest point to every z (compactness of the closed bounded sets); for K closed, convex and nonempty, p in K is nearest to z iff <z - p, y - p> <= 0 for every y in K, the nearest point proj K z is unique, and |proj K z - proj K z'| <= |z - z'|. | |
Lemma 5.1 of Koehler and Sohn (Moreau's decomposition): for a closed convex cone C, z = proj C z + proj (polar C) z with the two parts orthogonal, so |z|^2 = |proj C z|^2 + |proj (polar C) z|^2; p in C is proj C z iff z - p is in the polar cone and orthogonal to p; and sup_(y in C, |y| <= 1) <z, y> = |proj C z|. | ||
| feature_ | Under a law on 'rV[R]_N with E <z, theta>^4 <= A |theta|^4 for every theta, a measurable g with |g z| <= a + b |z|^4 for every z (a bound on the absolute value) is integrable, so the linear forms, their products and the quadratic forms z T z^T are integrable and z T z^T is square integrable; if moreover E <z, theta> <z, theta'> = theta Sig theta'^T, then E z T z^T = <Sig^T, T>. | |
A finite mixture of laws P i on 'rV[R]_N, each centered with second-moment matrix Sig i, E_i <z, theta>^4 <= A |theta|^4 and Var_i (z T z^T) <= tau^2 |T|_F^2 for T symmetric, is a regular feature with covariance Sigma = sum_i w_i Sig i, any K with A <= K^4 opnorm Sigma^2, and sqrt (tau^2 + c^2) in place of tau, when opnorm Sigma > 0 and sum_i w_i <Sig i - Sigma, T>^2 <= c^2 |T|_F^2 for every symmetric T (the law of total variance, with E_i z T z^T - E z T z^T = <Sig i - Sigma, T> for T symmetric). | ||
The same when opnorm Sigma > 0 and sum_i w_i |Sig i - Sigma|_F^2 <= c^2, which implies the hypothesis of mixture_regular_feature_dot by Cauchy-Schwarz for the Frobenius product; no structure of Sig i - Sigma is needed. |