## Lecture 18: Chernoff Bounds

The comment at the end about the moment bounds being at least as good as the Chernoff-type bounds is actually a really simple idea. Say we’re working with non-negative random variables. Recall the Chernoff uses something like

$\displaystyle \Pr[ X \geq \lambda ] \leq \inf_{t \geq 0} \frac{ \mathop{\mathbb E}[e^{tX}] }{ e^{t\lambda} }$

We could have instead used the “best” moment bound

$\displaystyle \Pr[ X \geq \lambda ] \leq \min_{k \geq 0} \frac{ \mathop{\mathbb E}[X^k] }{ \lambda^k }$

The claim (from this 1995 paper by Philips and Nelson) is

$\displaystyle \inf_{t \geq 0} \frac{ \mathop{\mathbb E}[e^{tX}] }{ e^{t\lambda} } \geq \min_{k \geq 0} \frac{ \mathop{\mathbb E}[X^k] }{ \lambda^k }$

This means whatever bound we could prove by taking the moment-generating function (on the left), it will never be better than the best moment bound.

Why is the claim true? Suppose the RHS minimum is ${\alpha}$. Then ${\mathop{\mathbb E}[X^k] \geq \alpha \lambda^k}$, for all ${k \geq 0}$. And hence, for any ${t > 0}$,

$\displaystyle \begin{array}{rl} \mathop{\mathbb E}[e^{tX}] &= \sum_{k \geq 0} \frac{\mathop{\mathbb E}[ (tX)^k ]}{k!} \\ &= \sum_{k \geq 0} \frac{t^k \mathop{\mathbb E}[ X^k ]}{k!} \\ &\geq \sum_{k \geq 0} \frac{ t^k (\alpha \lambda^k)}{k!} = \alpha e^{t\lambda}. \end{array}$

Which means the LHS infimum is also at least ${\alpha}$. Easy! The paper extends this to general random variables (using a simple conditioning trick), and also gives other results, including examples where the moment bound can be much better.