Convex Functions

definition: a real valued operator $f$ is convex if for any $x,y$ and for any $\lambda \in (0,1)$ , $f((1-\lambda)x + \lambda y ) \le (1-\lambda)f(x) + \lambda f(y)$ .

lemma: if $f$ is convex and $x < y < z$ . Let $S(u,v)$ be the slope of the line segment with endpoints $(u,f(u)),(v,f(v))$ . Then $S(x,y)\le S(x,z) \le S(y,z)$ .

Since $x < y < z$ there is a $t \in (0,1)$ such that $y = x(1-t)+zt$ . Since $f(y) \le f(x)(1-t) + f(z)t$ it follows that $f(y)-f(x)\le (f(z)-f(x))t$ and so

$S(x,y) = \frac{f(y)-f(x)}{y-x} \le \frac{(f(z)-f(x))t}{(z-x)t} = S(x,z)$

Since $(f(z)-f(x))(1-t) \le f(y)-f(z)$ . it follows that

$S(x,z) = \frac{(f(z)-f(x))(1-t)}{(z-x)(1-t)} \le \frac{f(y)-f(z)}{y-z} = S(z,y)$

lemma: When $x < y < u < v$ then $S(x,y) \le S(u,v)$

from prior lemma, $S(x,y) \le S(x,u) \le S(y,u) \le S(y,v) \le S(u,v)$

lemma: $S(x,y)$ is non-decreasing in both parameters.

Fix $x$ . Let $y < u$ . Then $S(x,y) \le S(x,u)$

Fix $u$ . Let $x < y$ . Then $S(x,u) \le S(y,u)$

lemma: When $y < v$ , $f'(y-) \le f'(u+)$

Let $x < y < u < v$ . Since $S(x,y) \le S(y,u) \le S(u,v)$ and $S(x,y)$ is non-decreasing in $x$ it follows that $f(y-) = \lim_{x \uparrow y} S(x,y) \le S(y,u)$ . Since $S(u,v)$ is non-increasing in $y$ , and bounded below by $f(y-)$ it follows that $f'(y-) \le \lim_{v \downarrow y} S(y,v) = f'(y+)$

lemma: for any $c$ there is a line $y(x) = f(c) + k(x-c)$ such that $y(x) \le f(x)$ .

Choose $k \in [f'(c-),f'(c+)]$ . Consider $z(x) = f(x) - y(x) = f(x) - f(c) - k(x-c) = (S(x,c) - k)(x-c)$ . When $x > c$ , $S(x,c) \ge f(c+) \ge k$ and so $z(x) \ge 0$ . When $x < c$ , $S(x,c) \le f(c-) \le k$ and so $z(x) \ge 0$ .

lemma: (Jensen’s Inequality) when $f$ is convex, $f(\mathbb{E} X) \le \mathbb{E} f(X)$

In the prior lemma, choose $c = \mathbb{E}(X)$ . Then $\mathbb{E} y(X) \le \mathbb{E} f(X)$ and so $f(\mathbb{E}(X)) \le \mathbb{E} f(X)$ .

lemma: When $f$ is convex and strictly increasing, $\mathbb{E} X \le f^{-1} \mathbb{E} f(X)$ .

Since $f$ is strictly increasing so is $f^{-1}$ . Since $f(\mathbb{E} X) \le \mathbb{E} f(X)$ it follows that $\mathbb{E} X \le f^{-1} \mathbb{E} f(X)$

lemma: If $f$ is convex, non-decreasing and $g$ is convex, then $f \circ g$ is convex.

Let $f,g$ be convex functions such that $f$ is non-decreasing. Then $g(\lambda x + (1-\lambda) y) \le \lambda g(x) + (1-\lambda) g(y)$ and so $(f\circ g)( \lambda x + (1-\lambda) y) \le f(\lambda g(x) + (1-\lambda) g(y)) \le \lambda f(g(x)) + (1-\lambda) f(g(y))$