CS代考程序代写 University of California, Los Angeles Department of Statistics

University of California, Los Angeles Department of Statistics
Instructor: Nicolas Christou
Exponential families
A probability density function or probability mass function is called an exponential family if it can be expressed as
􏰍k􏰎 f(x|θ) = h(x)c(θ)exp 􏰈 wi(θ)ti(x) .
i=1
Note: h(x), t1(x), . . . , tk(x) do not depend on θ and c(θ) does not depend of x.
Example:
Consider X ∼ b(n, p) with n fixed. Show that p(x) = 􏰒n􏰓px(1 − p)n−x can be expressed in
Statistics 100B
the exponential family form.
􏰍n􏰎
x px(1−p)n−x
􏰍n􏰎􏰍 p 􏰎x
x 1−p (1−p)n
p(x) = = = =
h(x)=􏰒n􏰓,c(p)=(1−p)n,t1(x)=x,w1(p)=log p . x 1−p
x
􏰍n􏰎 px
(1−p)nelog(1−p) 􏰍n􏰎 p
x x
(1 − p)nexlog( 1−p )
Therefore this pmf is an exponential family with
Theorem:
Suppose a random variable X has a pdf or pmf that can be expressed in the form of expo- nential family. Then,
􏰍k∂wi(θ)􏰎 ∂
(a) E 􏰈 ti(x) =− logc(θ).
and
i=1 ∂θj ∂θj 􏰍k ∂wi(θ) 􏰎 ∂2
(b) var 􏰈 i=1
􏰍k ∂2wi(θ) 􏰎 ti(x) =− logc(θ)−E 􏰈 ti(x) .
∂θj
Note: Here log is the natural logarithm.
∂θj2
i=1
∂θj2
1

Proof of (a):
x 􏰀􏰍k􏰎
h(x)c(θ)exp 􏰈 wi(θ)ti(x) i=1
dx
x
Differentiate both sides w.r.t. θj:
􏰀 ∂c(θ)􏰍k
􏰀
f (x|θ)dx
= 1 = 1
􏰎 exp 􏰈 wi(θ)ti(x) dx
h(x)
􏰀 k∂wi(θ) 􏰍k 􏰎
∂θj
+ h(x)c(θ) 􏰈 ti(x)exp 􏰈 wi(θ)ti(x) dx = 0
x
i=1
x i=1 ∂θj i=1
Multiply the first integral by c(θ) and note that ∂logc(θ) = ∂c(θ) 1 .
􏰀
∂c(θ) ∂θj
c(θ) ∂θj 􏰍 k 􏰎 c(θ)
ti(x)exp
∂θj c(θ)
dx = 0
􏰈 wi(θ)ti(x) dx i=1 c(θ)
exp
􏰀 k∂wi(θ) 􏰍k 􏰎
h(x)
+ h(x)c(θ) 􏰈
􏰈
􏰀 k ∂wi(θ)
x
x i=1 ∂θj After rearranging we get
􏰈 wi(θ)ti(x) i=1
ti(x)h(x)c(θ)exp
∂logc(θ)􏰀 􏰍k 􏰎
􏰍k
􏰎
􏰈 wi(θ)ti(x) dx =
x i=1
E
∂θj
− h(x)c(θ)exp
∂θj
􏰈 ti(x) = − i=1 ∂θj
i=1
logc(θ).
Or
∂θj x 􏰍k∂wi(θ)􏰎 ∂
􏰈 wi(θ)ti(x) dx i=1
To prove statement (b) of the theorem differentiate a second time and rearrange.
2