Engineering Probability & Statistics Sharif University of Technology Hamid R. Rabiee & S. Abbas Hosseini November 21, 2015 CE 181 Date Due: Azar 14th , 1394 Homework 5 Problems 1. A univariate Gaussian (Normal) distribution with mean µ and variance σ 2 is defined as N (x|µ, σ 2 ) = √ (x−µ)2 1 e− 2σ2 2πσ Suppose we have N i.i.d random variables sampled from a Gaussian distribution N (x|µ, σ 2 ). Derive the distribution over the set of random variables. This distribution also defines an N dimensional random variable x, for which, dimensions are statistically independent. Describe why this equivalence holds. 2. Let x ∼ N (0, 1). Show that P (x > x + |x > x) ≈ e−x for large x and small . 3. The Exponential Family of distributions over x, given parameter vector η, is defined to be the set of distributions of the form p(x|η) = h(x)g(η) exp {η T u(x)} Express these list of distributions as members of the exponential family and derive expressions for η, u(x), h(x), and g(η). (a) Gamma distribution Gam(λ|a, b) = 1 a a−1 b λ exp(−bλ) Γ(a) (b) Multi-variate Gaussian distribution N (x|µ, Σ) = 1 (2π)N/2 |Σ|1/2 exp n o 1 − (x − µ)T Σ−1 (x − µ) 2 4. Suppose that X1 , . . . , Xn form a random sample from a normal distribution for which the value of the mean θ is unknown and the value of the variance σ 2 > 0 is known. Suppose also that the prior distribution of θ is the normal distribution N (θ|µ0 , v02 ). Find the hyperparameters of the posterior distribution of θ given given that Xi = xi (i = 1, . . . , n). 5. Suppose that observations are to be taken at random from the normal distribution N (X|θ, 1), and that θ is unknown. Assume that θ ∼ N (θ|µ, 4). Also, observations are to be taken until the variance of the posterior distribution of θ has been reduced to the value 0.01 or less. Determine the number of observations that must be taken before the sampling process is stopped. 1 6. Suppose that X1 , . . . , Xn form a random sample from a distribution for which the p.d.f. f (x|θ) is as follows: θxθ−1 for 0 < x < 1 f (x|θ) = 0 otherwise. Suppose also that the value of the parameter θ is unknown (θ > 0), and the prior distribution of θ is the gamma distribution with parameters α and β (α > 0 and β > 0). Determine the mean and the variance of the posterior distribution of θ. 7. we have two independent RV’s X and Y which both are exponential with same parameter λ if we define Z = X + Y find its Probability Density Function(PDF) 8. Prove the following identity. E [y] = E [E [x|y]] Suppose that N is a counting random variable, with values {0, 1, · · · , n}, and that given (N = k), for k ≥ 1, there are defined random variables X1 , · · · , Xn such that E (Xj |N = k) = µ (1 ≤ j ≤ k) Define a random variable SN by SN = X1 + X2 + · · · + Xk 0 Show that E(SN ) = µE(N ). 2 if(N − k), 1 ≤ k ≤ n if(N = 0)
© Copyright 2025 Paperzz