Maximum Likelihood Estimation

General

• Obtain an estimate for an unknown parameter theta using the data that we obtained from our sample.
• Choose a value of theta that maximizes the likelihood of getting the data we observed.
• Joint probability mass function
  • If the observations are independent you can just multiply the PDFs of the individual observations.
  • $L(\theta )=\prod _{i=1}^{n}f(x_{i};\theta )$ (General formulation)
  • $f(x_{i};p)=p^{x_{i}}(1-p)^{1-x_{i}}$ for xi = 0 or 1 and 0 < p < 1.
    • If the Xi are independent Bernoulli random variables with unknown parameter p, replace the general notation with the bernoulli notation:
  • $L(p)=p^{\sum x_{i}}(1-p)^{n-\sum x_{i}}$
  • $logL(p)=(\sum x_{i})log(p)+(n-\sum x_{i})log(1-p)$