I have published some general feedback on the first assignment here: http://www-cse.ucsd.edu/users/elkan/291/feedback1.html

Assignment 2 is now available at http://www-cse.ucsd.edu/users/elkan/291/assignment2.html.

I've made it shorter, but there are two optional extra-credit problems.

Methodology is important, but so is conciseness. You should describe how graphs are generated mathematically, but don't give the
implementation details. This doesn't mean that you can be careless with the implementation; indeed you have to be even more careful because there is no way for the reader to help you by catching implementation errors.

Charles


QT - Eric Wiewiora wrote:
>
>
< replied-to message removed by QT >

Today before midnight will be ok. Charles




On Wed, 16 Oct 2002, QT - Cristian Estan wrote:

>
< replied-to message removed by QT >

What is de deadline for no penalty homework submission, today in class or today by the end of the day?

On a lighter note,

How important is methodology to the experimental part of the assignment. I think we've all seen examples of how wrong things can go when we estimate pdfs from data. Should we explicitly explain how we've created our pdf graphs, or can the details be brushed under the carpet?

Alex,
The problem definition says that t_1 is sufficient for \theta_1 when \theta_2 is known. Once theta_2 is known, it is a constant like any other and can be used as part of a statistic. Once theta_2 is known the distribution reduces to a single parameter distribution depending only on theta_1. As long as I define my statistic after the distribution is given to me, all is well (or so I think). If I were to define the statistic without the knowledge of \theta2, that is when the problem starts.

Sameer, I think there is a problem in your statement as far as the computation of t_1 goes. t_1(x) is clearly a function of the sample and only the sample, whereas you include mu as a parameter. I believe the problem definition states that t1 is sufficient for theta1 given theta2, ie P_theta1(t_1(x), theta2) = P_theta2(x, theta2). So theta2 may not be used in computing t_1(x), just for establishing its sufficiency.

Edited by author 10-14-2002 09:59 PM
hi,
I am at my wit's end about 2.6 and now I think I have found a counterexample. I would appreciate if someone could pointout where I am wrong.

consider the normal N(\mu,\sigma^2)

when \mu is known a sufficent statistic for \sigma is
t_1 = \sum_i (x_i -\mu)^2

and when \sigma is known, a sufficent statistic for \mu is

t_2 = \sum_i x_i

the pair (t_1,t_2), depends on \mu and I see no way of eliminating it, once we are given expressions for t_1, and t_2, since the problem does not place any restrictions on the form of the sufficent statistics, (t_1,t_2) should now be a sufficent statistic for (\sigma^2,\mu), which is not true.

what am I missing ?

> 1. Can we have tables instead of figures for the matlab results?

Yes, but in a paper figures are usually more impressive and easier for a reader to grasp quickly.

In many, many papers, a little arithmetic on the numbers in tables shows that they are inconsistent and there is some mistake somewhere. Don't let this be the case with your figures!


> 2. Do we need to do matlab validation for all problems? For
> example I cannot imagine a feasible way to validate problems 1.3
> and 2.6.

To check your result for 1.3, generate many vectors (x1 ... xn). Select those that have SUM xi within some narrow range. Now plot the
distribution of x1 from the selected vectors. Alternatively, plot the mean and variance of x1 as a function of SUM xi.

For 2.6, I agree, I can't think of a numerical validation immediately.

> 3. Do we have to do goodness of fit validation for e.g. problem
> 1.1 (that would validate how well the numerical results match
> the solution)?

No, intuitively plausible validations are sufficient. You don't need to do a whole second level of statistics, to check goodness of fit.

In general, people get too stuck on cookbook recipes for significance testing. This is important, but it is even more important to gain real insight and to focus on results that have practical significance, which usually means results that are far stronger than merely statistically significant. Also, significance testing is always based on assumptions that are usually not precisely true. So p-values, confidence intervals etc. are usually not numerically correct.

1. Can we have tables instead of figures for the matlab results?

2. Do we need to do matlab validation for all problems? For example I cannot imagine a feasible way to validate problems 1.3 and 2.6.

3. Do we have to do goodness of fit validation for e.g. problem 1.1 (that would validate how well the numerical results match the solution)?

(a,b)==(c,d) <==> a==c && b==d.

Keeping this in mind, try constructing a function whose equivalence class defines a partition that will work for the problem.

I did 2.6 use the factorization theorem. Since it gives a sufficient and necessary condition, it at least has no harm using it.

Does anyone have any hints on how to start on 2.6? I can't figure out a way to combine what t1 tells us about theta1 with what t2 tells us about theta2. Is the factorization theorem helpful for this? I thought maybe I could do something with Taylor expansions or something, if I were to use the factorization theorem. Any suggestions?

Thanks!

You have a lot of latitude to assume standard results. But do state them precisely and cite a source. I want you to approach these assignments as if each question was a piece of a research paper. Write it up well, cite your sources, contribute something clear and definite (i.e. the result you are asked to show) but don't reinvent the wheel and don't get stuck on any single small issue.

The Matlab numerical examples are important. So far, they are the computational, modern component of this class. The lesson to learn is how to use computation to advance understanding, to confirm symbolic results and to provide new insights that can be the springboard for further mathematical thinking.

For each problem, what you should put in the writeup is one or two small figures, and a short explanation that makes clear how the figure was generated (at the mathematical level, not the implementation aspects) and what conclusions can be drawn from the results shown in the figure. Again, the style should be similar to what you might write in the "results" section of a research paper.

I am not sure about just how much latitude we have in terms of assuming what is well known and how much detail is required in the proofs. If we are allowed to use a result like the uniqueness of the moment generating function, that makes some of the questions trivially easy.

In my case while proving that the sample variance of a normal distribution is chisquared, I need two results one which states, that the square of a standard normal is chisquared with one degree of freedom and the other about the sum of two chi squared variables.

Do I assume them as standard results or prove them as lemmas ?

How important is it to include the numerical examples from Matlab?

I have heard from several people who are working in study groups that the groups are making good progress on the first assignment. I really urge everyone not in a study group to form one!

According to "Introduction to Mathematical Statistics and Its Applications" by Richard J. Larsen and Morris L. Marx, on page 241, theorem 3.16.2:

Suppose W_1 and W2 are random variables for which mgf(W_1)=mgf(W_2) for some interval of t's containing 0. Then the probability distributions if the two random variables are identical.

Since these conditions hold for 1.1 a, it's safe to apply mgfs.

Here is a reference for a proof that the sum of two Gaussians is Gaussian that does not use MGFs: http://www-stat.stanford.edu/~susan/courses/s116/node114.html

Thera are quite a few mathematical results that are well-known and widely used, but whose proofs are not well-known, and/or difficult. It's ok to assume a well-known result when proving something else, but it's still good to know how it can be proved.

I don't want everyone to get stuck on this first problem. Please move on to the others and work on the MVUE problems!

Edited by author 10-11-2002 12:48 PM
I'm not sure that the MGF proof is not as simple as you wrote. You show that MGF(Gaussian) ---> some form. Then you show that
MGF(Gaussian+Gaussian) ---> same basic form. But how do you know that there is no other distribution F such that MGF(F) ---> form for a Gaussian?

Another way to prove the homework (I think) is if you first prove a couple of things about normals:
  Let: X ~ N(\mu_1, \sigma_1^2)
       Y ~ N(\mu_2, \sigma_2^2)
  Then: X+Y ~ N(\mu_1+\mu_2, \sigma_1^2+\sigma_2^2)
  Also: cX ~ N(c\mu_1, c^2\sigma_1^2)
I think the first can be proved through the "convolution theorem" which is relatively easy to prove with a bit of calculus, and the second claim can also be proved fairly quickly. (The convolution thm can be found in Grimmet & Stirzaker ("Probability and Random Processes").)

If it is a well-known result that the sum of two normal distributions is a normal distribution, and (in my opinion) lesser known results about moment generating functions, then why is it a better proof to use the moment generating function theorems than just to assume that the sum of two normally distributed r.v.'s is normally distributed?

The proof is rather simple. If you compute the moment generating function for both the normal distributions and multiply them, you'll see that the product is the mgf for a gaussian. Since the mgf(x1+x2) = mgf(x1)*mgf(x2) a little number crunching will prove the result.

I was wondering about how detailed our proofs have to be. Where should we draw the line as to what to simply cite and what to actually prove?

Edited by author 10-10-2002 09:27 PM
I don't immediately know how to prove that the sum of two Gaussian random variables is Gaussian. However this result is well-known :-) so there must be proofs of it published.

For this problem (and all others) you may use any and all published references. You should cite any references you use, and if you use an existing proof, you should write it up in your own words, as part of mastering it.

Eric, you're right. You need more than the mean and the variance of the $\overline{x}$. I know of no easy way to get there other than using moment generating functions.

For the first problem, I was wondering what is necesarry for proving that the distribution if the means is normal.

It is simple enough showing that the mean and variance is as stated, but I have a nagging suspicion there is more to the problem...

Please ask questions about the first assignment (due October 14) here.

If you know the answer to a question, or part of the answer, or where to find the answer, please do post a reply!