Wednesday, March 26, 2014
Marion's intro to the eclipsing binary lab
Academy Award nominations for best cinematography welcomed!
Monday, March 17, 2014
Dealing with outliers in the sunspot rotation period data
Some of you may have noticed that a few sunspot rotation period values differ a lot from the rest---they are outliers. How do we deal with them more systematically than just deciding to ignore them?
Here's a simple way. Compute the average of the data using all of it, and also the standard deviation \(\sigma\), given by
$$\sigma=\sqrt{\frac{1}{N}\bigg( \sum_{x\in data} x^2-mean^2}\bigg),$$
where the sum is over all points x in the dataset and there are N points. You now have a quantitative measure of how much the outliers outlie: for each datapoint, subtract the mean from it and then divide by \(\sigma\). Outliers will have the highest values of this quantity (it's called a "z-score"). You can now compute a new mean for your average weighting the data points by one over this factor squared: that way the outliers will contribute less to the mean. To compute the new mean, do the sum
$$<x>_{\rm corrected}=\bigg[\frac{1}{N}\sum_{x\in data}\frac{x^2}{z^2(x)}\times \sum_{x\in data} \frac{1}{z^2(x)}\bigg]^{1/2}.$$
The second term above is so that you don't end up multiplying the mean by some overall factor just because you've introduced a weighting; weighting by the inverse square of the z-score turns out to be better than the naive thing you might do, weighting by just the inverse of the z-score. The idea here is just this: you want points that are outliers, which will have higher z-scores, to contribute less to the corrected average. Imagine a point extremely far from the mean: its z score would be so large that it would contribute very little to the corrected average.
This technique should give you a more accurate estimate of the mean than just including the outliers. It won't necessarily be more accurate than ignoring the outliers altogether, but it is more objective in that you don't need to make the subjective decision as to which points are outliers. Indeed, one thing to try might be doing the mean this way, and then doing it by removing any points altogether that have a z-score larger than, say, 2.5. Compare results and see which you think is better!
Thanks to Scott Zhuge and Louise Decoppet for catching some mistakes!
Here's a simple way. Compute the average of the data using all of it, and also the standard deviation \(\sigma\), given by
$$\sigma=\sqrt{\frac{1}{N}\bigg( \sum_{x\in data} x^2-mean^2}\bigg),$$
where the sum is over all points x in the dataset and there are N points. You now have a quantitative measure of how much the outliers outlie: for each datapoint, subtract the mean from it and then divide by \(\sigma\). Outliers will have the highest values of this quantity (it's called a "z-score"). You can now compute a new mean for your average weighting the data points by one over this factor squared: that way the outliers will contribute less to the mean. To compute the new mean, do the sum
$$<x>_{\rm corrected}=\bigg[\frac{1}{N}\sum_{x\in data}\frac{x^2}{z^2(x)}\times \sum_{x\in data} \frac{1}{z^2(x)}\bigg]^{1/2}.$$
The second term above is so that you don't end up multiplying the mean by some overall factor just because you've introduced a weighting; weighting by the inverse square of the z-score turns out to be better than the naive thing you might do, weighting by just the inverse of the z-score. The idea here is just this: you want points that are outliers, which will have higher z-scores, to contribute less to the corrected average. Imagine a point extremely far from the mean: its z score would be so large that it would contribute very little to the corrected average.
This technique should give you a more accurate estimate of the mean than just including the outliers. It won't necessarily be more accurate than ignoring the outliers altogether, but it is more objective in that you don't need to make the subjective decision as to which points are outliers. Indeed, one thing to try might be doing the mean this way, and then doing it by removing any points altogether that have a z-score larger than, say, 2.5. Compare results and see which you think is better!
Thanks to Scott Zhuge and Louise Decoppet for catching some mistakes!
Improving the estimate of the Sun's rotational speed
To estimate the Sun's rotational speed in the AU lab, we took two points on the Sun's edges and used them to estimate the Sun's rotation speed. See the picture!
In particular, we took the two points with the greatest difference in speed and assumed they were on a line perpendicular to the Sun's rotation axis, and hence gave a good estimate of the rotation speed. This is because, for a solid body rotating at some angular frequency \(\omega\), the rotation speed of a point on the surface is
$$v=r\omega,$$
with r as shown on the picture below.
Since we are not accounting for any angle dependence in our computation (i.e., we used \(r=R_{Sun}\)), we have taken it that \(\gamma=90^{\circ}\). But in reality it may not be. Using two pairs of points we can determine its actual value, and thence correct the rotation velocity estimate. For the pair of points c and d, we have the velocity difference \(\Delta v\), which we define as \(\Delta v_{ij}=v_i-v_j\), to be
$$\Delta v_{cd}=2r\omega=2R_{Sun}\omega\sin\gamma,$$
while for the pair of points e and f we have
$$\Delta v_{ef}=2x\omega=2R_{Sun}\omega\cos\gamma.$$
Note that \(x=R_{Sun}\sin(90^{\circ}-\gamma)=R_{Sun}\cos\gamma\); you should convince yourself that the angles are really as I've drawn them. In so doing, you may want to remember that vertical angles are equal.
With our two equations for two different \(\Delta v\)s, we can solve for \(\gamma\) as
$$\gamma=\tan^{-1}\bigg[\frac{\Delta v_{cd}}{\Delta v_{ef}}\bigg].$$
The true rotation velocity difference is then just
$$\Delta v_{true}=\Delta v_{cd}\frac{1}{\sin\gamma}=\Delta v_{cd}\bigg[\frac{\sqrt{\Delta v_{cd}^2+\Delta v_{ef}^2}}{\Delta v_{cd}}\bigg]=\sqrt{\Delta v_{cd}^2+\Delta v_{ef}^2}.$$
Note the true velocity difference is larger than or equal to our uncorrected result, since \(\sin \gamma \leq 1\); the last result also clearly shows this! To get to the last result, I used the identity that
$$\sin\bigg[ \tan^{-1}\left(\frac{X}{Y}\right)\bigg]=\frac{X}{\sqrt{X^2+Y^2}}.$$
In particular, we took the two points with the greatest difference in speed and assumed they were on a line perpendicular to the Sun's rotation axis, and hence gave a good estimate of the rotation speed. This is because, for a solid body rotating at some angular frequency \(\omega\), the rotation speed of a point on the surface is
$$v=r\omega,$$
with r as shown on the picture below.
Since we are not accounting for any angle dependence in our computation (i.e., we used \(r=R_{Sun}\)), we have taken it that \(\gamma=90^{\circ}\). But in reality it may not be. Using two pairs of points we can determine its actual value, and thence correct the rotation velocity estimate. For the pair of points c and d, we have the velocity difference \(\Delta v\), which we define as \(\Delta v_{ij}=v_i-v_j\), to be
$$\Delta v_{cd}=2r\omega=2R_{Sun}\omega\sin\gamma,$$
while for the pair of points e and f we have
$$\Delta v_{ef}=2x\omega=2R_{Sun}\omega\cos\gamma.$$
Note that \(x=R_{Sun}\sin(90^{\circ}-\gamma)=R_{Sun}\cos\gamma\); you should convince yourself that the angles are really as I've drawn them. In so doing, you may want to remember that vertical angles are equal.
With our two equations for two different \(\Delta v\)s, we can solve for \(\gamma\) as
$$\gamma=\tan^{-1}\bigg[\frac{\Delta v_{cd}}{\Delta v_{ef}}\bigg].$$
The true rotation velocity difference is then just
$$\Delta v_{true}=\Delta v_{cd}\frac{1}{\sin\gamma}=\Delta v_{cd}\bigg[\frac{\sqrt{\Delta v_{cd}^2+\Delta v_{ef}^2}}{\Delta v_{cd}}\bigg]=\sqrt{\Delta v_{cd}^2+\Delta v_{ef}^2}.$$
Note the true velocity difference is larger than or equal to our uncorrected result, since \(\sin \gamma \leq 1\); the last result also clearly shows this! To get to the last result, I used the identity that
$$\sin\bigg[ \tan^{-1}\left(\frac{X}{Y}\right)\bigg]=\frac{X}{\sqrt{X^2+Y^2}}.$$
A note on Taylor series and the Wien law
Just a brief note with a comment on a mistake I saw in a number of blog posts on deriving the Wien displacement law from the blackbody distribution. Many of you used a Taylor series for \(e^u-1\) to solve the transcendental equation
$$5=\frac{ue^u}{e^u-1}$$.
You argued that by Taylor series, \(e^u-1\approx (1+u)-1=u\) so that the equation becomes, expanding both top and bottom,
$$5=\frac{u(1+u)}{(1+u-1)}=1+u\to u=4.$$
However, this isn't right. A Taylor series is only valid inside the "radius of convergence": fancy language for "small displacements from the point you are expanding about." This should be no surprise: for any function, if you approximate it by a line, that will only be a good approximation locally (unless the function is just a line!)
More formally, a Taylor series says
$$f(x+\Delta x)=\sum_{n=0}^{\infty} \frac{f^{(n)}(x)}{n!}(\Delta x)^n$$
and the expansion for \(e^u-1\) above takes only the n=0 and n=1 terms. But consider \(\Delta x >1\): then the powers of \(\Delta x\) above grow with n, and the higher n terms contribute more. In short, the series does not converge: you have gone outside the radius of convergence. This is all to say that you cannot use a Taylor expansion of \(e^u-1\) about u=0 for a value of u near 4!
Indeed, the correct logic was to recongnize that for u larger than, say, 2, \(e^u\gg 1\) so the denominator can be written as approximately \(e^u\), canceled with part of the numerator, and the correct answer,
$$u=5,$$
determined.
$$5=\frac{ue^u}{e^u-1}$$.
You argued that by Taylor series, \(e^u-1\approx (1+u)-1=u\) so that the equation becomes, expanding both top and bottom,
$$5=\frac{u(1+u)}{(1+u-1)}=1+u\to u=4.$$
However, this isn't right. A Taylor series is only valid inside the "radius of convergence": fancy language for "small displacements from the point you are expanding about." This should be no surprise: for any function, if you approximate it by a line, that will only be a good approximation locally (unless the function is just a line!)
More formally, a Taylor series says
$$f(x+\Delta x)=\sum_{n=0}^{\infty} \frac{f^{(n)}(x)}{n!}(\Delta x)^n$$
and the expansion for \(e^u-1\) above takes only the n=0 and n=1 terms. But consider \(\Delta x >1\): then the powers of \(\Delta x\) above grow with n, and the higher n terms contribute more. In short, the series does not converge: you have gone outside the radius of convergence. This is all to say that you cannot use a Taylor expansion of \(e^u-1\) about u=0 for a value of u near 4!
Indeed, the correct logic was to recongnize that for u larger than, say, 2, \(e^u\gg 1\) so the denominator can be written as approximately \(e^u\), canceled with part of the numerator, and the correct answer,
$$u=5,$$
determined.
Sunday, March 16, 2014
Some latex tips for blogging
Hi, everyone!
As you work on your blog writeups for the AU lab, here are some handy latex tips based on minor pointers I've been giving out as I've been reading blogs.
To get matching-ly sized parentheses, use \left( \right). These commands demand a matched set; if you don't want to worry about this, or for some reason need just one big parenthesis, use \big( or \bigg(.
Put negative signs outside fractions, as \(-\frac{1}{2}\) looks better than \(\frac{-1}{2}\).
Make sure all superscripts and subscripts with more than one digit or symbol are inside a curly brackets: this makes the difference between
\(2^10\) and \(2^{10}\), which can be a significant one!
Use the symbols for Sun and Earth: for the Sun, subscript \odot, and for the Earth subscript \oplus, which show up as (e.g. for the masses):
$$M_{\odot}$$
and
$$M_{\oplus}.$$
You may want to use tables to display data: this can be a tiny bit of a pain, so here is a super-cool internet tool to make it easy!
http://www.tablesgenerator.com
You make the table how you want it, and the site generates the needed latex code.
As you work on your blog writeups for the AU lab, here are some handy latex tips based on minor pointers I've been giving out as I've been reading blogs.
To get matching-ly sized parentheses, use \left( \right). These commands demand a matched set; if you don't want to worry about this, or for some reason need just one big parenthesis, use \big( or \bigg(.
Put negative signs outside fractions, as \(-\frac{1}{2}\) looks better than \(\frac{-1}{2}\).
Make sure all superscripts and subscripts with more than one digit or symbol are inside a curly brackets: this makes the difference between
\(2^10\) and \(2^{10}\), which can be a significant one!
Use the symbols for Sun and Earth: for the Sun, subscript \odot, and for the Earth subscript \oplus, which show up as (e.g. for the masses):
$$M_{\odot}$$
and
$$M_{\oplus}.$$
You may want to use tables to display data: this can be a tiny bit of a pain, so here is a super-cool internet tool to make it easy!
http://www.tablesgenerator.com
You make the table how you want it, and the site generates the needed latex code.
Subscribe to:
Posts (Atom)