Blog Profile / The Endeavour


URL :http://www.johndcook.com/blog/
Filed Under:Academics
Posts on Regator:1065
Posts / Week:5.2
Archived Since:April 26, 2011

Blog Post Archive

Replace data with measurements

To tell whether a statement about data is over-hyped, see whether it retains its meaning if you replace data with measurements. So a request like “Please send me the data from your experiment” becomes “Please send me the measurements from your experiment.” Same thing. But rousing statements about the power of data become banal or even […]

Clinical trials and machine learning

14 hours agoAcademics : The Endeavour

Arguments over the difference between statistics and machine learning are often pointless. There is a huge overlap between the two approaches to analyzing data, sometimes obscured by differences in vocabulary. However, there is one distinction that is helpful. Show More Summary

Fitting a triangular distribution

Sometimes you only need a rough fit to some data and a triangular distribution will do. As the name implies, this is a distribution whose density function graph is a triangle. The triangle is determined by its base, running between points a and b, and a point c somewhere in between where the altitude intersects the base. […]

A subtle way to over-fit

If you train a model on a set of data, it should fit that data well. The hope, however, is that it will fit a new set of data well. So in machine learning and statistics, people split their data into two parts. They train the model on one half, and see how well it […]

Mathematical arbitrage

I suspect there’s a huge opportunity in moving mathematics from the pure column to the applied column. There may be a lot of useful math that never sees application because the experts are unconcerned with or unaware of applications. In particular I wonder what applications there may be of number theory, especially analytic number theory. […]

Mathematical modeling in Milton

In Book VIII of Paradise Lost, the angel Raphael tells Adam what difficulties men will have with astronomy: Hereafter, when they come to model heaven And calculate the stars: how they will wield the The mighty frame, how build, unbuild, contrive To save appearances, how gird the sphere With centric and eccentric scribbled o’er, Cycle […]

Partitioning natural numbers with pi

Every positive integer is either part of the sequence ? n? ? or the sequence ? n?/(? – 1) ? where n ranges over positive integers, and no positive integer is in both sequences. This is a special case of Beatty’s theorem.

Extremely small probabilities

One objection to modeling adult heights with a normal distribution is that the former is obviously positive but the latter can be negative. However, by this model negative heights are astronomically unlikely. I’ll explain below how one can take “astronomically” literally in this context. A common model says that men’s and women’s heights are normally […]

Atavachron

In the Star Trek episode “All Our Yesterdays” the people of the planet Sarpeidon have escaped into their past because their sun is about to become a supernova. They did this via a time machine called the Atavachron. One detail of the episode has stuck with me since I first saw it many years ago: although people can go back […]

Why isn’t everything normally distributed?

Adult heights follow a Gaussian, a.k.a. normal, distribution [1]. The usual explanation is that many factors go into determining one’s height, and the net effect of many separate causes is approximately normal because of the central limit theorem. If that’s the case, why aren’t more phenomena normally distributed? Someone asked me this morning specifically about […]

Machine learning and magic

When I first heard about a lie detector as a child, I was puzzled. How could a machine detect lies? If it could, why couldn’t you use it to predict the future? For example, you could say “IBM stock will go up tomorrow” and let the machine tell you whether you’re lying. Of course lie […]

Quaternions in Paradise Lost

Last night I checked a few books out from a library. One was Milton’s Paradise Lost and another was Kuipers’ Quaternions and Rotation Sequences. I didn’t expect any connection between these two books, but there is one. The following lines from Book V of Paradise Lost, starting at line 180, are quoted in Kuipers’ book: Air […]

Technical notes

For the last fifteen Wednesdays I’ve been posting links to technical notes. This is the end of the series. You can find most of the links from previous Wednesday posts on one page by going to technical notes from the navigation menu at the top of the site.

Graphemes

Here’s something amusing I ran across in the glossary of Programming Perl: grapheme A graphene is an allotrope of carbon arranged in a hexagonal crystal lattice one atom thick. Grapheme, or more fully, a grapheme cluster string is a single user-visible character, which in turn may be several characters (codepoints) long. For example … a “?” […]

Too easy

When people sneer at a technology for being too easy to use, it’s worth trying out. If the only criticism is that something is too easy or “OK for beginners” then maybe it’s a threat to people who invested a lot of work learning to do things the old way. The problem with the “OK […]

Clinical trial software

This week’s resource post lists some of the projects I managed or contributed to while working at MD Anderson Cancer Center. CRMSimulator is used to design CRM trials, dose-finding based only on toxicity outcomes. BMA-CRMSimulator is a variation on CRMSimulator using Bayesian model averaging. Show More Summary

Finding the best dose

In a dose-finding clinical trial, you have a small number of doses to test, and you hope find the one with the best response. Here “best” may mean most effective, least toxic, closest to a target toxicity, some combination of criteria, etc. Since your goal is to find the best dose, it seems natural to compare dose-finding […]

Career advice from Einstein

“If I would be a young man again and had to decide how to make my living, I would not try to become a scientist or scholar or teacher. I would rather choose to be a plumber or a peddler, in the hope to find that modest degree of independence still available under present circumstances.” […]

The opposite of an idiot

The origin of the word idiot is “one’s own,” the same root as idiom. So originally an idiot was someone in his own world, someone who takes no outside input. The historical meaning carries over to some degree: When you see a smart person do something idiotic, it’s usually because he’s acting alone. The opposite of […]

Copyright © 2015 Regator, LLC