My Math and Writing scores are in the 700s, but I'm stuck at a 580 on Critical Reading. I've already taken the test three times, so I'm not going to take it again. I have scores in the high 700s on my subject tests in Math and Physics, and I'd say my application is very good overall other than this one score (awards, extracurriculars, and recommendations are all solid). Am I crazy to be looking at super competitive schools (i.e. Harvard, Princeton, MIT)?
Your Critical Reading score is far below average for the schools that interest you, so I think your three options are to try taking the ACT instead, really impress admissions with something else in your application, or aim for less competitive colleges.
You may find the ACT Reading and English sections easier than the Critical Reading section of the SAT. Usually, math-oriented people are more suited to the ACT, so it's a viable option if you have time. Take practice tests to see where your scores fall, and send your ACT scores instead of your SAT scores if you can get them up into the 33-36 range.
If the ACT isn't an option, it's possible that you'll be admitted on the basis of another impressive accomplishment included in your application. I wouldn't count on this - you should only consider it if you've done something extremely unique that you know will stand out even at these highly selective institutions. This article gives more details on what I mean.
Lastly, you could decide to just go for less competitive colleges. I don't want to push this option because it sounds like you're more than qualified in other aspects of your application, but you also shouldn't dismiss schools that have less name recognition. Your scores are in the appropriate range for a lot of excellent schools, so try not to look at places like Harvard and MIT as the only acceptable options.