The obvious answer is Amoeba, but it really is the best. It's even better than the other two Amoebas in the Bay Area. If you got time to spare, you WILL find awesome stuff in their dollar bins. I also like Freakbeat in Sherman Oaks. There's a little vinyl shop that just opened right next door to the Echo on Sunset, but I think it's overpriced.
Amoeba blows. Headline Records on Melrose is pretty great.
For punk I go to Headline and I did work there in 02.Freakbeat is a great store also. But now a days it's getting harder to find good underground R 'n' R vinyl in store so I have been buying more on line.bo!