Expensive LEDs - why?

One contributing factor to the high cost of LEDs - they have to be kept relatively cool to preserve their long lifetimes. This means the "bulbs" require expensive metal heat sinks and ways to get the heat from the LED into the heat sink. There are lots of great innovations in this area, but it still adds a lot of cost.
A separate but related issue is that the efficacy (luminous efficiency) of the LED drops off when you turn the brightness up. There's a great article in the latest edition of the IEEE Spectrum magazine that starts from a nice high-level overview, then digs down to some very technical information.
What it boils down to is, we can have very bright LED lights, but they'll be less efficient than they could be, or very efficient LED lights, but they will be less bright (or more costly, assuming we just put more LEDs in each bulb/fixture).
Engineering is always about making the right tradeoffs - there's a lot of research going into LEDs right now, and over time we'll see the benefits.