I'm much more a fan of vibrant colors than muted tones, but as other people have said, you need to make the decision based on where you want to go with the storyline. Some thoughts:
- If you're going to go with muted tones, please do not go the Rubicon route. One thing I felt about Rubicon is that, despite how realistic it was (i.e. it makes sense that all the rocks are roughly the same color), it's a very depressing experience. If you want your planet to feel desolate and inhospitable, that's a good step. Gameplay-wise, I find it to be a detraction, since when everything is the same color, it's hard to find your way around places, and it's just not much fun.
- On the flip side, if you give everything a ton of color, you run the risk of either being excessively vivid (Portal of Sigma) or, well, ugly (Eternal Mark V). Marathon 2 did it very well, in that the colors were bright enough to add a lot of depth to the environment, but without being an assault on the eyes.
- It's best either way to have plenty of textures that are just lighter and darker to begin with. That way, you can end up with bright textures when you want a room to be more friendly, and darker textures when you're exploring a tomb or something. Even in the same textureset; I've made a handful of levels that go from a base or something like that to the catacombs underneath. In that case, I would want to have a mixture of both.
