Why might a senator or congressman sit on a certain committee?

Because he might only have power over one part of the Committee.

The answer is pretty simple. A congressman would sit on a certain committee to have input on it. This way they could share there opinion and influence action on whatever the topic of that committee is.

For example if a congressman wants to try and chance foreign policy they would sit on the committee of foreign affairs. This wouldn't guarantee a change but it would guarantee a say.
 
This is very effective as it allows congressmen to focus  on whatever they have the most knowledge in. If all members had to vote on all issues certain things would be complicated.