Music theory question

I don't know much music theory so please forgive me. But ya know when you see a song is in the key of C major for example. I will try to play a scale in C any type of scale in C, it clearly is not right. Sometimes it does but many times it does not. What am I missing here?