Musically, the song is rather simple. The intro is the note F pulling off into the note E. The main riff uses power chords of the same notes and played in similar fashion. The chorus is the power chord F# played about seven times and then G played twice.
While the song comes across as a kind of standard-issue anti-authority rant, with Cobain complaining about being sent to "the office" by the titular cheese, that cheese was in fact a real person - Jonathan Poneman of Sub Pop records. "I was expressing all the pressures that I felt from him at the time because he was being so judgemental about what we were recording," Cobain told Michael Azerrad in Come As You Are.