As a general rule - with a correctly adjusted clutch, the throw out bearing will not make noise when the clutch pedal is released (even if the bearing is defective) - the throw out bearing should rotate only when the clutch pedal is depressed, at all other times it should not be in contact with the rotating clutch pressure plate.
On the other hand - if the clutch free play is not correctly adjusted, it is possible for the throw out bearing to be in contact with the fingers on the pressure plate, and it will make a noise that goes away when the clutch is depressed.
So as a general rule - if the noise is present when the clutch is depressed, then it is the throw out bearing, if the noise is present when the clutch is released, that rules out the the throw out bearing. If the noise is present with the vehicle stationary in neutral, clutch released, it'll typically be a transmission noise and as locjaw has said - if it's present in every gear except fourth, it's probably the transmission bearings.