The sway bar is there to keep one side of the suspension from moving very far independently of the other side in relation to the chassis. Basically, they can move independently only as much as the bar can be twisted. On the road, this helps keep body roll to a minimum, improving handling.
Off road, you want exactly the opposite. You want each side of the suspension to move independently of the other as much as possible (except in side-hill type situations).
You'll find varying opinions on this board of the necessity of having the bar for road driving, but it's definitely good to have.
For off-road, however, there's no debate: take it off when you wheel.