Case for the Sami
A Sami is a great first vehicle if you're a "tinkerer" and like to fix / modify things yourself (with a vehicle this old, you'll almost always be fixing or modifying something). It's easier than the Tracker / Sidekick (Trackick) to work on. It's also severely underpowered for freeway use, and you're NOT going to win any races EVER in it. It's cargo space is SEVERELY limited. If you're happy driving slow, knowing you're going to kick everyone's ass off road - this is the vehicle for you.
With your budget, what I'd spend my $$$ on:
* If you're more interested in looks than function, get a lift and 31 / 32" tires. You'll loose about 5 MPG, fry your clutch quicker, increase your chance of breaking things on your drive line, and go 50 MPH uphill on the freeway. But, you'll look really cool in your lifted / big tire / I look like I'm in a mini CJ-5 look.
* If you're more interested in off road function, and making people go "Holy *&^, did that little thing just do that?" and still having a prayer at doing 65 MPH comfortably on the freeway - lock your front and rear diffs, and get some aggressive 235/75/15 Mudd Terrain (MT) tires (which are 2" bigger than stock, don't require a lift to clear them, won't totally suck your remaining HP or add significant stress to the drive line). Also, do the Rears Up Front (RUF) conversion that allows you to use the much larger rear springs (from a doner vehicle) in the front of your vehicle - much better ride / articulation for cheap.
Lockers in order from cheapest to more expensive (but still in your budget): Welded gears (free if you have friend with welder), spool ($100), lockrite ($200) per diff.
Also consider being PATIENT, and looking at craigslist for the right Samurai to come your way. I have seen some very well built Sami's go for cheap in my area ($3000 for a fully lifted sami on 31" tires, SPOA lift, locked diffs, and rebuilt engines). As a general rule of thumb, you get back about 25-50% of what you put into a custom vehicle. In other words - if you put $2000 worth of lift / tires / lockers on your vehicle, you're only going to add about $500 - $1000 to the sale price of your rig. Take advantage of someone else taking the "loss" and buy one already built for MUCH less than what you'd pay to do it yourself.
Case for the Tracker / Sidekick (Trackick)
This may be more in your budget than you think. I just purchased a Tracker with 115,000 miles on it for $1900 (asking price was $3500).
You won't be winning any drag races in this one either, but it's extra power is IMMEDIATELY noticeable. It's also a much better wheeler than a lot of people give it credit for, and a lot cheaper to do a 29-31" tire-clearance lift on (much more expensive to do a 33" + tire-clearing lift on). My kick feels MUCH more stable than either of my sami's did, and the suspension feels much better. Take a look around - MANY of the Sami mods are stealing Kick parts (disk breaks, engine / transmission, power steering) because the kick is a better running vehicle with more power.
A Trackick benifits from modern engineering (fuel injection, better suspension, air conditioning, power steering, air bags, more reliable, better MPG & more power). However, with more modern components comes greater complexity in troubleshooting. Then again - sometimes the engine "self diagnoses" - you can just plug in a code reader (free at a lot of auto parts stores) and it'll tell you what's wrong... IF it's smart enough to know. The sami is still easier to work on though, as there's less electronics, sensors etc to worry about. That said, Kicks are more "reliable" because they're newer.
Unfortunately the styling of the tracker is "out of date" compared to the Samurai - which is gaining in popularity again as people are starting to like the "square" look again. If you're more interested in spending less $$$ on gas, keeping up on the freeway, having a smoother ride, and much more cargo space - the Trackick is for you.
What I'd spend my $$$ on:
With a kick, you have more power, so you can go to bigger tires without AS much of a loss. To clear 31" tires, you can do a little trimming, and / or throw in a little dough nut looking thing on the top of your springs (cost = $200 - 300, much cheaper than sami lifts). Welded diffs are still free, spool is still $100, but the lockrite goes up to $300. I'd still stick with 29 / 30" tires and a locker - you'd be AMAIZED at how much more economical / better this setup will be than a big lift / big tires / open diffs will be... you keep your low center of gravity, and all your horse power isn't sucked up by big tires.
I've fallen in love with every samurai and ever tracker I've owned. Each one has it's own personality, but both have been great offroad, and fantastic "starter" 4x4 vehicles. You have to decide which things are most important to you in a vehicle (vanity, power, ease of maintainence). Each one just "leans" more one way or than the other.