Depending on how far you want to drive, there are lots of scenic 2WD-friendly routes in San Diego County. I'll list a few here, and feel free to ask if want more specifics. Some of these are great all year, while others (like Anza Borrego) are not any fun at all in the heat of summer.
Anza Borrego - Blair Valley: Pretty much all of Blair Valley is accessible to a 2WD Xterra. Look back through the Excursion Pictures if you want to see what's out there.
Anza Borrego - Mine Wash: Mine Wash is quite scenic and can easily be done in 2WD all the way to the end. It's off of CA-78.
Anza Borrego - Clark Dry Lake: You can easily drive at least 10 miles up the trail that starts at Clark Dry Lake heading north off of S-22.
Anza Borrego - Buttes Pass Road area: There are a bunch of trails around here that you can do in 2WD. Some soft sand in places so use caution. An air compressor is handy so that you can air down, which will make a huge difference.
Anza Borrego - Fish Creek: You can generally get to at least the Wind Caves in Fish Creek with no problems at all in 2WD, and I've seen good drivers get a 2WD Xterra all the way to the far end of Sandstone Canyon. I've not been out there in a while, so not sure what the current conditions are. It's a long walk back from Sandstone to the pavement, so probably best to not do that one alone if you're in 2WD, unless you are confident in self-recovery.
Cleveland NF - Palomar Divide Road: It's been a while since I've been up there, but I got quite a ways in an old 2WD Datsun pickup years ago. It's a narrow one-lane shelf in spots, but should be fine in 2WD. I think the Forest Service keeps that one graded, you can check with them. If the weather looks threatening they often lock the gate though.
East of San Diego - Boulder Creek Road: A graded fire road with really nice views, runs from near Descanso up to near Julian.
Cleveland NF East of San Diego - The one-lane shelf (paved) up to the Los Pinos Fire Lookout, and then go back down the same way, then circle the Corral Canyon OHV Area on Corral Canyon Road. Note that the trails in the OHV area are 4WD, and some are too narrow for an Xterra. Note also that unless the Forest Service has done some major work, Espinosa Trail is 4WD with, at a minimum, good AT tires.
East of San Diego - Pine Creek Road: The route is actually more or less paved the whole way, but this is a really scenic one-lane forest service road that runs north from Old US 80 just west of Pine Valley, and goes up to a saddle and over into the Laguna Mountains Rec Area, ending up at Sunrise Highway.
There are a couple of books by Tony Huegel that are very helpful for finding easy but scenic trails. They each cover the whole State. California Coastal Byways and California Desert Byways. Some of the trails he describes do require 4WD, but a lot of them just need driving skill, good AT tires and some ground clearance.
Disclaimer: I've done all of these, but in a 4WD Xterra, although not always in 4WD. That said, you're on your own. Trails can change quickly, and what might have been 2WD-friendly in the past might now be really soft sand or full of rocks. Use common sense and know when to turn around.
