Sam Harris' challenge to obfuscating scientists: Reconsidering relative moralism

So I'm reading Sam Harris' book "The Moral Landscape".  When I got to the section titled "Moral Blindness in the name of "Tolerance", I knew he was referring to people like me and it was gonna hurt!  Having just finished an extended debate with an email buddy wherein I had been using the term moral relativism rather frequently, I knew that Mr. Harris was about to call me on it.  Indeed, after reading just a bit of the material (I'm only on chapter two) I am painfully aware that 1) I am not really a moral relativist at all, and 2) I had been using the "softer", "relative" form as a way to sustain connections across chasms of different philosophical leanings.

What I mean is: 1) I believe, and have said before, that our common human biology ensures that there are human-universal values; that we as a species have more in common (morality-wise) than we have differences. Thus, I do believe that there are, at least some, absolutes and have been using the term moral relativism improperly.  And 2) I guess I felt that if I said what I truley believed; that man, not God, was the source of morality, then my conversations would be over before they started.  Thus, my relativism was a form of cowardice on my part.  An inability to take a stand about what I believe to be true. Mr Harris is trying to convince us (most specifically scientists) to stop being weenies and take a position!

Ok.  I get that.  My next question then is; if Mr. Harris is correct, and morality is based on common needs and the well-being of humans and (some/most?) animals then is intervention to "right" a "wrong" always a moral responsibility?  For example, Mr. Harris points to the compulsitory wearing of Burqas for women as a moral wrong.  I agree.  Is it then my moral responsibility to attempt to right that wrong?  Am I immoral if I don't act?  What if there are so many wrongs needing "righting" that I have to decide how to spend my time?  What if I feel that capital punishment is a nastier form of immorality than burqas and perhaps one I can make an impact on?  Suppose I conclude that both concepts are morally wrong, but.... am I making a judgement on their relative wrongness when I decide which deserves my attention?  I have to admit to being somewhat confused now.  Am I being an absolutist with little time, or a relativist with lack of.....hmm... cojones?

I guess I do feel there are moral absolutes, but I don't yet know how that fact should be incorporated into my life.  Guess I better read the rest of the book!