I've got a number of items that can be used, ranging from a wood mallet, to a "Wood is Good" rubber head mallet, to a Japanese plane hammer to a "Vaughn hammer, with hard plastic on one side and rubber on the other, 16 oz.". Invariably the Vaughn or Wood is Good is used and through the years, I have not noticed any damage to the boxwood handles on my chisels.
I do have one chisel with a mucked up end and that is a heavy 1-1/4" Witherby with an ash handle I made. Like an idiot, in the middle of a job I grabbed the nearest hammer to tap with and hit way to hard. A few days later I chucked the handle in the lathe and made it pretty again. But wouldn't you know, a few weeks later, the same idiot was in my shop and dented the handle again.
If the thunder don't get you, the lightning will.