I tried adding caps across the rectifiers- no good.
I tried bypassing the main filter cap with a 20nf ceramic cap. No
good.
I tried swapping out the rectifiers. No good.
I added a 3 terminal voltage regulator, reducing the audio amp supply
voltage to about 10V. Buzz all but gone. I have to press my ear to
the speaker to hear any buzz now.

The distortion in the audio seems to have gone up a bit, so maybe an
amp that can operate from a lower voltage would be better. I'll look
into audio amp chips and see if I can find a suitable device.

Thanks for all input.

MR