It all sounds possible to me. It is not very different from some of
the schemes we use commercially to generate internal frequencies (in
spectral analysis equipment). Your idea of good filtering at 480MHz
(and also, I hope, at 120MHz) is a good one, to keep out other
combinations of multiplication. The one other thing I would be careful
about is the phase noise of the oscillator. The AD9954 should be
capable of very good phase noise performance, but you MUST feed it a
clean reference to keep the output clean. I have recently tested some
TCXOs which are very stable and accurate, but whose phase noise is not
as good as I would really like. It is important to feed the oscillator
from a very clean supply, but that alone is not a guarantee of good
phase noise. See http://www.techlib.com/electronics/finesse.html for a
nice idea to make a very clean supply for circuits that draw
essentially constant current.

Cheers,
Tom