I had used that chip for several years, with a 2.5kHz reference frequency,
and never ran into those kinds of problems. My first iteration of that
synthesizer did have some high level reference sidebands (-40dBc), but after
I redid the board with a top layer ground plane, all of the problems cleared
up. Are you using a ground plane on your board? Are you also using that
anti-slewing filter at the op-amp input, whereby you split one of the Rs,
and use a shunt C between the two?
If you would like a schematic of my loop filter, give me a shout.

Pete

Michael A. Terrell wrote in message
...
Damien Teney wrote:

Hello,
About the quartz, I haven't used a 10.24MHz one as in the given
schematic,
but a 5.12 MHz one. The R ratio is 1024, so the reference frequency is 5
KHz. The N ratio used is from 196 to 284 (with the 64/65 prescaler; VCO
goes
from 63 to 91 MHz). I don't think the loop is unstable, because the VCO
output is quite clean (only a bit blur, see
http://www.mcarsweb.com/_divers/vco.jpg).
The loop filter values have been calculated with the fomulas given in
the
motorola application note 980.

So, if you think I should use a additional RC network, could you tell me
how
to calculate the values :-$ ?
Thank you ;-)

Damien

Do you have a 5 kHz notch filter in the control voltage feeding your
VCO? It will reduce phase noise in the output. A simple L/C trap will
work, but either the coil or cap should be adjustable for maximum
rejection.
--


Michael A. Terrell
Central Florida