You haven't fully described your setup, so it's hard to tell. What are
you using for a bridge? If I recall correctly, your VNA is an HP8754A,
which won't do S-parameter measurements by itself; it requires an
external bridge (an S-parameter test set, or a reflection-transmission
test set, for example).

The fact that you see an arc on the outermost circle, assuming that you
have it set as a unit circle on the linear reflection coefficient plane
(the Smith Chart outer circle), implies that you have a (nearly) unity
reflection coefficient. Is something almost disconnected so it looks
like a tiny series capacitance?