You need first to realize that the "shield" IS the antenna. The whole
point of the "shielded loop" is that you can make it very symmetrical,
which is just what's needed to reject strong local electrical fields.
The symmetry does nothing to reject electromagnetic signals. BUT you
can make an "unshielded" loop which is as symmetrical as a "shielded",
if you are careful, and get the same advantages. If you really want to
build one like a classical "shielded loop" and maintain high Q, just
build the "shield" out of copper pipe and put the capacitor across the
gap. The wire inside the pipe is just the center conductor of a short
piece of coax connected to the feedpoint.

If you don't understand this, please see King, Mimno and Wing's
"Transmission Lines, Antennas and Waveguides." It's explained quite
nicely in the "antennas": chapter. It's also explained reasonably well
in Johnson and Jasik's "Antenna Engineering Handbook."

Cheers,
Tom