Did you actually read the paper you referenced?

Two points:

The authors note on the third page that they could not find any prior
derivation of the orthonormality relationships for this configuration.
The paper was published in 2000, so it should be an easy library
research project to find any subsequent papers that reference this work.

Appendix A goes through the derivation of the orthonormality
relationships in excruciating detail. Every step is outlined, including
the integration tricks. It is unlikely that you would find any paper, or
even a textbook, that is more detailed.

73,
Gene
W4SZ

wrote:
I am working on dielectric-lined circular waveguides and I am searching
for a reference on the orthogonality of modes. The only thing that I
found is in the link below (at the bottom of the message)
but I am looking for something with more details.

Assume that we have a uniform cylindrical waveguide with N concentric
dielectric layers [my case of interest is when N=2, and we have the
waveguide with a thin inner dielectric layer (second layer N=2) and the
rest is vacum (air- N=1- first "layer")].
I need an orthogonality relation that applies between any two modes.
The fact is that I already have found an orthogonality relation (see
the link below) but I need a more detailed analysis.

http://www.bnl.gov/ATF/experiments/D...y_of_Wakes.pdf

IF ANYONE KNOWS SOMETHING PLEASE ANSWER THIS MESSAGE.