writes:

Where can I get a detailed description of AMTOR-B's protocol?

The 2005 ARRL Handbook, p. 9-19, in the bibliography for the section
on AMTOR, contains the following:

CCIR recommendation 476-3, "Direct-Printing Telegraph Equipment in the
Maritime Mobile Service." Reprint available from ARRL Headquarters as
part of the Proceedings of the Third ARRL Amateur Radio Computer
Networking Conference.

With that hint, typing 'CCIR 476-3' into a search engine might turn up
a copy of the official definition online somewhere. "SITOR-B" or
"NAVTEX" might be other good search terms to try, since SITOR is what
the protocol is called in the maritime mobile service, and NAVTEX is
the service that the US Coast Guard provides using SITOR.

I realize that you want to write your own program, rather than copy or
use an existing one, but you might find the source code of hfterm
instructive. http://hfterm.sourceforge.net. It's a Linux program
that implements AMTOR, PACTOR, GTOR, and some other digital modes.

--
Chris Jewell Gualala CA USA
(No call sign yet: I'm taking elements 1, 2, and 3 in February.)