Dear all

I am looking for ideas for a software CW decoder with ASCII output.
There are a number of programs published - what algorithms do these use in
general?
I presume the software would have to have some sort of PLL to select and
filter a signal
and then an adaptive algorithm to allow for sending operator error in
keying, noise and variations in keying speed. I am looking for a solution
which trades off accuracy for minimum codespace and speed. If necessary,
some of the work could be done in hardware - eg I could limit and square up
the audio waveform and present it the processor as digital level
transitions, so AGC would not be an issue.
Any ideas?

Thanks in advance

Richard

On Tue, 30 Dec 2003 11:49:39 +0800, "Richard Hosking"
wrote:

I am looking for ideas for a software CW decoder with ASCII output.
There are a number of programs published - what algorithms do these use in
general?

There are two steps that you have complete: make a digital signal out
of the received analog output - and decode it.

Have a look at:
http://wwwhome.cs.utwente.nl/~ptdebo...algorithm.html

73s
OE1MWW
Wolfgang

Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as
though it might be somewhat hard to implement in a small/slow codespace such
as a microcontroller. (It may well be impossible to do this job in such a
device.)
Still I thought I would try.
Off the top of my head I thought I would start with a hard limited audio
signal, with transitions at digital hi/low level, rather than a sampled
signal. This would obviously limit the signal to noise ratio, but I was not
planning to use this for weak signal work. The clock/demodulation algorithm
would then deal with the timing between transitions of the signal, rather
than a spectral analysis. This would be relatively simple to implement with
a timer to count the interval between transitions. The controller could
track carrier freq with a Freq Locked Loop algorithm, over a limted
range.(say 600-900 Hz) If there are no transitions or randomly timed
transitions (noise), then the controller assumes a space Similarly, the
controller tracks sending speed by timing the length of dots and dashes, and
adjusting the algorithm to track this. This length could be initially
acquired by timing an initial series of dots/dashes. The length would
presumably fall into two groups, with the shorter length being dots. The
average of these shorter "on" periods would be the bit rate. The controller
could keep a moving average of the signal, discarding any "on" periods that
are longer than say 2.0 times the current average. Finally, I would use the
bit rate to time gaps (say) 2.5 times the bit rate to separate characters
and decode them with a simple lookup table.
No doubt there are lots of problems with this approach!
I would be interetsed in your comments

Richard


Wolfgang K. Meister wrote in message
...
On Tue, 30 Dec 2003 11:49:39 +0800, "Richard Hosking"
wrote:

I am looking for ideas for a software CW decoder with ASCII output.
There are a number of programs published - what algorithms do these use
in
general?

There are two steps that you have complete: make a digital signal out
of the received analog output - and decode it.

Have a look at:
http://wwwhome.cs.utwente.nl/~ptdebo...algorithm.html

73s
OE1MWW
Wolfgang

"Richard Hosking" wrote in message
u...
Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as
though it might be somewhat hard to implement in a small/slow codespace
such
as a microcontroller. (It may well be impossible to do this job in such a
device.)
Still I thought I would try.
Off the top of my head I thought I would start with a hard limited audio
signal, with transitions at digital hi/low level, rather than a sampled
signal. This would obviously limit the signal to noise ratio, but I was
not
planning to use this for weak signal work. The clock/demodulation
algorithm
would then deal with the timing between transitions of the signal, rather
than a spectral analysis. This would be relatively simple to implement
with
a timer to count the interval between transitions. The controller could
track carrier freq with a Freq Locked Loop algorithm, over a limted
range.(say 600-900 Hz) If there are no transitions or randomly timed
transitions (noise), then the controller assumes a space Similarly, the
controller tracks sending speed by timing the length of dots and dashes,
and
adjusting the algorithm to track this. This length could be initially
acquired by timing an initial series of dots/dashes. The length would
presumably fall into two groups, with the shorter length being dots. The
average of these shorter "on" periods would be the bit rate. The
controller
could keep a moving average of the signal, discarding any "on" periods
that
are longer than say 2.0 times the current average. Finally, I would use
the
bit rate to time gaps (say) 2.5 times the bit rate to separate characters
and decode them with a simple lookup table.
No doubt there are lots of problems with this approach!
I would be interetsed in your comments

Richard

Many years back, I built a TTL deocder from a QST article- by Petway??
Anyway he used a number of storage registers- one for average dot length,
one for average character spece length and one for average word space
length. After initialization, a clock would run at the receipt of each
element. The count was then compared to previous average- if the clock count
was larger, the average was incremented by 1; shorter, it was decremented by
1.
The machine did a surprisingly good job on hand sent Morse. Although the
hardware technology is antiquated by today's standards, the algorithm was
elegant and easily understood. I would estimate this article (2 or 3 parter)
appeared in QST from the early 70's.

GL,

Dale W4OP

On Tue, 30 Dec 2003 20:10:53 +0800, "Richard Hosking"
wrote:

Wolfgang
Thanks for your reply

Richard,

I will dig into my paper piles - I do have a very small and tiny Basic
source with a flow chart, published years ago. I wanted to write this,
but time went by and in the meantime ready built software is available
in the net. Stay tuned - I will post it.

73s and DX for 2004

Wolfgang
OE1MWW

Disclaimer: I've never decoded morse in my life, so most of what I write is
educated guessing.

For my master's thesis I built a receiver that demodulated 400 baud MSK.
The RF chain delivered a signal centered around 2kHz to a 68HC11 which
sampled at 6400Hz and demodulated the data entirely in software -- and it
only had 2k bytes of code space. Granted, it was all in assembly, and it
used 94% of the available processor time to do it, but it did work. In
spite of having some really ugly compromises in the signal processing (the
sampling rate was really too low, and the only "multiply" was to add, not
add, or subtract from an accumulator) the receiver had performance within a
dB of the theoretical optimum. I think with something modern like an AVR or
a PIC a 16-baud OOK signal should be possible (fun, too).

OK, so you still have the problem with the math.

Consider using not one, but two comparators to square up your signal, so a
"non-signal" rarely gets above the threshold. You're back to needing AGC
(at least to set the comparator), but you can close the loop in your
software by looking at the comparator outputs for the right signatures (too
much output = too low threshold, too little output = too high threshold),
and set the threshold from the uP. With a good signal to noise ratio this
should have a nice broad center to it that'll be easy to find. With a good
narrow CW filter your signal to noise ratio will get better...

Once your threshold is set you can integrate the "buzz" from the comparators
to see if you're receiving a carrier. You should probably integrate for
some period of time (the length of the fastest code you want to receive
would be good, it'd give you a sorta-kinda-matched filter) and take the
presense of signal 1/2 of the time to indicate the transmitter is "on".

I like the idea of using the dibits to form a PLL, but for receiving bad
code at an unknown rate you'd probably be better off making a little
histogram of the on and off times of the code. You should be able to see
two peaks in the histogram for "on" corresponding to dot and dash, and three
for "off" corresponding to intra-letter, inter-letter and inter-word
spacing. At this point you can start looking for dashes, dots, and spaces
(maybe going back over your data to catch the beginning of the message, if
you have enough RAM) and decoding words. You could also estimate the speed
of the code that you're receiving and feed it back to your KSMF (that's
kinda-sorta-matched filter, of course) for better noise figure.

Once you have your stream of dashes, dots, inter-charater and inter-word
spaces you can separate the charaters, look them up in a table and just
reject anything that doesn't seem to fit. I suspect that your performance
would be pretty good at this point, and if not the theoretical ne-plus-ultra
of decoders it would be better at receiving the lousy senders than
Wofgang's.

"Richard Hosking" wrote in message
u...
Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as

Thanks to all for your replies
With regard to Tim's idea of the dual comparators, I presume these would in
"series". For the integration idea to work, the comparators would obviously
have to have their gain set carefully, otherwise random noise would
integrate to 1/2 time and be interpreted as signal. The circuit I was
thinking of would set the DC comparison point at halfway (say 2.5V for a 5V
logic level) with the DC level of the signal also at 2.5V. I presume in your
system the uP controls the comparison voltage and sets it with a DAC and the
signal DC level is 0V. Is this correct?
In my idea I was going to use the timing of signal transitions in a hard
limited signal. If there were several transitions with the same timing, I
would presume a "tone" was present and signal was being received. Randomly
timed transitions would indicate noise.The uP could follow the tone as an
adaptive filter to track carrier freq. Once carrier is received the uP then
captures dot, dash and space timing with the histogram as you describe. If a
moving average filter of these is kept, then the uP could track sender speed
variations.
Anyone have ideas as to the theoretical performance of such an idea or any
potential problems?

Thanks again
Richard
Tim Wescott wrote in message
...
Disclaimer: I've never decoded morse in my life, so most of what I write
is
educated guessing.

For my master's thesis I built a receiver that demodulated 400 baud MSK.
The RF chain delivered a signal centered around 2kHz to a 68HC11 which
sampled at 6400Hz and demodulated the data entirely in software -- and it
only had 2k bytes of code space. Granted, it was all in assembly, and it
used 94% of the available processor time to do it, but it did work. In
spite of having some really ugly compromises in the signal processing (the
sampling rate was really too low, and the only "multiply" was to add, not
add, or subtract from an accumulator) the receiver had performance within
a
dB of the theoretical optimum. I think with something modern like an AVR
or
a PIC a 16-baud OOK signal should be possible (fun, too).

OK, so you still have the problem with the math.

Consider using not one, but two comparators to square up your signal, so a
"non-signal" rarely gets above the threshold. You're back to needing AGC
(at least to set the comparator), but you can close the loop in your
software by looking at the comparator outputs for the right signatures
(too
much output = too low threshold, too little output = too high threshold),
and set the threshold from the uP. With a good signal to noise ratio this
should have a nice broad center to it that'll be easy to find. With a
good
narrow CW filter your signal to noise ratio will get better...

Once your threshold is set you can integrate the "buzz" from the
comparators
to see if you're receiving a carrier. You should probably integrate for
some period of time (the length of the fastest code you want to receive
would be good, it'd give you a sorta-kinda-matched filter) and take the
presense of signal 1/2 of the time to indicate the transmitter is "on".

I like the idea of using the dibits to form a PLL, but for receiving bad
code at an unknown rate you'd probably be better off making a little
histogram of the on and off times of the code. You should be able to see
two peaks in the histogram for "on" corresponding to dot and dash, and
three
for "off" corresponding to intra-letter, inter-letter and inter-word
spacing. At this point you can start looking for dashes, dots, and spaces
(maybe going back over your data to catch the beginning of the message, if
you have enough RAM) and decoding words. You could also estimate the
speed
of the code that you're receiving and feed it back to your KSMF (that's
kinda-sorta-matched filter, of course) for better noise figure.

Once you have your stream of dashes, dots, inter-charater and inter-word
spaces you can separate the charaters, look them up in a table and just
reject anything that doesn't seem to fit. I suspect that your performance
would be pretty good at this point, and if not the theoretical
ne-plus-ultra
of decoders it would be better at receiving the lousy senders than
Wofgang's.

"Richard Hosking" wrote in message
u...
Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as

Thanks to all for your replies
With regard to Tim's idea of the dual comparators, I presume these would in
"series". For the integration idea to work, the comparators would obviously
have to have their gain set carefully, otherwise random noise would
integrate to 1/2 time and be interpreted as signal. The circuit I was
thinking of would set the DC comparison point at halfway (say 2.5V for a 5V
logic level) with the DC level of the signal also at 2.5V. I presume in your
system the uP controls the comparison voltage and sets it with a DAC and the
signal DC level is 0V. Is this correct?
In my idea I was going to use the timing of signal transitions in a hard
limited signal. If there were several transitions with the same timing, I
would presume a "tone" was present and signal was being received. Randomly
timed transitions would indicate noise.The uP could follow the tone as an
adaptive filter to track carrier freq. Once carrier is received the uP then
captures dot, dash and space timing with the histogram as you describe. If a
moving average filter of these is kept, then the uP could track sender speed
variations.
Anyone have ideas as to the theoretical performance of such an idea or any
potential problems?

Thanks again
Richard
Tim Wescott wrote in message
...
Disclaimer: I've never decoded morse in my life, so most of what I write
is
educated guessing.

For my master's thesis I built a receiver that demodulated 400 baud MSK.
The RF chain delivered a signal centered around 2kHz to a 68HC11 which
sampled at 6400Hz and demodulated the data entirely in software -- and it
only had 2k bytes of code space. Granted, it was all in assembly, and it
used 94% of the available processor time to do it, but it did work. In
spite of having some really ugly compromises in the signal processing (the
sampling rate was really too low, and the only "multiply" was to add, not
add, or subtract from an accumulator) the receiver had performance within
a
dB of the theoretical optimum. I think with something modern like an AVR
or
a PIC a 16-baud OOK signal should be possible (fun, too).

OK, so you still have the problem with the math.

Consider using not one, but two comparators to square up your signal, so a
"non-signal" rarely gets above the threshold. You're back to needing AGC
(at least to set the comparator), but you can close the loop in your
software by looking at the comparator outputs for the right signatures
(too
much output = too low threshold, too little output = too high threshold),
and set the threshold from the uP. With a good signal to noise ratio this
should have a nice broad center to it that'll be easy to find. With a
good
narrow CW filter your signal to noise ratio will get better...

Once your threshold is set you can integrate the "buzz" from the
comparators
to see if you're receiving a carrier. You should probably integrate for
some period of time (the length of the fastest code you want to receive
would be good, it'd give you a sorta-kinda-matched filter) and take the
presense of signal 1/2 of the time to indicate the transmitter is "on".

I like the idea of using the dibits to form a PLL, but for receiving bad
code at an unknown rate you'd probably be better off making a little
histogram of the on and off times of the code. You should be able to see
two peaks in the histogram for "on" corresponding to dot and dash, and
three
for "off" corresponding to intra-letter, inter-letter and inter-word
spacing. At this point you can start looking for dashes, dots, and spaces
(maybe going back over your data to catch the beginning of the message, if
you have enough RAM) and decoding words. You could also estimate the
speed
of the code that you're receiving and feed it back to your KSMF (that's
kinda-sorta-matched filter, of course) for better noise figure.

Once you have your stream of dashes, dots, inter-charater and inter-word
spaces you can separate the charaters, look them up in a table and just
reject anything that doesn't seem to fit. I suspect that your performance
would be pretty good at this point, and if not the theoretical
ne-plus-ultra
of decoders it would be better at receiving the lousy senders than
Wofgang's.

"Richard Hosking" wrote in message
u...
Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as

"Richard Hosking" wrote in message
u...

I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as
though it might be somewhat hard to implement in a small/slow codespace
such
as a microcontroller. (It may well be impossible to do this job in such a
device.)

In the August '98 QST, IK3OIL published an article on a CW decoder based on
a PIC16F84, which may well be, by today's standards, the most limited
microcontroller in use by hobbyists. (Well, I suppose you could tie one hand
behind your back by adding a Basic compiler on top of it or something like
that!) That particular device, while far from perfect, works a lot better
than the various PC based programs that I've tried.

Last I looked, he still had most of the information on his web site. I know
others have done work to improve on his program, but I don't know of any of
that work that has been published. One nasty feature is that a long series
of dahs can greatly confuse his algorithm and it can take a while to recover
from that. Were you to implement his algorithm in a '628 or something like
that, or even to simply replace the rather lengthy code he has around
displaying the speed with a little more work on the algorithm, I'm sure it
could be improved.

He did use an external decoder to recognize the CW signal, a 567 or
something of that ilk. Some folks have worked on that end, too. It seems
like there should be a better approach, but I haven't seen anything
dramatically better.

...

"Richard Hosking" wrote in message
u...
Wolfgang
Thanks for your reply
I am not very sophisticated mathematically and I have not studied
communication theory. Your article is most helpful. However it looks as
though it might be somewhat hard to implement in a small/slow codespace
such
as a microcontroller. (It may well be impossible to do this job in such a
device.)
Still I thought I would try.
Off the top of my head I thought I would start with a hard limited audio
signal, with transitions at digital hi/low level, rather than a sampled
signal. This would obviously limit the signal to noise ratio, but I was
not
planning to use this for weak signal work. The clock/demodulation
algorithm
would then deal with the timing between transitions of the signal, rather
than a spectral analysis. This would be relatively simple to implement
with
a timer to count the interval between transitions. The controller could
track carrier freq with a Freq Locked Loop algorithm, over a limted
range.(say 600-900 Hz) If there are no transitions or randomly timed
transitions (noise), then the controller assumes a space Similarly, the
controller tracks sending speed by timing the length of dots and dashes,
and
adjusting the algorithm to track this. This length could be initially
acquired by timing an initial series of dots/dashes. The length would
presumably fall into two groups, with the shorter length being dots. The
average of these shorter "on" periods would be the bit rate. The
controller
could keep a moving average of the signal, discarding any "on" periods
that
are longer than say 2.0 times the current average. Finally, I would use
the
bit rate to time gaps (say) 2.5 times the bit rate to separate characters
and decode them with a simple lookup table.
No doubt there are lots of problems with this approach!
I would be interetsed in your comments

Richard

Many years back, I built a TTL deocder from a QST article- by Petway??
Anyway he used a number of storage registers- one for average dot length,
one for average character spece length and one for average word space
length. After initialization, a clock would run at the receipt of each
element. The count was then compared to previous average- if the clock count
was larger, the average was incremented by 1; shorter, it was decremented by
1.
The machine did a surprisingly good job on hand sent Morse. Although the
hardware technology is antiquated by today's standards, the algorithm was
elegant and easily understood. I would estimate this article (2 or 3 parter)
appeared in QST from the early 70's.

GL,

Dale W4OP