DSP lightning noise reduction for WEAK signals

[K1JJ]

Hola,

I am trying to determine the best approach to reduce static crashes when listening to weak ssb (or AM) signals on 75M.

I've been reading about the TimeWave 599ZX and JPS NIR-12 DSP boxes that process off the audio.  The reviews seem to indicate they don't do squat for lightning crashes, but some say they help.  The problem seems to be that the AGC is too slow on most receivers, so they recommend to turn OFF the AGC, ride the RF gain and put a pair of back-to-back diodes in the earphones for protection.  I am tempted to try one of these boxes, but they are older - five years old and older technology. Plus they come off the audio, which sounds rather hokey.

Then there's the software/computer route where we tap off the radio's 455 kc I.F. and process from there. This appears to be the better approach, but how GOOD is the latest software for processing LIGHTNING crashes and voice? 

What concerns me is that everyone who runs the newest $12K riceboxes with firmware I.F. derived DSP (or DSP software radios) still complains like hell about the static crashes in the DX window... and I can still hear as well or better than most of them when the static is brutal - without any DSP processing at all in my older FT-1000D.  I wonder if the trained brain and directional antenna system is still the only viable lightning "DSP" at this point in time? 

All I do now is put in 20db of front-end attenuation.  There must be a better way.

I would be interested in hearing of usable solutions or tests done that have proven lightning noise reduction other than antenna directivity.  What I am reading is when it comes to VERY WEAK signals at or near the noise floor, the latest DSP techniques simply mask the signal when lightning is involved and it's better without DSP.

It would be a minor job to put a computer in the shack and buy a $16 I.F. /software interface for the FT-1000D, but I would like to know if it's worth the effort for lightning first, before spinning my wheels with software that has not evolved there yet.

TNX.

T

[WU2D]

JJ,

The difficulty in your question is that these are weak noise bursts, not strong ones that you can handle with conventional blankers and noise limiters. You need gain.

It may be worth investigating what you could do with a separate noise receiver attached to the same antenna, but tuned just out of the band to a quiet frequency. Got an extra receiver laying around?

This noise-only signal could be linearly phased against the primary signal at the IF.

This could be done pre-demodulation or in your case pre-sampling.

Better yet sample both signals and correct in the DSP.

Mike WU2D

[WA1GFZ]

I2PHD's Ver .99 software gave a very slight advantage over the stock Racal running fast AGC. Fast AGC in the racal works best for static. Flex software isn't all that great. Actually the stock Racal blows it away in high static conditions. I noticed this when I changed over to HPSDR interface with Flex software. One night nothing but noise was coming out of the SDR while I could still copy the Racal audio. Read Sherwood's article on DSP. The best method is run a sense antenna and feed the noise into your RX out of phase to cancel it. I think Jay W1VD has a setup like that. You just have a null pot to adjust for best attenuation. Software will get better in time but it isn't there yet. I suspect it will be moe possible with new SDR configurations like QSR1 and HPSDR Mercury with the A/D conversion right after the preselector.
The problem with noise in an RX is the further in it goes the wider it gets due to high Q circuits. The wider it gets the harder it is to eliminate.  

[Bacon, WA3WDR]

When I listen to a lightning crash, I hear a certain quality that must have something to do with the timing of the individual arc snaps.   I hear a fast sshhh that consists of a cascade of little snaps, and I hear multiple pops at fairly well defined intervals.

I think that examination of the mechanics of the lightning will show why some of these effects happen, and allow some degree of prediction of them.  DSP should be able to distinguish a lot of the louder lightning effects and block them, ecspecially if we tolerate some throughput delay for processing.

But it is not the typical application of DSP.  It is more of a time-domain issue than a frequency domain issue.  The thinking needs to change back to older time-domain thinking.

I would think that the correlator noise reduction technology from all the way back in the late 80s would help, if only because it reduces the initial burst of noise.  But intelligent time-domain muting of the frequency domain data seems to me to hold the most promise, because it would allow the fastest muting and unmuting, without the old intermodulation effects of the analog systems.  What I envision is a sensor that identifies a lightning artifact, and momentarily blocks the signal much as the old analog blanker systems did.  But instead of electrical hardware trying to clamp an analog voltage with a switching action, it should do this to numeric data in a spectrally smart way, using raised cosine or Gaussian curves, etc, and tailored to each frequency band so that periodic blanking does not 'beat' against the throughput audio any more than it has to.  The effect should be that the short snap never gets to the filter, the filter doesn't ring from it, the muting does not mangle the signals any more than it has to, and the remaining time slots blur together to the filter and to the ear, and we hear a quieter signal.

[WQ9E]

For a conventional radio I doubt that DSP is going to do much for lightning and the DSP built into my 1000MP certainly doesn't help much.  If you go back to the 70's and 80's radios with blankers, they were generally designed to do their work before the pulse got wider due to narrow filters in the later IF stages.  I imagine once the pulse gets through the detector these add on audio dsp units will be of little value beyond their own ability to peak limit and protect your ears.  As suggested by another poster, it would be interesting to see how you could do with the noise input using a separate receiver source to generate the blanking pulse and apply that early in the receiver signal chain.  As I recall the optional KWM-2 noise blanker used this approach to grab ignition noise so that might provide some initial direction.

 

[w3jn]

Noise blankers aren't too effective on lightning crashes, even if properly designed (ie signal sampled BEFORE the IF filtering).  The KWM-2 blanker used a strip in the low VHF range, as I recall, to receive the pulses rather than from the receiver IF itself, the thought being that most of the RF energy from noise pulses was in that range. 

THe acme of NB design is the Squires-Sanders SS-1 - a separate IF strip directly off the mixer toob amplified the 1 MHz wide (at that point) spectrum and rectified the result.  The wideband noise pulses would therefore be dominant over even very strong, but narrow, signals in the bandpass.  The rectified pulses then drove a transistor which shunted the IF to ground just after the IF filter, IIRC.

The best thing (and even it isn't that effective) is the ol' ANL as found in HROs, NC-183s, etc.  The diodes in the headphoniums trick is a variation on the old ANL, except it's just a limiter rather than being biassed by the AGC (hence the term "automatic") as in the old buzzard receivers.

The DSP on my Yahoo FT-1000 MK V is pretty worthless for lightning static also.  Pretty good for buzzies and other crap though.  Any DSP rig is gonna have big challenges having an effective DSP noise blanker, if it is relying upon filtering somewhere up the chain.

[K1JJ]

JJ,

It may be worth investigating what you could do with a separate noise receiver attached to the same antenna, but tuned just out of the band to a quiet frequency. Got an extra receiver laying around?

This noise-only signal could be linearly phased against the primary signal at the IF.

This could be done pre-demodulation or in your case pre-sampling.

Better yet sample both signals and correct in the DSP.

Mike WU2D

Thanks for the great info, guys.

The method mentioned by Mike above and by Frank makes sense.  It seems so simple that I wonder why it's not being commonly done, being that lightning static is such a big issue on the lower bands, especially for DXers.  Or is it being used ? - in the form of some better noise blankers as mentioned above.

Using this method, I would imagine that when a crash occurs, instead of totally blanking out the voice, you would hear a broken up version of the voice "modulated" by the crash -  but still holding some information for the brain to fill in. Slick.

I've been talking with Jay/VD lately about other tech stuff and hopefully he will chime in exactly what he does to phase/cancel out static crashes. I just picked up one of his 20 db gain RF preamps for the beverages. Works great. Now the bevs and quads are at the same level in the headphones.

Right now I use two receivers in dual diversity... one for the quads and the other for the beverages. (FT-1000D sync'd receivers)  I was thinking of running TWO DSP software programs at once - one for each receiver.    If need a third receiver on a clear freq for the crash samples, that could be done too.  It would be slick to keep the two channels separate in stereo (as it is now) to maintain the dual diversity and also add the crash canceling to the channels too, all thru DSP software.

My gut feeling is to disable the two receiver's AGCs and let the software handle it. As long as the receivers are not overloading maybe that will work best.

Once I get some more ideas, I will definately give this cancellation method a go. Any more details are appreciated.

Gary/INR said to use the $16 software/interface that samples the 455 kc IF and feeds a computer. I think it's called Lite? Anyone have a link to this?  Is this the best choice for this project?

T

[WA1GFZ]

Gary and I are running the same software. the HPSDR interface is as good as it gets for sound card interface. All you need for the cancel circuit is a BB transformer and a pot. Jay can give you the exact values.
 

[K1JJ]

JJ,

The difficulty in your question is that these are weak noise bursts, not strong ones that you can handle with conventional blankers and noise limiters. You need gain.

Mike WU2D

Mike,

Let's see if we're in the ballpark:

Last night the noise floor was about S5 and the static crashes were quite frequent and loud at S9 +10 to +20 over.    I was able to have QSOs with stations that were S6, but when the crashes occured, I would miss full words completely, etc.

Using the data above - strong crashes and weak signals - when using this cancellation method, would it produce voice information through the crash at all, or am I asking too much?  The key here is the desired signals are very weak in comparison to the crashes...

Frank: I went to the HPSDR site and it's a maze.  What is the model/name or link to this $16 455 IF interface I've been hearing about?

T

[Pete, WA2CWA]

Not to re-invent the wheel, maybe you need one of these:



They say, among a lot of other things in product brochure:
"Wipe Out all types of Noise
Wipe out . . . lightning crashes from distant thunderstorms, severe power line noise from arcing transformers and insulators, lamp dimmers, touch controlled lamps, electric drills, motors, industry process, TV birdies . ."

The full story is here:
http://www.mfjenterprises.com/Product.php?productid=MFJ-1026

[k4kyv]

That MoFoJ device reduces distant static crashes essentially by combining the signal and noise antennae in such a way as to form a steerable directional array, with a null aimed towards the distant thunderstorm.

Local lightning discharges do not all come from the same point source, making them impossible to null out.

With strong lightning crashes and weak signals, the amplitude of the noise obliterates the weak signal, so that no amount of processing will be able to recover the signal.  No known, practical receiving system has enough dynamic range to handle the peaks of the lightning pulses while leaving the signal intact and above the noise floor of the receiver.  The best you can expect is to silence the output from the receiver for the duration of the pulse, saving your ears.  This noise blanking action will do less damage to the desired signal if the blanking is made to occur in a stage preceding the i.f. selectivity filter.

[AF9J]

Not to re-invent the wheel, maybe you need one of these:



They say, among a lot of other things in product brochure:
"Wipe Out all types of Noise
Wipe out . . . lightning crashes from distant thunderstorms, severe power line noise from arcing transformers and insulators, lamp dimmers, touch controlled lamps, electric drills, motors, industry process, TV birdies . ."

The full story is here:
http://www.mfjenterprises.com/Product.php?productid=MFJ-1026

I have the JRC version of that, which is marketed by Timewave: the ANC-4.  Here's what it looks like:

http://www.timewave.com/support/ANC-4/anc4.html

It's better tham MFJ's by a fair amount.   The only problem - like the MFJ, it uses vox keying to mute it during transmit, and it is mainly made for use with transceivers.  In my personal opinion, it's best to use it with hard keying.  I blew out the front end protection resistors twice in my unit (and it's pain to get the cover off to make repairs; oh yeah, and as I found out the hard way, they use RoHas solder in them), due to the vox muting not switching fast enough during transmit.  Unfortunately, you have to modify them for hard keying. 

Also, they work best with an outdoor sensing antenna, that's in the same plane as your receiving antenna.  As was mentioned above, you can have intermod issues with your receiver if the phasing, and signal levels from the noise sensing antenna aren't set up right.  Also, it is relatively narrowband in effect - maybe 20 or 30 kHz tops. So, don't expect it to be set and forget for the entire 160, 80/75, 40m band, etc.

I got mine to deal with my line noise issues.  It was OK for it, but not fantastic.  On lightning - well it has some effect on it,  but not a huge amount - maybe 1 or 2 S-units of reduction.  Also, the noise cancelling effect wasn't consistent from day to day.  Some days I'd get a noise drop of 4 S-units.  Other days, an S-unit if I was lucky.  I was always having to reset things on a daily basis.  Atmospheric noise reduction is very much a secondary function with these units.  I still have my Timewave ANC-4, but I haven't used it since last year.

73,
Ellen - AF9J

[K1JJ]

Very interesting on the MFJ noise canceling box and the TimeWave. Tnx.   

I was aware of these units but never thought of them for lightning-noise reduction use.

I can see how they would help under certain conditions. One potential issue is if the T-Storm is in the direction of the desired signal. And, a local storm may be so high angle, there is no directivity to work with.

After using an Adcock DFing array on 75M years ago, I found that signals within 100 miles were too high angle to DF. But lower angles were quite easy to DF.  Same with canceling distant T-storms, I'll bet.

So, maybe a mild combination of the two... a cancellation unit in the front end RF and the DSP software in the I.F. might help to improve things at times. Even a slight improvement is an edge and can make the difference between hearing the next lower layer of stations or not.   Trying the various 75M antennas phased against each other and the beverage too, might yield some results.

This stuff will be fun to mess around with it. Lord knows we will have plenty of lightning to work with over the next five months.

T

[k4kyv]

oh yeah, and as I found out the hard way, they use RoHas solder in them)

I assume it's the lead-free variety.  What was your "hard" lesson?  Trouble with tin whiskers?  Couldn't you go ahead and make the mods using ordinary solder?

[AF9J]

Hi Don,

No.  My problem was the higher temp required to work with RoHas (bismuth) solder.  I couldn't even use my 30W soldering iron. I had to use my 100W gun.  And even then I had to just about mash it into the solder joints for about twice as long as I usually do with an iron, before the solder would melt.  I'm not wild about soldering like that, on a circuit board.  At the present time, my Timewave ANC-4 is sitting in its box, until I decide to get around to repairing it again.  BTW, the Timewave, and the MFJ are not cheap. The MFJ is about $160, and I coughed up $200 (based upon the good reviews people gave the ANC-4, and the fact that JRC designed it [they make some good stuff, albeit pricey]) for my Timewave ANC-4.  Some days it worked great, other days, mediocre.  The Timewave and MFJ units work better against fixed manmade noise sources, than variable atmospheric noise sources.  The ANC-4 also comes with a telescoping vertical, you can screw into the housing of the unit.  It can also be used as an active receiving antenna, thanks to it's built-in sensing antenna pre-amp.

73,
Ellen - AF9J

[KF1Z]

Frank: I went to the HPSDR site and it's a maze.  What is the model/name or link to this $16 455 IF interface I've been hearing about?

The HPSDR site is a maze.... and the entire unit for RX will run over $700.

The "$16" version is the Softrock lite.

Information is scattered at best...

The Main site is yahoo group..   http://groups.yahoo.com/group/softrock40/

HOWEVER... The kits may no longer be available!!!!

"04/22/2008

To: softrock40@yahoogroups.com
From: "kb9yig" <raparks@ctcisp.com>
Subject: [softrock40] SoftRock Lite kits now sold out


Good Morning All,

With SoftRock Lite kit orders received through April 21 my supply of
SoftRock Lite circuit boards is at a level where enough boards are
left to fill existing orders. A total of 1700 SoftRock Lite kits
have been sold and the kit is now retired with no additional orders
to be accepted."

.........

Read more about that, and the new version of the kits to be sold  go here....

http://www.softrockradio.org/SoftRock

[K1DEU]

I own the MFJ-1026

Here is weak signal DXer Tom W8JI's comments and mods for the MFJ 1025/1026.

    http://www.w8ji.com/noise.htm

    http://www.w8ji.com/mfj-1025_1026.htm

Posting with the New Ubuntu Linux  Open Source 8.04 and dual booting with (yuck) WinXP. 73  John

     http://www.ubuntu.com/

[WA1GFZ]

The present HPSDR takes the place of a sound card then you use a softrock to make I/Q. Soon the softrock and RX will be eliminated with mercury. The TX card should be in the mail within 2 weeks. Mercury is  very similar to QSR1

[K1JJ]

Never Mind....

[WA1GFZ]

That's all you need Tom. Put out a request on the "softrock40" gang and you might get lucky.

[K0ARA]

 T
I've got a 40m, 80m, & a 160m kit. I  just don't have time to build with summer coming. LMK
                                                                        73
                                                                        Mike
                                                                        KA0ARA

[WA1GFZ]

So Tom pick up one of these and contact Tony Parks. He will send you a crystal for the IF you want to demodulate.

[AB2EZ]

A thought experiment:

Is there a way to build in protection against lightning crashes, without going all the way to the use of digitized audio (with forward error correction)?

As a hypothetical example:

Using an SDR transmitter, place the analog audio on the USB, and place a 1-second-delayed version of the analog audio on the LSB. Include the carrier.

An ordinary receiver with a sharp filter would be able to receive this signal in AM mode or "exhalted carrier" mode, or synchronous detector mode (if it has a synchronous detector).

A special receiver, that receives and separates both the upper and lower sidebands (an SDR with the appropriate DSP-based demodulator)... and which includes a 1 second delay... would be able to insert the audio recovered from the upper sideband or the lower sideband... depending upon which one is the least noisy (which one doesn't have a lightning crash on top of it).

Again... just a thought experiment.

Stu

[KF1Z]

BTW, anyone have a SoftRock Lite for sale?  (I think that's all I need to get started with 455 IF DSP off the computer)

T

Tom, I've got one...

It was for 8.83mhz...

The basic circuit is all the same...
Youd have to change the xtal, and the bandpass filter components to those for 455khz.
One toroidial inductor, one transformer, and 2 or 3 caps....

PM if ya want....

[K1JJ]

Thanks for the offers, guys.  First let me see exacly what I need...

I just checked the FT-1000D and realized the jack output is for 73.62 mhz. They call that output "for a monitor scope."

Checking the specs, I see they then convert from 73.62 mhz down to 8.215 mhz, 455 kc and 100khz.

The filters I have in it are 8.2mhz filters.  (500hz, 2.8khz, 10khz, etc)

At what I.F. freq should I tap into the rig to best use the SoftRock Light board for this DSP lightning static processing attempt?

Bear in mind I will also be attempting to use the MFJ 1026 phasing canceller box ahead of the RF front end to do some static canceling. The DSP I.F. stuff is strictly to try to pull some more intelligibilty out, if possible.

I take it that the SoftRock Lite boards are mono-band... ie, I need an 80M board?

Tnx.

T