Repeater Lock-Ups

Many people have reported Fusion repeater lock-ups that required the repeater to be shut down (power removed) and rebooted. I’ve had repeater lock happen as well with an HRI-200 directly connected to the DR2X. In this particular case I believe the event was triggered by sending a number of commands from the radio (such as the GM mode) which resulted in a race condition within the firmware.

I have previously observed that invalid data (I.e., with a lot of bit errors or badly formed packets from a hotspot) can also cause the lock-up. My assumption is that this is a problem of not validating inputs to a function prior to executing the function. The invalid data is not handled correctly and causes one of the lock-up faults I mention below to occur.

I have observed that data is more likely to be invalid if there is a lot of noise on the signal. Also a major source of problems are badly behaving hot spots that inject all sorts of crap into the data stream. There’s no control over these devices and it just gets worse as there is more work being done converting from one thing to another thing, etc.

Communications software is EXTREMELY HARD to write and much of what I see out there is poorly documented (which leads to bad code) and poorly tested.

Reflectors such as YSF or FCS are pretty dumb when they relay signals. They look for an incoming packet from one of the connected hotspots and just send that same packet out to all the other hotspots (including bridges to WiRES-X). Garbage in, garbage out. To address this I have modified the YSF reflector software to be very aggressive at dropping packets with CRC errors, invalid data in fields, and most importantly invalid FICH headers. The FICH header is extremely important for everything to work right. (You can look up FICH headers in the System Fusion documentation published by Yaesu.) This modified software is running on MNWis, Princeton Kentucky, MVARC (Idaho) and America Link. It has dramatically reduced the number of problems with WiRES-X problems.

So what is a “lock-up”. A lock-up occurs when the processor looses track of what it was doing and either halts or ends up executing invalid code. In either case it is no longer processing the code it was intended to run. Events that cause the processor to, using a technical term, go out to out to lunch can result of errors in the software that cause it to happen, corrupted memory, invalid states in a state machine, incorrectly handled race conditions, or a really nasty thing called priority inversion (where a high priority task which is pre-empting use of the CPU is waiting for a low priority task to complete, which can’t complete because, well, it’s not high priority.)

How to solve the problem?
First of all, the repeater firmware should never crash. Since it is very hard (but not impossible) to write code with no bugs, most continuously operating real-time systems will incorporate a “watch dog timer”. It is the purpose of this timer to detect when the “wheels have fallen off the processor” and perform a reset.

Since we can’t modify the repeater’s software, there are some actions we can take.

First would be to prevent bad data from getting into the system. A careful examination of the logs possibly combined with audio recordings could be helpful.

A quick fix might be to install a timer on the power supply such that the repeater’s power is removed for a short time once or several times a day.

I have a technical concept for a generic lock-up detector that would reboot the repeater in the event of a lock-up without the need to make any modifications. Not sure if I’ll work on this because I don’t know how much interest there would be. This solution would also work with non-Yaesu repeaters.

One further note. I made an observation that this problem was more likely to occur on 2 meters than on 440. Why? My theory is that there is a lot more interaction with the environment on 2 meters. Computer networks, small power supplies, computers, almost everything digital generally creates a lot more noise on 2 meters than it does on 440 MHz. I observed a situation where mixing products were occurring on a 2 meter repeater in a noisy environment. This repeater would lock up regularly. My theory was that the mixing products contained a lot of noise along with the C4FM signal causing a high number of bit errors and thus crashing the repeater. This was observed on a DR1X.

If you are struggling with this problem, I suggest you contact Juan at Yaesu Customer Service with your details. Provide Juan with as much information as you can.




Running two HRI-200’s on one IP Address

I use VyprVPN from GoldenFrog. It is necessary to turn of NAT for your account. That means that each VPN connection is a raw connection straight to the Internet. YOU HAVE NO PROTECTION OTHER THAN WHAT YOU PROVIDE!!! (The same company will rent you a VPN that you can host on your own cloud provider where you can control port forwarding – but I suspect this is well beyond most folks.)

 
I’ve been using this solution for over a year. If you’ve ever seen me demonstrate WiRES-X, I’m either using AT&T/cell or the local Wifi along with the VPN to get access to the incoming ports.
 
Good news! My fully patched and maintained Win7 boxes have not yet been hacked. One of which has been online via VPN for over a year.
 
Most VPNs won’t work because they don’t give users their own IP. The VPN host shares IPs between multiple users which means they can support more users with fewer IPs (and better obscures your traffic). I’m only aware of two VPN providers that allow you to specify incoming ports.
 
The previous post of using the Ham’s IP space is a good one. I plan on giving that a try when I have time. But VyprVPN is very much plug-and-play.
 
Note: You will need to PAY for VyprVPN (~$70/yr). The free service does not allow you to disable NAT.



On Repairing PA’s of Repeaters and Radios

If you are experiencing low or no power output…..
 
1. Replacing a blown final does not necessarily fix the problem that caused the blown final. Never replace a blown final in anything where you haven’t gone through and figured out if there was a reason the final blew. These are very rugged transistors that should last a very long time.
 
2. Some failures are related to heat, most are not. The DR repeaters have excellent cooling. Unless the fans have died, temperature should not be a problem.
 
3. There have been, er, problems in the past. Being that I’ve been in Fusion since the very beginning, I have had a few radios where finals blew. The root cause many failures in the repeaters has been resolved a long while ago. DR2s never had any issues. Later DR1s were fine.
 
Suggestion: Add a 10 amp fuse between the power supply and the Tx. This can prevent burning of the PCB should the final end up in a “high current” mode. Not a likely event, but good insurance against a potentially much more expensive repair. (If the transistor is biased on with no RF input, the current draw will go to about 15.5A. All of that power is used to heat up the transistor and the PCB.)
 
4. The official Yaesu replacement transistor is not available on the open market. You must purchase it from Yaesu. I have successfully replaced PA’s, but be advised your specific non-Yaesu replacement transistor might have characteristics that won’t play well with the other components on the board.
 
My suggestion: Unless you’re really set up to do this stuff and are willing to match a replacement transistor to the circuit, let Yaesu do the repair. If Yaesu did not do the repair, we shouldn’t get concerned when someone reports multiple PA failures.
 
From another post:
First, replacing the transistor may not fix the problem. One has to wonder why it failed. These transistors should not fail under normal use (i.e., no lighting). High bias voltage can easily kill a PA.

 
I had a similar problem with an FTM-400. Blown PA, no bias. Forget what I did, but I could have kicked myself for not checking the bias and working on that problem before I put the new transistor in. As I recall I just worked my way through the bias circuitry and fixed what was wrong. My recollection is that the bias was bad on both the driver and the PA so I ended up looking at the common circuitry.
 
Note that the PA transistors you can buy from places like RF Parts are different transistors from those that Yaesu uses. For a number of years now they have been purchasing Mitsubishi transistors with a custom part number. My guess is that they have been selected for consistent parameters in the 430-450 range whereas the standard part is specified for 450-470. This means variation in the standard part may require fine tuning of the input/output circuitry.
 
I had a DR-1X who’s power output went to zero. Fortunately I had placed a 10 amp fuse in series with the Tx which blew and protected the PA transistor. (Otherwise it sits at 16 amps and melts itself.) The problem was a failing solder joint on an inductor that was used to provide RF isolation to the transistor that engaged the Tx low pass filter network band switch relay. It really helps if the Tx frequency is on the same band that the output filter is on! This is an example of a non-PA failure that could have killed the PA. These transistors should not fail and if they do there’s probably a reason.
 
Aside: In the above I ended up bypassing the relay so it always used the 440 MHz low pass since I only use the repeater on UHF. Figured one less failure point. Funny thing is that it worked fine on VHF (of course) but that the spurious emissions were still pretty low. It would easily meet FCC spec with a duplexer on the output.



Repeater Power Output

The DR repeaters are calibrated in the middle of the band. For UHF that is 440.000 MHz. You may want to check the power output at that frequency as you are operating right at the band edge. While you’re at it, measure the Tx current. At the lower power output the current should be lower, otherwise it could indicate a problem.

Under no circumstances should the Tx current be above 10 amps (I recommend a 10 amp fuse in the Tx power supply line) and will more typically be around 7 amps. At 30 watts it should be less.

There are differences in the DR-1X depending on when it was produced and/or which updates it has received. Early units should only be run on medium power, especially on UHF.

The DR-1 does not sense SWR. It simply looks at voltages in the output filter that should be represent a higher SWR as an apparent higher output power thus causing input power to be slightly reduced. Variations in the frequency response characteristics of the filters and amplifier gain will cause the output to vary over the 430-450 MHz range. You can always recalibrate the output at your frequency of interest but be careful. It has been my experience that the output can also be greater than 50 watts on certain frequencies.




Battery Hints and the FT-70

What’s up with the FT-70 battery indicator? Why does it seem like the battery doesn’t last very log?

The FT-70 doesn’t actually know the state of charge of the battery. It only knows the voltage – which can be misleading while the battery is being charged. The Battery Management Unit within the battery pack is responsible for managing the charging and discharging of the battery. While computers and smart phones generally use the BMU to display state of charge, I’ve never seen an HT that does that.

You are correct that the charging indicator does not show a battery level, it displays a timed barograph that slowly fills in since the HT has no data from the BMU.

Also note that the first several charge/discharge cycles on a Li-ion battery will be vastly different from the subsequent cycles. The first couple of cycles are need to condition the battery. Generally capacity will increase for the first few cycles.

To have the battery last as long as possible:
1. Don’t over discharge the battery. My rule-of-thumb is to stop using the battery (if convenient) when the Rx voltage reaches 7.4 V. At this point 80% of the battery capacity has been used. Most of the capacity is given up between 7.4 and 8.0 volts.
2. Don’t charge a hot battery. Heat is the enemy of Li-ion. It really likes nice, comfortable room temperatures around 70F/20C. Best not to leave it in the sun in a parked car!!!
3. Don’t overcharge. It’s fine to pull the battery out of the charger before it is fully charged. Generally the last 10% (8.0V+) goes slower because of the reduced charge current. Pulling it out at 8.0V will extend the number of charge cycles and have very little impact on stored capacity.
4. Avoid using 5 watts. At high power the radio generates a lot of heat – with the heat sink right up against the battery. It also places the greatest demand on the battery – especially if the battery is more than 70% discharged. The higher discharge current also creates more internal heating within the battery.




What Does the HRI-200 Actually Do?

What’s the difference between using an HRI-200 and directly connecting to the radio?

1. It is the only option for connecting a repeater to WiRES-X.
2. It uses UDP communications sent directly to the Room. So network communications are very efficient.
PDN uses TCP instead of UDP. (Why use UDP? UDP is simple. If anything gets lost, it is dropped. Good for real-time communications. If a packet is missed with TCP, it will keep retrying up to a limit which is not good for real-time. VoIP, VPN’s, Netflix, etc. all use UDP.)

The PDN TCP packets (non-HRI-200) are sent to a server in Japan. This server converts the packets to UDP and then sends them to the selected room in the same way the HRI-200 does. So by using the HRI-200 you bypass a drip to Japan and a potentially problematic conversion from TCP to UDP and back to TCP for the return trip to your station.

Yaesu uses the TCP approach to get around forwarding of ports. The WiRES-X PDN software establishes a TCP connection with the Yaesu server. It’s through that connection stations can contact you even though you don’t have an open port. With the HRI-200, stations just get your IP address and send you packets directly. This is why PDN cannot efficiently host a room – all room traffic would have to go to Japan for the conversion process.

In a previous post (2016) I described what the HRI-200 does. Why it is the way it is is somewhat historical from the history of WiRES, WiRES-II, and now WiRES-X. Obviously an Ethernet port on the back with an embedded processor to run things would be a better way to go. I can only guess that the ROI is simply not there for Yaesu. They do, after all, have to make a profit.

One further note. It’s probably a good thing non-Yaesu equipment doesn’t connect to WiRES-X. The biggest problem in bridging hotspots (YSF, FCS, etc.) to WiRES-X is that the hotspots are bad neighbors. They generate bad packets, do crazy things, and can really trash a network. As a room operator we have no control over who tries to connect, so at least the Yaesu equipment has gone through compatibility testing. Believe me, very little testing is done on hotspot software.

MNWis (21493) bridges to YSF (21493) via a YSF Reflector where I have extensively modified the code to drop bad packets, control information, and all manner of things that shouldn’t see there way into the WiRES-X node. I had to do this because we were constantly having hotspots connect and mess everything up. It got very, very tiring working with the (mostly new) hotspot owners to fix their problems. I mean the conversations were endless and constant! So I just dropped anything that looked bad in the YSF Reflector software and the problems went away as well as a few hotspots who could no longer connect or talk to people.

System engineering is critical. We need Yaesu to play the roll of system engineering to keep all the Fusion products working with each other. It’s so nice that all the Fusion equipment basically works the same way and does the same thing all of the time! 

BTW, you can bring your questions/comments to the MNWis Fusion Technical Net held Monday’s at 7:30 PM Central in the above mentioned locations.




WiRES-X Online/Offline

When a WiRES-X node keeps going online then offline, this type of problem is almost always the result of a networking issue. A networking problem will occur and will wack out the HRI-200 software. Once the software has been wacked out, it will continue to cycle in and out of whatever room it is connected to. Restarting the WiRES-X software usually solves the problem.

Router problems:
1. Turn off UPnP. It should never be used!!!!!!
2. Turn off all quality of service features like those to improve Xbox gaming or VoIP functions. (Yes I know we’re using VoIP so you’d think it would help.)
3. In fact turn off everything you don’t have to have.
4. Some routers are total crap and need to be rebooted every few weeks.
5. You’re not using Wi-Fi are you? Wired is always better than wireless – we should know! Drop-outs in Wi-Fi can cause this issue.

ISP problems:
1. Something as simple as the ISP dropping your connection for a few minutes in the middle of the night when they’re doing maintenance.
2. The ISP just doing a bad job of getting packets to you during prime evening Netflix viewing.
3. The ISP switching your IP address. (I think CenturyLink just does this for the fun of it.)

Computer:
1. Make sure Win10 doesn’t put the USB interface to the HRI-200 to sleep. It really, really likes to do this. Just going to the obvious place in the control panel to turn this feature off doesn’t really turn it off. See the documents on running Win 7/10 24x7x365 remotely in the Fusion Help section at HamOperator.com.
2. Make sure the computer isn’t crap. Cheap computers may not have quite the reliable communication interfaces that we need.
3. Banish all RFI and ground loops. Make sure RF is not getting into the USB interface to the HRI-200. This can cause message errors and really screw things up.
4. Make sure you computer actually has enough power to run the HRI-200 reliably. Get one of those USB power thingies and make sure you have a solid 5VDC under load.

Over the air:
1. Bad data on the WiRES-X network can mess things up. This happens mostly with hotspots that are bridged into the WiRES-X room. On MNWis and a few other networks we’ve banished this problem by banishing FCS and running YSF server software that banishes bad data from hotspots. So if you have this problem in one room, say AmericaLink and not others, there may be nothing you can do.
2. Yaesu radios that are running really old firmware or have not had the firmware updated correctly (I.e., “I did the main CPU but I’ll get around to the DSP later.”)

Side note:
The port check should not be relied upon to definitely prove it’s working or not working. The test is not exactly the same as actually communicating with the Yaesu list servers and the room server. So it can lie to you with false positives and false negatives.

You may also experience a similar problem if the Yaesu list servers are not able to keep up with the demand or if maintenance is being done. (When they do maintenance in the wee hours of the morning, that’s right in the middle of the day for us.)

And keep in mind the MNWis Fusion Technical Net held in MNWis WiRES-X and YSF # 21,493 Monday nights at 7:30 PM Central.




WiRES-X and USB Problems

If you’re having problems with USB problems when using WiRES-X with or without other programs, read on.

The WiRES-X software is pretty good at identifying the HRI-200 vs. connecting directly to the radio. Windows, however, is very bad at managing USB ports. The WiRES-X software makes two USB connections: One as a virtual COM port and the other as a sound device. Windows is also very bad at managing sound devices.

Virtual COM ports can be identified by the port number (a real pain as it is always changing – see below) or by identifying the hardware via the manufacturer and product number that is burned into the USB device. Generally software that does not provide an option to specify the COM port (as with the HRI-200) is using the hardware identification to establish the association.

When Windows sees a USB device it will enumerate it and create a driver for it. If you move it to another port it will, for reasons we can only guess at, create another driver. It will keep making new drivers as long as you have USB ports and USB devices left. Why, we can only guess. Periodically it may be necessary to go through a clean out all of these “drivers” that Windows keeps making and saving until the end of time. You can do this from the command line or from within Device Manager.

I have found WiRES-X to play nicely with other devices. For example I have no problem running hotspots, Ham Operator Deluxe, WSJT-X, etc. on the same computer that is running WiRES-X.

You may find it beneficial to use a USB 2.0 vs. 3.0 port. Why? USB limits the number of connections that can made per USB controller. Keep in mind that several of the ports on you computer may use the same controller. Also, each USB device may use more than one connection. As I recall sound devices generally use two connections: One for the audio and one for control. A USB 2.0 controller supports 255 connections. How do I know? I actually ran up against this limit using a couple of 16 port USB hubs. USB 3.0, while faster, does not permit as many connections. I don’t remember the exact number as it has been some time, but the number SIX sticks in my brain. As you might imagine, it is very easy to burn up six connections. Also keep in mind that the USB controller may be servicing hardware that is internal to the computer. It all depends on how the computer was designed. Top end computers will generally have a higher controller to port ratio. Bottom line: you may want to use USB 2.0 if you have been using USB 3.0.

Perhaps try something like this:
1. Blow away the WiRES-X and any conflicting software. Make sure everything is unplugged from USB.
2. Blow away all of the USB drivers. Just uninstall them. Don’t worry, the files to recompile a driver are still on the computer. [Device Manager with admin. Enable option to view all devices. Delete drivers under COM ports. Delete drivers under sound devices (except for hardware built into your computer.)]
3. Install the WiRES-X software. Allow it to associate with the USB devices it needs by plugging in the HRI-200, etc. If you’re not using an HRI-200, you may find it beneficial to use an external USB audio device for WiRES-X. If you can use USB 2.0 ports.
4. With WiRES-X happy running, install the other software. Plug in the necessary USB devices so it can similarly make the association. At this point everybody should be happy with both programs running just fine.
5. Write down which device was plugged into which USB port. In the future, always use the same USB port for the devices. We don’t want Windows creating more drivers.

Good luck solving the USB problems!