Return to flip book view

VT BlogRobert Furmanak’s LinkedIn Postings about the Smart UPS VTFrom GreenLightUPS, Inc.

I continue to hear about how disliked the VT series is. They are considered difficult to repair, and impossible to field service. However, I have found them to be not that difficult. Here are the top reasons why I like them:1: They don’t blow up! The American version of the VT uses a 225v architecture, as opposed to the 400v found in many other units. My experience is that there is a huge difference in failure modes between 225v and 400v. I have yet to see a blown VT power module2: They have a fail-safe mode. A backup power supply for the bypass ensures that the critical load isn’t dropped, even if there is a complete failure of the UPS.3: They tell you what is wrong with them. The raw data contains information about 103 operating and failure states.4: They can be repaired in the field. With diagnostic tools, a repair kit, and our hydraulic lift table, I estimate that over 50% of the broken units we receive could have been field repaired in less than 4 hours.5: Modular construction. It is rare for me to “total” a VT. In contrast, we scrap a large percentage of other models of UPS.In conclusion, please don’t scrap that VT! Instead, send it to me!In defense of the APC VT.

We had a rare request for two VTs in parallel. While the parallel capability has been available for many years, it never became a popular option. This is the first request we have had in years. As a result, we had to relearn how to do it. The first thing we learned is that APC does not sell a complete parallel kit. What they do sell should be labeled a parallel expansion kit, for it includes only one adapter box. You need to buy two kits, which means you purchase twice the number of cables that you need. While APC offers the kit for sale, no instructions are included with the kit, and the installation manual only provides instructions for routing the wires, not connecting them. Plus, the operator's manual provides no information on setting up parallel systems.Fortunately, it is not difficult to set up and is very forgiving of mistakes. Improper programming or communications connections will simply result in an error message. Unlike Eaton's 9155s in parallel, there is no master interface. Each VT continues to appear as a standalone system and the only function that is coordinated between units is the bypass command. While this would not be our first choice, it is a viable option for VT owners who want to expand their system.Operating APC SUVTs in parallel.

Keeping busy with the SUVT.Three units arrived on the same day last week, which made for a busy day. With the SUVT, diagnosis is typically easy, and many repairs are straightforward. Eve the worst case in this bunch, which had a flashover due to an excessive amount of dirt, was easily repaired.One thing I appreciate about the SUVT is that APC made improvements over the years as feedback from users was received. One of the largest problems had to do with what happens when dirt accumulates inside the unit. It can create a conductive path which results in the damage seen here. APC made 3 significant improvements to reduce this problem: They added an insulating sheet between the boards, coated the boards with Paralyne, and changed the jumpers to an insulated design.Even with these improvements, SUVTs in dirty areas should be periodically opened up and the fans removed for a thorough cleaning.Keeping busy with the SUVT.

I decided to rebuild an entire 20kva SUVT using Humiseal on all of the main boards. Apart from the flashover damage and a considerable amount of dirt, the unit was in good condition, and worth the restoration effort. One downside to the SUVT design is that it has a virtual forest of connectors and irregular surfaces spread over a large area that is only 15mm high. This is an area that is frequently neglected, for to clean it properly, one would have to remove the power module, take the cover off, and then remove the fan assemblies. Even then it can be difficult to clean this area.I am hoping that the coating provides two benefits: Preventing flashovers, and creating a smoother surface that dirt has a more difficult time sticking to.Board Protection: Humiseal Part II

An APC SUVT came in for repair recently. This unit was completely dead, which is usually an easy to diagnose problem. But not this time. The usual suspects were all in good working order, but the unit remained unresponsive.That left only one possibility: That there was something wrong with the circuit board itself. I traced the problem to a spot where three tiny vias connect a trace on the top with a trace on the bottom. Somehow these had failed, even though there were no other problems. Perhaps they were simply too weak to handle the current on this circuit, which is fused for 16 amps. I bypassed the vias with a piece of wire connecting the battery charger heat sink to the fuseholder. This can be done without removing the board from the unit, which is a complicated process. The unit now runs well, having needed nothing more than 5 centimeters of wire to fix it.Repairing the box by thinking outside it.

The static switch board remained the last part of the SUVT for me to reverse engineer. These are buried deep inside the unit, and require nearly everything to be removed to access. They are also difficult to take live measurements on due to the lack of access. When they fail, it is nearly always an obscure logic failure, and not an easily identified power side fault.I reverse engineered it so that I can develop a diagnostic tool for it. I found the design very interesting, especially in the use of a ripple counter to provide a variety of clock frequencies which are then used for non-digital applications like powering the isolation transformers in the SCR driver circuits. Most interesting to me is what appears to be a quick start function created by idling the SCR drivers at a lower frequency, and resultant lower voltage. This reminds me of the final days of vacuum tube televisions, when "instant on" was introduced by leaving the valve filaments slightly warm when the TV was off.Reverse Engineering the APC SUVT static switch.

A good client of ours must have heard that I was on limited duty for a while, for they sent me 5 SUVTs to repair. These were enjoyable for they presented some unique challenges. Most had issues with their controllers, and reported many errors that initially didn't make any sense. I had to dig deep into the logic to find the answer. Three revelations led to a breakthrough. First is that the SUVT will report errors even when those features are not being used. For example, faults in the parallel system will appear even when the units are operated alone. What made diagnosis difficult is that it is not always obvious that these are parallel system faults. The second revelation is that if a bad opto-isolator chip is found, there are probably others. I'm not sure why this is so, but it is rare to find only one bad chip. Finally, I realized that the RS485 system is apparently being used only to send simple on/off signals, and not data. (Data is sent over the CAN bus system.) This explained why a stuck opto-isolator would give a false "in bypass" signal. Much still needs to be proven, but these theories fit the faults I am experiencing.A breakthrough in understanding the SUVT controller.

I received an APC 30kva SUVT with a problem that occurred very infrequently, which made it difficult to find at first. It would not start, and if it failed, it would report one of 5 different errors. That added to the confusion.The breakthrough in understanding this problem was realizing that the input contactor was not closing because the controller discovered a PFC fault, but because the controller did not realize that the contactor failed to close, and tried to start the PFC circuits anyway.The next step was to identify the fault with absolute certainty, so I could be confident that it was truly fixed, and I wasn't just experiencing a random period of correct operation.The final step was understanding how the fault actually occurred. In this case, it is possible that it was defective from the beginning. In normal operation the contactor operates only rarely, and once closed will stay closed. Even though the movable contact in the auxiliary switch was turned sideways, it would still operate most of the timeIntermittent problems are the most difficult to solve!

Our SUVT repair business is growing rapidly, and I needed a faster way to test the power modules. They are impossible to test inside an assembled unit and very little can be done on the workbench without the proper sources of power, and a way to generate control signals to test its operation. I designed a tester that attaches to the top of the power module in the exact same way the power module connects to the main board in the UPS. I can connect a power module to test within seconds, and test all the functions in about a minute. As an added bonus, all testing is done at a very safe 24 volts instead of hundreds. This is possible because an IGBT does not care about the voltage it is operating at, as long as it is above a threshold of a few volts. This test is strictly for testing whether the various parts are functioning, and not for simulating a UPS in operation. However, if it passes these tests, it will likely run when installed in the UPS.On the first attempt at using it, I successfully diagnosed a dozen power modules in about a half hour. I also learned a lot that I can incorporate into the next generation.A Fast Way To Test APC SUVT Power Modules:

The tester performed very well for an early prototype, and I learned what I needed to make the next generation board.What makes static switch testing complicated is that it uses a set of digital frequencies to communicate to and from the controller. In addition, they are sent as a pair of high and low level signals. Using frequencies as signals serves as a way to verify that both the static switch and controller are active, and not frozen or failed, in a way that a simple on/off command could not. The static switch sends either a 16kHZ or 24kHZ pulse stream, while the controller sends either 31.2kHz or 62.4kHz. The two values represent the control states of either online or bypass. To provide a simple visual indicator, I divide the pulses by 1/131072. This slows them down to blink the LEDs at about once a second when online or half that when in bypass.This early version can only monitor a switch installed in a running system, but the next one will run the static switch independently. It will incorporate a local oscillator and power for the board along with a few other enhancements.My first generation APC SUVT static switch board tester:

One of the challenges to reverse engineering the ABus portion of the parallel system is a limited amount of documentation, and a lack of peripherals to connect to the ABus to test it. Fortunately, we just received a never-used Philips Fluoro UPS VT system which has ABus peripherals to monitor the transformer and external fans. Ironically, we received this unit with the parallel system gone! Someone in the 2 years this has been sitting in some warehouse somewhere decided they needed one. That merely added incentive for me to complete development of my parallel kit.One part of reverse engineering is the hardware, and the other part is studying how the equipment behaves under different situations. This meant discovering what indications there are of proper or improper operation. In this case, the VTs expect that if there is a peripheral on one ABus, there should be on the other. If not, it displays an ABus comm failure. The second failure is a lack of a terminator if there is nothing on the ABus. This is required even if there are no peripherals on either VT.The Philips system is now for sale, and may be an ideal solution for other medical imaging applications that require international 400v power.My APC SUVT Parallel kit, now with ABus capability.

This one realizes my goal of running the static switch board on the bench, independent of the other parts of the UPS and the main controller.My tester mimics the commands sent by the controller, and the static switch controller responds as it would when installed in the UPS.The control scheme is simple enough. The main controller sends a set of pulses at either 16 or 24kHz and the static switch responds with either 32 or 64kHz. Other commands like EPO or external bypass are simple ON/OFF signals.I used a Linear LTC6909 oscillator for it has a built in frequency divider from 3 to 8. This enabled me to easily get a 2:3 frequency ratio, which I could then divide from 768 and 512kHz to 24 and 16kHz. I then used a pair of NPN transistors to generate high and low signals with opposing phases. Both the input to the SSW and the output use this method, which is why there are 2 LED output lights on my board.I had six boards of unknown quality in my inventory, and this enabled me to test all of them in a few minutes. Four passed, one was completely unresponsive, and the other turned on but did not respond to commands correctly.My second generation APC SUVT Static Switch Tester.

With this generation, I've accomplished my goal of having a full featured and easy to use tester. While the first two generations demonstrated proof of concept, they fell short in a few spots. First, some tests required pressing 3 buttons at once, and the skill of a concert pianist to do so. Too inconvenient. Also, some combinations would overload the power supply and cause it to turn off. A combination of relays and diode logic circuits solved both problems. Now only one button is required for all tests, and pressing a positive button prevents its negative counterpart from being engaged at the same timeGetting a useful signal from the current transducers while operating at very low currents was a more difficult problem. The transducers are meant to operate at several amps, and have a built in 2000:1 reduction. I added analog amplifiers to boost the signal to a useful level. Not an elegant solution, and it requires a manual zero adjust, but it works. While it would be fun to make a highly sophisticated micro-controller based tester, a balance had to be struck between cost and return on investment. This tester is an economical and effective design for the job it is required to do.My 3rd Generation APC SUVT Power Module Tester.

While the second generation worked well, there is always room for improvement. I had to fix some minor errors on the circuit board with wire-wrap wire on Gen 2, and I also wanted to add some features. I have discovered that the static switch board can produce an intermittent error regarding the status of the emergency 15v supply, so I added a way to monitor the board via a data port. However, building the actual monitor will have to wait for now. I also added an on/off switch to the local oscillator to simulate the effects of a failure of the control board.With this tester, and my power module tester, plus years of experience with the VT, we can diagnose any problem rapidly.My third generation SUVT Static Switch Board Tester.

While I have been repairing 208v SUVTs for many years, the 400v version is rare in the USA. The only applications appear to be when a customer has equipment that requires 400v, 50Hz, 3 phase power. In that application a 480v to 400v transformer is used to power the UPS. Recently a customer with 2 broken SUVTs came to us for service. Before I could repair them, I had to understand the differences between the low and high voltage versions. The most significant difference is in the inverters, as seen in the schematics below. The only interchangeable assemblies are the controller and the low voltage power supply. The controller has to be reprogrammed for the application, but is otherwise identical. I changed all the DC capacitors in both units, 64 in all. I located a replacement power module to replace the blown one, and coated all the resoldered areas with a conformal coating. These units recorded 7 years of running time, and I expect them to last several more years.Reverse Engineering the 400v APC SUVT.

Since about 2013, APC has been applying a thick conformal coating to their SUVT circuit boards. While this is a great help in keeping the delicate electronics protected from the environment, it makes troubleshooting and repair far more difficult. In this case, the attempt to remove the coated chip damaged the board beyond repair. Unfortunately, replacement controllers are not easy to find, so it was worth the effort to invent a solution for repairing the board.My idea is simple in concept. Just bypass the damaged area and connect directly the input and output. To make this simple to implement required two boards. One connects to the bottom of the output connector, where the pins are long enough to overlay the bridge board and solder to them. The second board holds the RS232 chip and associated components. It connects to the bridge board with 6 pins, and to the opto isolator chips with 6 blue wires. It is actually easier to install this kit than replace a coated chip.Repairing an APC SUVT control board