Interview with Gary Crowell

Featured Engineer

Gary A. Crowell Sr., P.E., CID+

Interview with EEWeb Member and EMC Engineer Gary A. Crowell Sr.

What are your favorite software tools that you use?

I’ve done PCB layout on probably 8-10 different tools, and multiple versions on most of those. I currently use PCAD2006 most of the time, and it suits what I’m doing very well. Many lower end tools are too hand-holding, and fight you when you try to do something unusual. The higher end tools require things to be done a particular way, and they also fight the unusual tasks. Most of the boards I’m doing at the moment are test boards that often require odd copper structures that are out of the ordinary. PCAD seems to let me get away with anything with the least struggle. Of course PCAD is now unsupported, dropped by Altium in favor of Altium Designer. I’m using AD tentatively, and sometime I’ll probably dive into it fully, but I expect I’ll always keep PCAD on the side.

What are your favorite hardware tools that you use?

Favorites change. Back in high school, just about my only tool, and thus my favorite, was a Heathkit VTVM. For the younger set, that’s a Vacuum Tube Voltmeter. Before everyone had a DVM with a FET input, you had to have a vacuum tube to get a high impedance meter.

When I first got to start fiddling with digital electronics, it was with a Heathkit H8 computer (that the GI Bill generously bought for me). For that, I built and debugged DRAM and several other cards with only a Logic Probe – so I guess that at that time it was my favorite tool. It was also a tremendous education in digital logic and troubleshooting methodology. With only a little blinking light to go by, you really had to think about what was happening.

As the lone hardware engineer at a limited budget start-up, the best tool was an HP combination oscilloscope and logic analyzer – I forget the number. They could be cross-triggered, and were just the thing at that time.

Lately, sadly, my favorite tool is a magnifying glass. I just can’t see well enough to do much work on the bench.

What is the hardest/trickiest bug you have ever fixed.

The next one. But really, you’ve made me think about how many boards and systems I’ve debugged over 30 years. Makes it hard to narrow it down to one system or bug. (Long pause.) Ok, I’ve just thought of the most interesting bugging task I’ve had. Not a typo; I said bugging task. At Huge Aircrash (aka Hughes Aircraft) in the ’80’s, the system I worked on was the MiniPro Signal Processor; a unit that was used in the Navy AN/BQQ5 sonar and the ADCAP Mark 48 torpedo. I got to write the repair manual for the MiniPro. Much of that involved inserting faults into the system (literally opening or shorting H/L every pin on every card), running the test suite, and documenting the results. I also had to explain the logic behind the fault, why it failed at that point in the suite, and create a troubleshooting flowchart. Some poor Navy pukes had to use that manual to repair the MiniPro. If any of them are out there, let me know that worked out – I never got any feedback on it while I was at Hughes.

What is on your book shelf?

Top of the list would have to be “High Speed Digital Design: A Handbook of Black Magic”. It totally changed the way we did digital design, and arrived at just the right time, as SI problems were starting to crop up in our projects. SI gets more respect now, but in the mid ’90’s it was still new to most of us. Any of the Signal Ingegrity books by Ritchey, Bogatin, Brooks, and Montrose are also good. I keep a pretty extensive list here: http://tinyurl.com/2ads6tq

For more general project skills I recommend “Developing Projects in Half the Time” by Smith and Reinertson, and, “The Goal” by Goldratt. Actually, I consider both to be pretty much required reading. Actually the ‘Developing Projects’ book might be somewhat dated now. I’d be interested in finding something similar that is more current.

I’m currently reading “Aftershock: The Next Economy and America’s Future”. I can’t wait to see how it comes out.

Do you have any tricks up your sleeve?

Unlike software, where you can edit away your mistakes, any mistakes you make on a PCB board file are going to come back in copper and fiberglass to proclaim your error to the world. I prefer to avoid that whenever possible. I mainly work with PCAD, but with any layout tool, even after you’re run the normal DRC and DFM checking, when you hit that final key to send a layout to a fabrication there is always that nagging feeling that something is still wrong. I ease that feeling somewhat by reading the Gerber files into a CAM tool (in my case Camtastic – part of the Altium package), and extracting the netlist from the Gerbers. Then I also import the netlist from PCAD, and run a netlist compare on the two. The board house is probably going to do this same check anyway, and by doing it myself I can avoid ‘the call’ (the one that says your board is on hold). There must be a perfect match or you have to explain each discrepancy before you can proceed with fab. While it’s in Camtastic, I also take the time to step through all the netlists and just take a good last look at each highlighted net, and all the rest.

What has been your favorite project?

In the early ’90’s I worked for a startup that was working on a database processing system. This was an array of processors, each with disk storage, with the array processors linked as a torus and the database distributed across all the disks. It was an ambitious project, and it could have been a hit, but in hindsight I believe it was probably started five years too late for the market; it got overwhelmed by larger disks, NAS, and network distributed processing. The system we built for development was housed in three equipment racks, holding 100 processors and 200 disk drives. It was quite a trip to watch the thing. Starting it up, the disk drives had to be spun up sequentially to keep the power draw within reason. When it was issued a query, starting from the I/O node there would be a wash of activity lights sweeping across the array. Nerd city lightshow.

There is no question that I learned more from that job than any other. So many things that I had to figure out how to do for myself (as the only hardware engineer there), and so many lessons about new product development – many learned the hard way. Some of those would be an entire other article.

Do you have any note worthy Engineering Experiences.

While not exactly engineering, the most interesting experience I’ve had was while working as a crypto maintenance technician in the Air Force. I was stationed at Elmandorf AFB in the early ’70’s, working in the main comm center in the basement of the Alaskan Air Command HQ building. That building was a large concrete monolith, so the basement was a really interesting place to be during an Alaskan earthquake, standing between rows of multiple 800 lb. equipment racks swaying back and forth. I’d still be under there if it had been the big one.

Elmandorf is right outside Anchorage, so while it was considered an ‘overseas’ assignment, it wasn’t a particular hardship. My wife was there and we lived in an off-base apartment. However Alaska had a dozen or more remote radar stations, and an assignment out there was an entirely different story. I never had to do one of those year-long remote assignments, but the tech’s from Elmandorf regularly had to do short ‘emergency maintenance assist’ visits to those remote stations. The remote stations usually had only one crypto technician, who might be overwhelmed with a problem; or maybe no tech at all, if they were between assignments. We did these assist trips in rotation, and when my number came up, I was off to King Salmon Air Force Station.

KSAFS was only about 150 miles from Anchorage across the Cook Inlet, and there were no roads there. You had to fly in. So I got my first military helicopter ride, in a CH-53. As we were flying across the water of the inlet, the Crew Chief was doing normal stuff, opening interior access panels and peering inside with a flashlight. Soon enough we were over the inlet and descending. Funny, no airfield in sight… and we landed in a field. Middle of nowhere. Turns out an exterior access panel had come unlatched, and the latch was busted. We sat around while they removed the panel, then we had to fly directly back to Elmandorf.

Round two. The next available flight was a C130, and that sounded a lot safer to me. The part I didn’t catch was that they were doing a navigation training flight, with KS as the ~~last~~ stop. 5 hours rattling around in a C130 to go 150 miles was not pleasant.

King Salmon was a radar station only; there was no nearby town or village. And it was in two parts. The base camp was the support area, and several thousand feet up a mountain, the top camp was the radar station. Top camp was mostly isolated during the winter, so I was sure glad it was summer. So up the twisty, windy road I went, to the camp where I’d be stationed until I got their crypto systems back in shape.

Top camp housed about 300 people, in buildings that were interconnected by enclosed corridors so that you never had to go outside. Of course there wasn’t anything outside but wind, rocks, an incredible view, and the wind. The next hilltop over though, had a BMEWS (Ballistic Missile Early Warning System) radar station and I got to go take a look at it. That was amazing technology, and I was surprised to find it run entirely by civilian government employees – no military.

Not exactly cutting-edge technology, the King Salmon crypto consisted of a pair of KW26 transmitter/receiver pairs, plus teletypes. This was a land-line encrypted teletype. The KW26 used tubes, and magnetic logic (ping-pongs instead of flip-flops), and that monster is a story in itself. But the immediate problem was that one system was down, so they had no redundancy; if the other one failed, they would have no secure communications at all. But this was actually pretty simple. The KW26 was old, but it was still around because it never really failed. Replace the driver tubes, and dust it off (or hose it out), and it was usually working again. This was the case here, and it was all running in a few hours.

However, in looking over the station I realized that they had a much larger problem. A station usually had a patch panel so that any Tx/Rx/teletype could be paired with any other Tx/Rx/teletype. With what they had, they could patch in one stack, or the other stack, but no interconnecting of the two. This was severely limiting, making it difficult to troubleshoot, and limiting their redundancy. I could have left. I had done my job. I opened my mouth instead.
I volunteered to rewire their station. I had never done this; all the stations I had encountered had been installed with patch panels. But I pretty much knew what it needed, and all the parts were there. But I had to get permission to take their entire crypto off line overnight. But I had to guarantee the station commander that he would have it back when I was done. But… my butt. What was I thinking? It’s the curse of the engineer – it was just too interesting of a job to pass up.

But actually, it all worked out fine. It did take all night, but the greatest difficulty was in trying to get the teletype technician out of bed to come rewire his teletypes. I finally pulled out the manuals and did it myself – just one more thing that night that I’d never done before.