We've had sporadic Altus Metrum customer reports about RF susceptibility issues with their TeleMetrum installations. In almost every case, these problems have been completely resolved by either making sure the system battery has sufficient charge before launch, or through the application of standard engineering techniques such as twisting wire pairs to reduce differential coupling. However, even when every technique we could think of had been applied, once in a while someone still had issues.

Around the time of LDRS this year, the incidence of such reports seemed to increase. One customer, in particular, had an installation in which he virtually always saw continuous resets of the board once his 54mm airframe was put on a launch rail, and several customers reported seeing board resets during ejection charge firing. Keith and I saw a board reset during main charge firing happen in person at NCR's Oktoberfest, and with a couple days available to work together after that launch, we decided it was time to figure out what was really going on.

Here's what we've learned.

In bench testing, it quickly became clear that the problem was the 3.3 volt power supply rail getting pulled down far enough to reset the CPU. This most frequently happened during ejection charge firing, when the input of the LDO regulator is pulled down by the near-short presented by the e-match when a pyro FET is turned on. To keep the 3.3 volt rail voltage up during firing, we include a 100uF bulk capacitor on the regulator's output. In all of our prior bench testing, we never saw the 3.3 volt rail droop significantly. Clearly something had changed... or maybe several things had changed?

The first thing I wondered about was whether the new Kalman filtering code, which requires more compute cycles from the processor, might be consuming enough more power to pull the rail down faster during charge firing. After poking around at it, though, we have no data to suggest the new code makes a measurable difference in power consumption.

The next thing we pondered was that at least some of the e-matches we and others are using in the hobby now come from the fireworks industry, where it is apparently considered a feature for the match to retain continuity after firing. This means the input of the LDO gets held down for longer than with the e-matches we used to use and Quest Q2G2 igniters that open when fired. But that still didn't make sense as the root cause, as we chose the FET firing time such that even with a dead short across the igniter terminals, the 3.3 volt rail wouldn't be pulled down far enough to cause trouble during firing.

One of the big changes between v1.0 and v1.1 on TeleMetrum was that the newer boards incorporate a better reset circuit. This helps ensure the GPS chip always comes up running at power on, which was a problem at temperature extremes with older boards. However, a side-effect of this change is that a v1.1 board will reset any time the 3.3 volt rail drops below 3.15 volts, whereas older boards didn't trip until a much lower voltage. So the recent increase in reports might just be related to more v1.1 boards being placed in service?

While experimenting on the bench, we observed that injecting RF energy into the input of the LDO regulator had the effect of pulling down the output voltage, presumably because the internal reference source accumulates charge and is fooled into thinking the output is too high. Since our designs all have the power switch contacts ahead of the LDO, the wires going out to the switch and back are effectively an antenna... as are, to a lesser extent, the wires going to the e-matches. There is some variability from part to part in just how badly the LDO reacts. But by attaching a tuned length of wire as an antenna to the LDO input and playing around, we were finally able to reproduce the problem reliably on a test board at my bench!

On further analysis, we realized that the output of the USB battery charger chip and the input of the LDO both expect a 1uF bypass cap to ground. At some point, those looked redundant and we eliminated one of the two. Unfortunately, we weren't internalizing the fact that the switch leads were between the two caps, and the one we left was on the output of the charger and not at the input of the LDO. Placing a suitable bypass cap right at the input of the LDO turns out to have a truly dramatic effect on RF immunity!

Once we realized that RF getting into the LDO input was the problem, Keith pointed out that we used to see "noise" in the accelerometer data on earlier boards that was caused by the 3.3 volt rail moving slightly during radio transmit, which we fixed with a hardware change on v1.1. We are now convinced that this was at least partly related to RF coupling to the LDO input, not just the change in power consumption on the LDO output. We didn't realize what was going on in earlier testing because we often didn't have ematches wired up, so RF coupling was minimal. But going back to flights logged with v1.0 boards that included deployment, and studying the magnitude of the "steps" in acceleration data observed when the transmitter was on, Keith was able to compute the amount the 3.3 volt rail must have sagged if the real acceleration wasn't changing... which in some cases was as much as 180mV! We think this proves that RFI could cause the LDO to drop its output voltage below the reset controller set point on v1.1 boards.

Based on these observations, I'm making two hardware changes for the next version of TeleMetrum (version 1.2), and Keith is also making a software change. We have tested all of these changes on real boards both on the bench and in test flights, and the net effect is a major improvement in the RF immunity of TeleMetrum.

The first hardware change is moving to a slightly lower trip voltage on the reset controller. Instead of 3.15, the new part trips at 3.00 volts nominal. This gives us more "headroom" to tolerate 3.3 volt rail droop during charge firing, and will allow the board to operate longer on a given battery charge. This change is not relevant for v1.0 and prior.

The second hardware change is adding an appropriate bypass capacitor right at the LDO input. This requires a PCB update, but it's possible for me to update existing production boards by adding an 0402 cap right across the appropriate pins on the regulator chip.

The software change prevents our altimeters from turning on the radio transmitter while an ejection charge is firing. Since the RF transmitter draws substantial additional power, this should help keep the 3.3 volt rail from drooping. This may not really matter, but it feels like the right thing to do. This change will be part of our next stable firmware release.

We think most TeleMetrum customers need not worry about these updates. But if you have seen odd resets on the rail or during ejection charge firing in flight with a TeleMetrum v1.1, feel free to contact me about updating your existing board to include these improvements.

Keith and I released version 1.0.2 of AltOS, the open source firmware and software system associated with our Altus Metrum hobby rocket avionics systems.

This release includes a fix for the defect in version 1.0.1 that prevented rebooting an altimeter from idle to pad mode over the radio link. This affects both TeleMetrum and TeleMini boards, and re-flashing the altimeter firmware is required to pick up this fix.

We also restored the pre-1.0 mode selection behavior for TeleMetrum at power on. This is also an altimeter firmware change, but does not affect TeleMini.

Keith and I just released version 0.8 of AltOS, the open source firmware and software system associated with our TeleMetrum hobby rocket avionics system.

The AltosUI ground station user interface is significantly improved, with lots of new features including a much cleaner view of what's happening during different stages of the flight experience, a post-flight data plotting interface with pan and zoom, and a very cool live view of where the rocket is overlayed on data from Google Maps! All existing TeleMetrum owners are encouraged to download, install, and begin using this new version immediately.

About a week ago, Altus Metrum announced that our 2nd generation ground software was available, and that it runs identically on Linux, Mac, and Windows computers.

Thanks to feedback from early users of this new software, particularly those who flew rockets at various sites over the holiday weekend and reported on their experiences, we're found and fixed several bugs, and made a significant improvement to the Windows installation experience.

Version 0.7.1 is now available for download from our AltOS page. All existing TeleMetrum owners are encouraged to download, install, and begin using this new version immediately.

Exciting news! As of today, TeleMetrum starter kits are available through Apogee Components in addition to the Garbee and Garbee web store!

Contratulations to Bob Finch and Nathan Dalrymple for becoming our first customers to fly a rocket using a TeleMetrum board last Saturday! The rocket was Nathan's stretched BSD Horizon flying on a CTI Pro29 6xl 305-H226-14A Skidmark motor as shown in this launch photo.

I met both Bob and Nathan at the monthly club launch run by the Albuquerque Rocket Society at the same site in Rio Rancho, NM, earlier this year. That was the day I flew my first test flight with on-board GPS. They got excited about what they saw, and became two of our first customers for production boards.

Other than some rough edges with our first-generation ground software, Bob reported that everything worked as expected, and the GPS location received in the downlink got them to within about 10 feet of where the rocket set down.

In communicating their success to us on IRC, Bob said "MANY thanks to you guys, not only for the product but also for all the help getting me to the point of being able to use it !!". The pleasure is all ours, though, because after Bob asked us lots of new-user questions online, he wrote up his notes for us as "documentation for mere mortals" that will soon form the basis of a "getting started" section in our user manual.

Pretty cool stuff!

First, I was pleasantly surprised to discover that TeleMetrum is now the subject of a post on the Make magazine site. Very cool!

As reported last week, I've been waiting for better GPS antennas to arrive, but heavy snow here in Colorado last Friday delayed delivery. They finally arrived, and as I hoped, they appear to completely solve our signal strength problem!

So... [drum roll, please!]... I am very happy to announce that the first production build of TeleMetrum boards and starter kits are now "in stock" at the Garbee and Garbee web store, along with other products designed by the Altus Metrum community!

Keith and I are still working on the new Java-based ground station software and user documentation for the system. Watch this space for updates. I hope we'll have both ready for download by the time customers actually receive hardware...

I'm pleased to announce that the new Garbee and Garbee web store offering products designed by the Altus Metrum community is now open for business!

Rocketry folks, please note that I refuse to sell anything that isn't in stock, so if you want TeleMetrum boards you'll have to wait a few more days. Keith and I just weren't happy with the GPS receiver performance using the patch antennas we selected for version 1.0, and I'm changing to a slightly better (and more expensive) antenna that we expect will make everyone happier. Parts are due by Friday, and so with any luck you should be able to buy TeleMetrum boards and starter kits early next week.

In the meantime, we have TeleDongle boards and other accessories in stock now and ready to ship. Last week, Steve Conklin became our first customer by showing so much enthusiasm for TeleDongle over snacks at a conference we all attended for work that we pointed him to the web store under construction and he has already posted an enthusiastic blog entry about his plans!

More as it happens!

The first production run of TeleMetrum boards arrived from our assembler today, and overall things are looking good!

Unfortunately, I specified the wrong value for a capacitor associated with the new and improved 150mA 3.3 volt regulator. The resulting symptom was interesting to debug... everything seemed to be fine except that the GPS chip wouldn't talk to us. After some investigation, it became clear that the 3.3 volt power supply was taking much longer to stabilize than it should... long enough that the power-on reset circuit was relaxing before the supply was stable! The cc1111 apparently handles this just fine, but the GPS chip doesn't. Since the new cap is optional, just removing it caused everything on the first test board to start working! The ultimate solution will probably be to both replace the new bypass capacitor with one of the correct value, and to swap out the cap in the reset circuit for a somewhat larger value to ensure we have plenty of margin in the reset circuit.

I now have 4 of the new boards completed, turned on, and passing initial tests. Hopefully tomorrow I'll be able to wrap up the rest of the required functional testing. I then need to focus my attention on a business trip all of next week... but if all keeps going well, we're very close to taking our first orders.

Stay tuned!

I've been too busy (work, family, and fighting a nasty cold!) to write much text lately, other than the comments I put on launch photos uploaded to my Flickr stream, but the remainder of our testing of the prototype TeleMetrum v0.2 boards went really well!

The only significant change we decided to make before going to production was to change the 3.3 volt regulator from a 100mA to a 150mA part. This will ensure adequate power for the companion boards we have planned, even when the GPS chip is in maximum power mode searching for satellites during a cold start.

So... [drum roll, please]

Keith and I just placed the orders for our first production lot of TeleMetrum v1.0 boards! With any luck, in 3-4 weeks we should have a pile of altimeters to sell, along with the associated TeleDongle ground stations already in stock.

Stay tuned!