The radar unit was tested from cranes in Italy in static suspended tests and mounted on a helicopter in dynamic flight tests in Morocco (possibly also due to not that dissimilar terrain?).

See here for pictures of the flight tests.

Those event-triggered multi-threading systems are very hard to test and debug.
There may have been set a timer, which triggers the thruster shut-down if some event hasn't occurred within this time. If, for example the velocitiy variable is initialized with zero, and adjusted by the radar after an assumed time, a delayed setting of the variable may return standstill, although the variable simply hasn't been set. This could be resolved to some degree by adding an invalid flag. But how should the system behave in case of an invalid variable? Default would still be thrusters off, since waiting too long near ground with thrusters on would prevent a landing, and thrusters on during flight would be wrong, too.
There are certainly thousands of possible errors of this kind you don't see in a fixed set of simulations. With the actual data, they can now set-up a new series of unit tests to get closer to real conditions near Mars. And once the error is nailed down, some will certainly say, that this could have been known before. But before the real-world test, finding a needle in a haystack is much easier.
I'm wondering now, whether another test with a low-cost lander should be performed, or if the expensive payload should be risked without a prior fully accomplished landing test.

I thought the radar comes on line only ONCE the backshell have been released. Now, it's said that parachute and backshell have been released too early. I'm a bit confused...or wrong.

A sensible designer would never use such a structure for a flight control application. There have been well-understood flight control algorithms in use for almost 50 years that can be tested. Note that Viking landed on Mars with a very simple computer the very first time. http://history.nasa.gov/computers/Ch5-6.html

Just to clarify on the question of proving computer program correctness…

In the study of formal program verification, one seeks a mathematical-style proof that a program will correctly meet the formal requirements that are defined. This is, as was noted, quite hard to do for complex systems. Moreover, a program that is proven correct may be part of a failing system because the formal requirements are not an accurate statement of the real world application. For example, an airplane autopilot program that assumed that winds are always under 10km/hour might be proven logically correct, but lead to immediate failure because the real world has winds higher than that.

I doubt if any spacecraft failures take place because an algorithm could have been proven correct in the above sense, but the effort to prove it was not made or was not tractable in a computational sense. It's far more likely that ignorance, oversight, or negligence would arise in defining the formal requirements or – more likely still – the notion of proving an algorithm correct in the formal sense was never part of the development process in the first place, for the very good reason that it's usually not tractable for complex systems.

Formal program verification, as the literature defines it, is not a part of any software development process that I've been around. I'd liken it to a doctor trying to use the periodic table of elements to diagnose a patient's illness. Technically, an illness pertains to how your atoms are arranged, but that's not a useful level of description for a sick person, in most cases.

For example, the two problems that afflicted the Cassini-Huygens relay: (1) The Doppler shift in radio frequency because of the relative velocity of Cassini and Huygens during the mission was not put into the formal requirements. Moreover, empirical testing that would have revealed this would have been unthinkable. The algorithm was absolutely correct – assuming that Cassini and Huygens were not moving very fast. The fact of their high relative velocity during the mission was simply excluded from the design phase, but fortunately this oversight was caught before arrival at Saturn. (2) The software that was supposed to turn on radio receivers listening to two partially-redundant radio channels did not turn on one of the two receivers, which meant that Cassini transmitted it in vain, and half of the images that it took were never received, and lost forever. Here, I'm not sure that enough information was released to the public for us to say exactly what went wrong. It becomes a bit arbitrary as to whether the algorithm was correct, but the specifications written incorrectly, or if the specifications were written correctly but the algorithm was not proven correct. My sense is that the paradigm of logical program verification is probably not a good description of their development process, but that may not be public information.

I have a PhD in computer science and I develop flight software for a living (among other things) and I use whatever tools I can, including formal correctness. But I guess I'd agree that it's not common.


My understanding is that this was user error (miscommanding) rather than a software failure per se.

Software can be complex, and the simpler it is, the more likely it is to be right. Landing on Mars should not be that hard from a software perspective.

What a wonderful insight into the computer system on the Viking landers. Interesting that they say in there that a system could be run completely off preset timings but it was decided on that mission to allow the craft to fly itself a bit based on accelerometer input. Instead of using radar to decide altitude they basically used computed speed and timing which says to me that one can know fairly accurately where the craft is at any time without sophisticated add ons. Hindsight seems to say that radar is a wonderful addition that enhances the entry but nothing replaces the basic science and physics that can model the path and timings very precisely. I suppose there's a nerd factor that takes over when designing complex systems, a pull to reject simplicity. Sometimes, as in the case of the Curiosity sky crane, it all comes together but we all had nightmares about all the things that had to go exactly right. In many ways the real genius was shown in the very simplistic idea of the airbag delivery system, something that ESA could have certainly used for its first attempts since their lander was so light. I understand they wanted the base of the lander on the actual surface so once the airbag is deflated and retracted flip the thing off onto its head. Oh the ease of armchair quarterbacking. Laughs.

You are reading way too much into that basically non-technical history. Radar is essential for Viking or any Mars lander that has to do anything based on altitude, as the altitude relative to the ground can't be determined any other way.

I agree with mcaplinger: Landing on Mars should not be that hard from software perspective
Flight/lander software is robust (50 yrs of it) and even newer and tested architectures are out there (Deep Space 1, Remote Agent, late 90s)

An executive like Remote Agent could have said something like: "No Parachute, listen to me, not the Radars, stay with me until I say so...we are on a timeline and we have minutes to go....and as for you Radar, stop sending me crap, and get me good altitude and velocity and quit scaring Parachute... and Rockets, don't even think about firing for only 3 seconds.... everybody get back on the script, and back on the timeline..."

I doubt that the primary problem was radar, as it does not explain the early release of the backshell. My gut feeling is that for whatever reason (bad accelerometers?) the state vector deviated too much from reality and basically the lander though it was lower than it really was and could not incorporate the radar data into the state vector and ended up ignoring it.

Here is my timeline of what was supposed to happen... and what happened
My suggestion is that the heat shield did not come off (or hung on) and spoofed the radars, computer, and rockets
Where is heat shield on the surface, and how close is it to the other hardware?
HiRise should supply answers.
(ESA times, altitude, and speeds)

That would be a good theory, but the CTX image shows a possible impact (see http://www.planetary.org/blogs/emily-lakda...pdate-ctx.html)
HiRISE should confirm soon enough...

Out of curiosity did the test lander fail before or after planned decent images?

There is an extra ")" at the end of your url. This one should work.

Here is a detailed technical description of TGO, including the EDM.
Have a closer look at this graphics, particularly at the Radar Doppler Altimeter (RDA) block within the Surface Platform subdiagram. You'll see a thin blue line injecting the block, meaning connection via CAN bus b.
Access to the propulsion sysem via CAN (again via the technical EDM article) :


Communication over CAN bus systems is usually performed via sending function blocks.
Systems are usually described by message sequence charts according to the MSC-2000 "standard". This would normally mean asynchronous exchange of messages between processes with all the difficult testing.
This doesn't necessarily mean, that the system design of TGO works exactly like this, but I would be surprised, if not.

See also page 9 of this presentation: