Opportunity General Health Options

[Doug M.]

I've been clicking around for a general health status for Opportunity, but haven't been able to find one.


Power -- The solar panels seem to be showing signs of degradation over time but it's not clear how much. NASA reports regularly on power output, tau/opacity and dust levels, but not on the status of the panels themselves. They did clock well over 500 watt-hours as recently as May, so it doesn't look like their performance is a serious issue. As for non-solar power, the radioisotope heaters seem to be fine; they're Pu-238, so they would only have lost about 10% of their power since launch. So the WEB is still toasty.

Motors and joints -- I know we've got one bad wheel motor (which means we spend a lot of time driving backwards) and the bad arm azimuth joint and the separate issue with the arm potentiometer.

Electronics -- There was the flash memory issue earlier this year but I don't know if that was a one-off or a sign of age.

Instruments -- The Mossbauer spectrometer is done because its radioactive cobalt source ran out. MiniTES got dust on its mirror after the big 2007 dust storm and stopped working. AFAICT the other instruments are okay? Pancam, Navcam, Hazcams, and the APXS all seem to be working fine. There was a NASA press release last month that said "Opportunity imaged the Rock Abrasion Tool (RAT) bit to assess remaining bit life", but it didn't say anything about what they saw and I haven't been able to find any more information. Clearly the RAT is still working, at least for now.

What else?


Doug M.

[fredk]

If you mean would the panels still provide the power they did at landing if they were clear of dust (and the illumination was the same), I don't recall any mention of that. My guess would be that the battery's ability to charge would be more of a problem, but again I don't recall any mention of its status. Remember that Spirit hit over 900 Whr at around sol 2000.

I don't think you mentioned the frozen steering actuator. Probably not fair to call it a "bad" driving motor, since it works but just gets a little hot sometimes.

Probably the best source to find lots of detail about all of these problems is the PS updates.

[Doug M.]

If you mean would the panels still provide the power they did at landing if they were clear of dust (and the illumination was the same), I don't recall any mention of that.

Well, all solar panels degrade gradually over time, and in space they go faster. The ISS arrays have a 15 year nominal lifespan; it's expected to be longer than that in practice, but OTOH the ISS arrays were designed and built with a large margin over the ISS' actual power needs. But given that nobody seems to be mentioning it, I suspect that it hasn't been a significant issue yet, or at least not as compared to dust, opacity, et al.

Your point about the batteries is a good one -- I hadn't thought of that. Googling turns up a number of papers from the rovers' first few years, basically saying "wow -- these lithium-ion batteries are holding up great!" Not seeing anything since 2010 one way or the other.


Doug M.

[djellison]

Note - on the surface, under an atmosphere and, for half the time, in the dark - those solar arrays are not getting the same level of abuse they would be getting if they were in free space.

[fredk]

Yeah, a better comparison than the ISS might be the Mars orbiters. Maybe someone knows something quantitative about those.

In the absence, 900 Whr at sol 2000 is pretty good. And the panels weren't completely free of dust at that time.

[djellison]

FWIW - I'd expect that the 'dust factor' value inherently includes solar array degradation. It's simply the ratio between predicted solar power from new, clean arrays given known atmospheric opacity - and the actual power generated.

[Aldebaran]

(Long term lurker here)

It's quite amazing how much is still working, given that Opportunity will have been on the surface of Mars for 10 years as of next month.

[pospa]

Hi, yesterday was 12th anniversary of Oppy's launch from Earth (2003-07-07). Its amazing what she has achieved.
Would anybody know actual status of rover battery pack (available capacity after n-thousands of cycles) and estimation of today's RHU's heat output?
Eventually any update about other component / equipment / tool / instrument degradation or lost vs. remaining functionality.
Many thanks

[craigmcg]

Just curious - has anyone created a log of the decrease in MER capability over time? For example, loss of various instruments, loss of solid state memory, etc? Has there been any planning about what loss of future capability would push it past the line where the cost/benefit ration of running the mission was too low?

[fredk]

There are so many things that could go wrong or partially wrong that once you got into their combinations and permutations it seems to me that the cost/benefit ratio of doing such planning would be pretty high. The benefits to continuing the mission would also depend on the scientific interest at the site she was located at the time.

[TheAnt]

Fredk already covered the question well. IMO opinion the cost/benefit hardly will apply unless the cost is as high as for sending another rover = hugely expensive.

Yet even if all instruments and cameras are dead (The radio has to be working else no mission right? ) - there's one thing that Opportunity can do, and which actually were one task planned for Spirit if it had survived that last winter - and that is for the rover to use the radio as a stationary platform that could give hints of Mars interior and perhaps even Mars-quakes - which could be indirectly detected by a change in the planet rotational period.

[MoreInput]

Your point about the batteries is a good one -- I hadn't thought of that. Googling turns up a number of papers from the rovers' first few years, basically saying "wow -- these lithium-ion batteries are holding up great!" Not seeing anything since 2010 one way or the other.

Here is an older paper about the performance of the batteries of the rover (only until sol 670): http://www.researchgate.net/publication/23...loration_Rovers
Here is the link to the battery manufacturer: http://yardney.com/

I couldn't find any newer material about this.

[serpens]

One has to give full credit and plaudits to the JPL engineers that designed the battery control board. However, after 4000 odd cycles since landing the battery storage capacity must have dropped significantly (around 50%?). The reducing capacity between maximum charge and minimum voltage cutoff means an increasing risk over those long winter nights. Given that these Li-ion batteries are pretty much first generation post production their longevity is nothing short of amazing.

[nprev]

"Amazing" is not too strong a word at all re Oppy's battery performance. I'm surprised that nobody's published any more recent studies at this point; surely there are MANY good lessons to be learned here.

[craigmcg]

> Yet even if all instruments and cameras are dead...

So some value comes if the only parts left working are the power system and the radio

The other part of my original question was about a "log" that documented the various failure dates of key components. I was thinking about creating some kind of timeline that showed this visually.

[MoreInput]

I did not know about a good log for this problems. In the english Wikipedia is a small chapter of the failures of Spirit: https://en.wikipedia.org/wiki/Spirit_(rover...ar_and_failures

I am also interested in such a timeline, it could be interesting for the Wikipedia.

Here is a list about the equipment failures I know:
Opportunity:
- A) The shoulder of the instrument arm stopped working at Sol 654 (November 25, 2005). The Joint-1 azimuth motor stalled because of increased electrical resistance (Source: https://en.wikipedia.org/wiki/Opportunity_m...ulder_troubles).
As I know opportunity does not stow the arm anymore and drives in a "fisherman" position over the martian surface.
- Also Opportunity had problems with the heater. Opportunity happens to have a heater stuck in the "on" position that draws additional power. (see http://mars.nasa.gov/mer/mission/status_op...tyAll_2007.html, Sol 1316)-
- C) One wheel cannot be steered anymore, and is locked in a specific position. But the navigators have learned, to drive with this issue.
- D) Also, Opportunity drives backwards since many year , because there always had been elevated currents in one of the wheels.
- E) The MiniTES does not work since the sandstorm 2007.
- F) The Mössbauer instrument does not work anymore, but I think there is no exact time stamp since it stopped working, it was just a permanent degregation (as expected)
- G) The amnesia events since one year (Wikipedia EN: Early September 2014)

[hendric]

There's a couple of advantages the MER batteries have vs typical LiIon batteries here on Earth:

1. No high-temp storage or operation. How often is a laptop/phone left in a car or trunk in the heat? And turned on without waiting for the battery to cool down? Low operating temps extend battery life and prevent self-discharge.

2. Relatively low drain vs their capacities. While fairly large batteries, their drain doesn't look that large vs their capacity. During the highest drain activities in the daytime , the solar panel provides additional current. During nighttime the primary activity might be radio comms without movement. Laptop and cellphone batteries have frequent high-drain activities while disconnected from their charging system, ie watching a video over the air.

3. Relatively slow charging cycles vs their capacities. People need fast charging for their batteries, so consumer charging circuits typically charge at max rates - 1C/Ah typically vs what looks like .2C/Ah peak for the rovers.

4. Low-vibration environments. Well, other than launch and EDL, the vibration environment is pretty benign on Mars. Having to design for launch/EDL levels of vibration might also have endurance side effects - better connections, etc. Phones/laptops are often dropped or jarred on tables, etc. Probably a minor effect.

5. Individual cells are balance-charged. Lithium chemistries really, really hate being overcharged. What typically kills cheap laptop batteries is a design that uses a single charging circuit, passing through all the cells. These cheap designs cutoff charging once all the cells have reached nominal. If a cell reaches nominal early, it gets overcharged while waiting for the slowpokes, hurting its capacity and making it hit nominal even earlier the next charge cycle. Typically when new all the cells are more-or-less matched, but over time they drift apart. This is more important for LiPoly chemistries than LiIon, but it still matters for battery lifetime over a few hundred charges. People can tear apart used laptop batteries to get the "good" remaining cells for other uses.

6. Not charging to full rating. Continuously charging a Lithium battery to its full rating will also hurt lifetime. Newer laptops can be setup to only charge to 50-80% during the week and 100% on the weekends, or on-demand. Looks like other than EDL, the MER batteries kept below ~75% capacity. Old laptops used to keep their cells at 100% all the time while plugged in for extended amounts of time, seriously compromising their performance. Smarter chargers now charge to the requested level, and shutoff until the level has dropped an amount, typically 3-5%. They also can be setup so that the system load is not run directly off the batteries, allowing the charger to provide power directly to the system without routing it through the battery first. Also, the MER charging is always done in a pretty benign environment, with batteries that are presumably pretty cool.

7. No intentional deep-discharges. This, IMO, is the biggie. Looking at the graph in the paper, the only time MER batteries were ever deep discharged was the Spirit anomaly, otherwise they are kept at about 50% DoD. The anomaly drained her batteries to the cutoff voltage (Lithium chemistries are damaged when drained completely, unlike NiCd or NiMH. Batteries typically have a self-protection circuit that cuts the ground line below a minimum voltage during discharge.) People often drain laptops and phones until the battery is "dead" with the internal cutoff circuit activated - what's worse, they often turn the phone "back on" - with the phone off the battery rises slightly above cutoff, and during boot the power consumption is much lower so the battery continues to drain until the radios are activated and then the phone dies again. These deep discharges really hurt battery lifetime.

I don't know how accurate the capacity measurements are for the MER batteries, typically a gas-gauge chip is placed in series with the battery and reports capacity basically by comparing current IN versus current OUT. Over time they need to reset their count by doing a "training cycle" - to relearn the battery capacity. Without these training cycles the reported numbers get more and more inaccurate - if you have ever had a device that went from 30% charge down to "OMG I'm gonna die" it's because its gas gauge needed to be retrained. The reported % number is based on a certain capacity, if your battery has degraded below that capacity number, as it drains the battery self-protection circuit flags a warning before it cuts off the battery, hence the OMG message. Typically there is a maximum amount the capacity can go down during each training cycle, so it might take more than one to become accurate again. Since there are two packs per MER, it's possible one battery pack could do a training cycle while the other is in use, but it doesn't seem like that's reflected in the graphs. Maybe they are estimating capacity based on the cell voltage, which is pretty inaccurate (hence the invention of the gas gauge chips). It does look like newer gas gauge chips model aging internally, not sure what was space-rated 15 years ago during MER development though. http://www.ti.com/product/bq27741-g1 is a sample gas gauge chip.

I've done some embedded HW/SW development for cellphones and mobile devices using LiIon and LiPoly batteries, but I am not a battery engineer.

[ddeerrff]

Do I remember right that Oppy's wheel motors have brushes? If so, how in the world (or how on Mars) can those brushes not be totally worn away? Yet Oppy keeps on going, and going, and going.....

[serpens]

Yes, both drive and steering motors on Oppy are brushed which made good design sense given the 90 day, 700 metres driving distance mission success profile. The loss of the steering motor is possibly due to brushes as was the drive motor failure on Spirit, but given the staggering driving statistics for Opportunity the longevity of the motors is astounding. Well to be honest the Lithium Ion batteries are also performing brilliantly. I suspect that the lions share of the credit for both the performance of both batteries and drive motors accrues to the management of the MER engineering team and drivers.

[RoverDriver]

The wear and tear on the brushes is proportional to each actuator use. If you take the current rover odometer, divide by one wheel circumference and multiply by 1500 (the gear ratio) you get a decent number (100 million revs) which is not too shabby but not too out of specs.

Oppy RF steering actuator if I remember correctly (it happened close to 14 years ago!) the most likely theory was that the magnetic detent came unglued and got jammed into the motor.

Spirit RF (and at the end the RR) drive actuators instead were suffering from what appeared to be a contact failure between the brush and commutator as if there was a non-conductive layer deposited over time. We called that "napping motor".

This is also different from the Joint 1 actuator on Oppy's IDD. That one was attributed to a broken winding wire due to fatigue caused by thermal expansion/contraction ultimately due to the stuck shoulder heater.

All actuators on MER are brushed motors, except the stepper used on the MI dust cover mechanism. That one was built as an open loop control. Open loop is now also used on the three RAT actuators, but that is due to the loss of signal from the encoders due to wear and tear on the IDD flex cable).

Now you know almost as much as what we tell to new rover driver trainees.

Paolo

PS: questions... longevity... incept date... Eyes, I only do eyes!

[hendric]

I envision a ceremony where new rover drivers are given The Tome, a collection of all knowledge on the rovers, that has a page on Rover Longevity that is just a URL to this discussion.

[marsophile]

Enhanced FHAZ of RF wheel from Sol 5066 (Parallel-eye 3D). Some Martian material seems to be sticking to the wheel strut.
Compare to this image from Sol 1069:
https://mars.nasa.gov/mer/gallery/all/1/f/1...VAP1151R0M1.JPG
Most of the FHAZ images are dark so it is difficult to tell when the material first appeared.

Interesting that the material is sticky. Is it possibly corrosive and might it pose an issue for rover safety?

[djellison]

If you look at the three subframed Right FHaz images taken as we did the backwards 180 over the weekend - it looks more like shedding tape than sticky soil.

https://mars.jpl.nasa.gov/mer/gallery/all/1...91P1220R0M1.JPG
https://mars.jpl.nasa.gov/mer/gallery/all/1...99P1220R0M1.JPG
https://mars.jpl.nasa.gov/mer/gallery/all/1...A7P1220R0M1.JPG

It's been there a while...

https://mars.jpl.nasa.gov/mer/gallery/all/1...QEP1243R0M1.JPG (4774)

https://mars.jpl.nasa.gov/mer/gallery/all/1...B7P1293R0M1.JPG (4591)

https://mars.jpl.nasa.gov/mer/gallery/all/1...29P1110R0M1.JPG (3950)

https://mars.jpl.nasa.gov/mer/gallery/all/1...UBP1215R0M1.JPG (3489)

https://mars.jpl.nasa.gov/mer/gallery/all/1...QOP1214R0M1.JPG (2595)

https://mars.jpl.nasa.gov/mer/gallery/all/1...__P1201R0M1.JPG (2474)

https://mars.jpl.nasa.gov/mer/gallery/all/1...WMP1201R0M1.JPG (2468)

But I don't think on 2464
https://mars.jpl.nasa.gov/mer/gallery/all/1...Q0P1212R0M1.JPG

And clearly not on 2454
https://mars.jpl.nasa.gov/mer/gallery/all/1...QP1212R0M1.HTML

That would pin it to somewhere between 2464 and 2468.

[peter.neaum]

The wear and tear on the brushes is proportional to each actuator use. If you take the current rover odometer, divide by one wheel circumference and multiply by 1500 (the gear ratio) you get a decent number (100 million revs) which is not too shabby but not too out of specs.

Oppy RF steering actuator if I remember correctly (it happened close to 14 years ago!) the most likely theory was that the magnetic detent came unglued and got jammed into the motor.

Spirit RF (and at the end the RR) drive actuators instead were suffering from what appeared to be a contact failure between the brush and commutator as if there was a non-conductive layer deposited over time. We called that "napping motor".

This is also different from the Joint 1 actuator on Oppy's IDD. That one was attributed to a broken winding wire due to fatigue caused by thermal expansion/contraction ultimately due to the stuck shoulder heater.

All actuators on MER are brushed motors, except the stepper used on the MI dust cover mechanism. That one was built as an open loop control. Open loop is now also used on the three RAT actuators, but that is due to the loss of signal from the encoders due to wear and tear on the IDD flex cable).

Now you know almost as much as what we tell to new rover driver trainees.

Paolo

PS: questions... longevity... incept date... Eyes, I only do eyes!

I know I'm replying somewhat after the post, but I was wondering, given the fairly finite lifespan of a brushed motor (friction / wear-down) what were the advantages / decision tree of of the brushed vs brushless?

I did find an interesting article on space-rated motors (below) where it was noted that a predicted (and seemingly solved) problem with brushless motors was that that the hall sensors are vulnerable to radiation damage. The below article includes testing & good pics of friction wear & tear caused by brushes.

Article here: http://esmats.eu/amspapers/pastpapers/pdfs/2012/phillips.pdf

[stevesliva]

"Relatively finite." Are you using inductive reasoning to conclude that Opportunity's lifetime is approaching infinity?

[peter.neaum]

"Relatively finite." Are you using inductive reasoning to conclude that Opportunity's lifetime is approaching infinity?

Imagine the headaches that would cause! ;-)

The article (linked above) shows brushes + friction = resulting in predictable wear / gouging.
That cannot go on forever (!), so the motors, relative to other items on the rover have an end of life date.

My thought was 'what wear is there in a brushless motor?' - which then leads to why didn't they use brushless - which then leads to advantages vs disadvantages etc...

[RoverDriver]

...
My thought was 'what wear is there in a brushless motor?' - which then leads to why didn't they use brushless - which then leads to advantages vs disadvantages etc...

We'll see Curry actuators how they survive there. Although I'm no expert, brushless require a more complex electronics, and if you think that electronics is more robust than mechanical components, think again. Out of 17 actuators we use almost every day, only two have issues) while most of the flash file system (which has no moving parts) have issues, and even Curry had some serious issues with one of the flash gates.

If I had to rebuild MER all over again, I'm not so sure the brushed motors would be the first items on my list.

Paolo

[James Sorenson]

If I had to rebuild MER all over again...

The MER design has proven itself to be solid and reliable. I hope the design could someday be used again, perhaps with some upgraded electronics components and a different set of instruments.

[marsophile]

...
- it looks more like shedding tape than sticky soil.
...

Indeed. In this well-lit image, it appears to be an original equipment plastic cover that is fraying and coming loose. Not Martian at all! A warning to be wary of over-enhancing a dark image...

[RoverDriver]

The MER design has proven itself to be solid and reliable. I hope the design could someday be used again, perhaps with some upgraded electronics components and a different set of instruments.

There are two things I would redesign:

1) Move the UHF and HGA to a higher place. So many times I had to negotiate the vehicle attitude with communication requirements!

2) have the solar deck tilted about 20 degrees. No more hunting for high tilt in Wintertime.

3) and of course I would like Google Fiber from Mars.

Paolo

[vjkane]

There are two things I would redesign:

Maybe ultraflex solar panels for more power

This thread isn't totally idle speculation. The Mars community is considering a line of future MER-size rovers to explore the diversity of surface types with past histories of water. With modern instruments, it could be really nice upgrade.

I suspect that a precision landing system would also be high on the list for a future rover line.

Would be nice to have a single general purpose heat source RTG (MMRTG's have 8) to supplement the solar array for survival heating and power during dust storms. Of course, the mission cost just sky rocketed due to the certifications.

[fredk]

In this well-lit image, it appears to be an original equipment plastic cover that is fraying and coming loose.

Not surprizingly sharp-eyed members here spotted this years ago - check out the animation here.

[James Sorenson]

Maybe ultraflex solar panels for more power.

More power maybe, but what about deployed rigidity? A rover going over rocks and pot holes in the road no doubt will put alot of vibrational stress on the bonded cells and structure. Was there ever testing done on this for the MAX-C concept?

[mcaplinger]

Was there ever testing done on this for the MAX-C concept?

As far as I know MAX-C was a series of viewgraphs "designed" by scientists and systems people. Engineers generally come in and make things work only after the mission is selected.

Let's put this speculative discussion somewhere else as it's off-topic for MER. The rover after M2020 will likely resemble MER only in that it will have wheels and drive around.

[mcaplinger]

My thought was 'what wear is there in a brushless motor?'

The bearings and gear teeth still see wear in a brushless motor.

Predicting the lifetime of any mechanical system, or any system at all for that matter, is a bit of a black art.

[hendric]

Looks like some research is being done for low-temp batteries. https://www.nature.com/articles/d41586-018-02631-9

[Explorer1]

The meantime... the newest A.J.S. Rayl update shows that apparently the power situation is so good, the team is attempting to use the excess(!) for more astronomical observations:
http://www.planetary.org/explore/space-top...-extension.html

I am wondering what the issue is with the batteries remaining fully charged; I've heard about this before, but does anyone have more details? Not just asking because I'm a laptop user who wants to preserve my own as much as possible...

[mcaplinger]

I am wondering what the issue is with the batteries remaining fully charged; I've heard about this before, but does anyone have more details?

http://batteryuniversity.com/learn/article...based_batteries

Your battery would last longest if you constantly maintained it at somewhere between 50% and 75% of capacity. Of course you want to discharge it sometimes since otherwise there's not much point in having a battery.