Anyone know the latest Voyager status? I've hear rumors, but I'm wondering if anyone has anything more concrete (I won't share the rumors, as I really don't know much about it, so...)
Curious, http://voyager.jpl.nasa.gov/mission/weekly-reports/index.htm was published in August, exactly when V1 hit 100au milestone...
Anyway, the two spacecrafts still alive, as confirmed by http://voycrs.gsfc.nasa.gov/heliopause/heliopause/recenthist.html updated to one week ago and V2 ftp://space.mit.edu/pub/plasma/vgr/v2/ha/key/ updated yesterday...
The Voyager status reports are always late: very small team in an extended, extended, extended mission. But the DSN tracking schedules are up to date, and they show normal activity. There has been the standard once per year memory readout recently. So there is no hint of a spacecraft issue I can see right now.
But Voyager 2 should cross the termination shock about now, Voyager 1 did this a couple of years ago. I have no insight and understanding of the science data. They should show this. And Voyager 1 should cross the next "shock" (I can never remember these solar system bondaries) in the not so far future.
Analyst
You might be thinking of the Heliopause Analyst. Voyager 1 should be in the Heliosheath where the solar wind begins to mix with the Interstellar medium. The bow shock could be a bit further away than this image shows. It might be some time before Voyager 1 reach the bowshock, but it would be wonderful if it did. There wont be any TAU mission in the forseeable future so the two Voyagers will be the best shot we have of studying this region.
. . . BEEP . . .
cold and dark
. . . BEEP . . .
still cold and dark
. . . BEEP . . .
yep, it's really cold, and really dark
. . . BEEP . . .
yawn
. . . BEEP . . .
really, really cold, and still dark
Space is Big.
Space is Dark.
It's hard to find
a place to park.
. burma shave.
http://en.wikipedia.org/wiki/Marco_Polo_%28game%29
.......
......
......
Shame it's too late to get http://en.wikipedia.org/wiki/Hawkwind's 1973 recording of http://www.youtube.com/watch?v=ugTLEkPi1Jc added to the Voyagers' records...twelve string guitar, psychaedelic synths and memorably cheesy lyrics... Yep, that'd do me as I drifted off into the endless AUs...
The path goes onward through the night
Beyond the realms of ancient light
Andy
On that note, my bit of wistfulness for the Voyager music as it travels thru interstellar space would be the song from John Carpenter's early film Dark Star...
Benson, Arizona,
Warm wind thru your hair,
My body roams the galaxy,
My heart longs to be there.
Benson, Arizona,
Same stars in the sky,
But they look so much better,
When we watch them, you and I.
Coincidentally, Benson is less than an hour down the road from LPL.
It's not merely a song about Benson, AZ. It's the somewhat improbable theme song of the John Carpenter student film-cum-cult-classic, Dark Star. It's a country-and-western song in format, but the lyric is about a lonely guy, flying through interstellar space at relativistic speeds, and thinking of everything -- and one special person -- he left behind.
If I can recall the words...
A million suns shine down,
But I see only one.
When I think I'm over you,
I find I've just begun.
The years move faster than the days,
There's no warmth in the light.
How I miss those desert skies,
Your cool touch in the night.
CHORUS:
Benson, Arizona, blew warm wind through your hair.
My body flies the galaxy, my heart longs to be there.
Benson, Arizona, the same stars in the sky,
But they seemed so much kinder when we watched them, you and I.
Now the years pull us apart,
I'm young and now you're old.
But you're still in my heart,
And the memory won't grow cold.
I dream of times and spaces
I left far behind,
Where we spent our last few days,
Benson's on my mind.
(CHORUS)
-the other Doug
"Why do I always have to feed the alien?"
"Cause you brought the stupid thing on board in the first place!"
How could it be alive it was just a bag of gas?
"......and dont dare play that record one more time!"
We have to be careful not to teach the Voyagers phenomenology...
"How could it be alive it was just a bag of gas?"
Yeah.. but I'm still convinced it was smarter than the entire crew put together.
This is the same switch they did on Voyager 1 in early 2002. Switching HYBIC means you have to use the redundant star tracker (roll) and sun sensor (pitch and yaw) as well, even if the current used ones are just fine. On the other hand, the scan platform pointing information (azimuth and elevation) is no longer needed. So there is some risk because you use other sensors with different and not completely known biases. And there is the possibility HYBIC 1 is not working and you have to switch back to the dedraded HYBIC 2. The AACS computer in charge remains the same, there is no switch planned.
Interesting note: One branch of attitude control trusters for pitch and yaw failed in 1999. But they are not critical on that because they can use the (larger) TCM trusters if the second branch fails.
Analyst
"...because they can use the (larger) TCM trusters if the second branch fails."
Might mean a much higher rate of use of attitude control propellant, leading to eventual end-of-mission before other expected problems <like low voltage or inadequate suntracker sensitivity> ends mission.
Since the main receiver failed on Voyager 2 shortly after launch back in 1977,
and they had to rely on the backup receiver which is apparently tone deaf, for
lack of a better technical phrase, how is that rather critical piece of equipment
holding up? And how are they keeping it so finely in tune after all this time?
Thanks for the info. The only thing I knew was that we lost one of our 70m passes due to some kind of emergancy with one of the Voyagers. This seems to fit quite well with the details included here, so... Thanks for your help!
http://www.nasm.si.edu/research/dsh/artifacts/GC-Voyager.htm is another spare HYBIC, flight-qualified, but not quite on location.
Analyst
The January JURAP meeting did discuss the status of the HYBIC swap, but the report of that meeting hasn't been released yet. JURAP minutes are released on a somewhat irregular basis, so it's hard to say when we ordinary people will get to see them.
Last things I've read on the Grand Tour spacecraft:
Voyager 1 crossed the termination shock at 94 AU in December 2004, 100 AU in December 2006 and estimates show it will pass the Heliopause by 2015.
Voyager 2 is now at ~82 AU and is likely to cross the shock sometime this year.
Fingers crossed both will still have some electrical power to keep operating.
By The Way: this year is the 30th anniversary of the summer 1977 launches!
That's 3 AU per year. Will these overtake the Pioneers?
Indeed, both voyagers will overtake the Pioneer 10 and 11 as the Voayager 1 & 2 have a significant speed advantage. I didn’t calculate when they will out-distance the Pioneers but here are the formulas for the distances traveled by both Voyagers:
Voyager 1: 76.34 + 3.50 ( future year – 2000 ) = distance in AU
Voyager 2: 59.75 + 3.13 ( future year – 2000 ) = distance in AU
( 1 AU = Astronomical Unit is the average distance between Sun & Earth : approx 150 million kilometers )
ehm...I thought they already did??
Voyager 1 already did in 1998, Voyager 2 will in about 2022.
http://www.heavens-above.com/solar-escape.asp?lat=0&lng=0&alt=0&loc=&TZ=CET
Analyst
The tracking schedules on the Voyager http://voyager.jpl.nasa.gov/mission/soe-sfos/sfos_2007.html from February/March 2007 don’t show the permanent HYBIC switch as planned in November last year. Maybe some unexpected results from the short test swap in early December.
Analyst
Has anyone tried to determine if Voyager 1 or 2 will fly
anywhere near a Kuiper Blet Object?
Is there enough fuel on the Voyagers to conduct
a Mid-Course Maneuver to enable a close flyby of a KBO?
Assuming all of that, what is the status of each camera
on the Voyagers? When were they last used?
I bring this up because the Voyagers are now in the Kuiper
Belt, whose components were discovered after the launches
of the Voyagers, and because "new" KBO's are being detected
all the time. Perhaps one of those ice balls will be in the right
place at the right time.
Another Phil
The scan platforms have been switched off for many years, and there would not be the power to operate enough systems to make this a feasable exercise.
Doug
Aren't the majority of the Kuiper belt objects more or less (+/- 10 or so degrees) in the plane of the ecliptic? The Voyagers are going north and south by 25 or 30 degrees and are therefore in a region with not so many Kuiper belt objects, if I am correct with my assumption. The last time a Voyager camera was used was in February 1990. The heaters on the scan platforms are turned off and even if there is power to spare it is doubtful the cameras would work again.
Analyst
Not only have the scan platform instruments been switched off.. the scan platform HEATERS have been switched off. Hardware on the platforms is probably at pretty seriously cryogenic temperatures.
And while the majority of KB objects are near the ecliptic, that is increasingly seeming to be an artifact of where we're searching. a *LOT* of them have higher inclinations and spend much of their time outside the near-ecliptic search zones and have been found by accident as they crossed the zone.
Isn't Eris, the biggest KB Planet/Dward-whatever in a 45 deg orbit?
Eris's orbit is indeed inclined nearly 45 degrees, though that doesn't necessarily mean it was 45 degrees away from the ecliptic when it was discovered. Your point about the artifact of where astronomers look for objects is a very good one. The Minor Planet Center's http://www.cfa.harvard.edu/iau/lists/OuterPlot.html is a great illustration of this. There's a big hole! And - surprise! - it's in the direction of the center of galaxy, where finding really dim objects is not so easy. (I suppose the fact that New Horizons is flying right into that big hole is great reason for optimism that some nice juicy targets will be found for post-Pluto encounters.)
I'd guess we should expect large numbers of TNOs at high inclinations, based on what I've read about planetary migration driving up TNO inclinations.
As always, this kind of discussion reminds me of the joke about the drunk who's looking for his car keys under the streetlight, not because that's where he thought he lost them, but because it's easier to look for them there.
http://www.tucsoncitizen.com/daily/local/43056.php
I noticed some false information in this article.
Regarding KBO's, which were undiscovered prior to Voyagers' launch: do the spacecraft have any way to scan ahead to see if they might run into or near anything, as unlikely as it would be? Even if they couldn't image anything, it would be a great feather in the cap of the Voyager team to detect a new object more than 30 years into the mission!
May be, it's a little bit off topic but would it be possible to power on the voyager camera again with the remaining power? Unnecessarily to say that they couldn't see much, because it's distance to the sun.
My earlier question pertained to whether any of Voyagers' operating instruments might pick up an indication of a distant KBO or even some wandering interstellar object of significant proportions. Pioneer may have already captured the prize, having possibly been deflected by an undiscovered KBO at around 8Bn km. It was affected by the object in 1992, and it took 7 years to figure out what might have happened.
I realize my original question is just a speculative exercise in a fantastic "what if", but here's what I've found in a quick search:
I read through the VIM proposal for the 2005 NASA funding review. As mentioned above and in other Voyager threads, the only devices left on are for measuring helioshperic and extraheliospheric features.
"The entire Voyager 2 scan platform, including all
of the platform instruments, was powered down in
1998. All platform instruments on Voyager 1, except
UVS, have been powered down. The Voyager 1 scan
platform was scheduled to go off-line in late 2000, but
has been left on at the request of the UVS investigator
(with the concurrence of the Science Steering Group)
to investigate excess in UV from the upwind direction.
The PLS experiment on Voyager 1 which had been
turned off in 2000 to provide power to extend UVS
lifetime, was turned on again in 2004 when there was
evidence that the spacecraft was in the vicinity of the
26
termination shock. UVS data are still captured, but
scans are no longer possible."
While it seems unlikely for the Voyagers at 100 AU to closely encounter any KBO's, they'll be in the KBO neighborhood for a long time -- Sedna's 10,000 year orbit takes it out to 900AU!
As to whether either Voyager craft could maneuver towards a newly-discovered object:
"The thrusters currently in use are expected to
last the rest of any mission projection. Nearly 1/3 of the
original propellant remains available."
While they're only using thrusters to keep the craft in optimal contact with earth, it is kind of amazing how much fuel is left. They sure saved a lot of juice using the "Grand Alignment" of the outer planets for gravity-assists to sling-shot the Voyagers out of the solar system!
Real world note: One of the many compelling reasons to keep the program alive is that the Voyager craft are making the first beyond-the-shock measurements of Radio Wave events generated during the declining phases of solar cycles. I have the mental image of the Voyager craft being the first to measure waves lapping at the shore of the "lake" that is the solar system.
I am aware of the futility of examining 'do overs' but just for old times sake, here's one:
IIRC, the 'window' for possible Uranus flyby dates was roughly a week long, and the nav team selected one that gave a nice close up of Miranda, good resolution on Ariel and Titania, and so-so for Umbriel and Oberon.
It seems a good satellite configuration existed just before the opening of the Uranus window (although I have no information on the specific satellite config at that time).
Would the surplus manuvering fuel on Voyager II have allowed this encounter, and would it have been sufficient to put Voyager II back on the the exquisite Neptune 'polar crown' trajectory 3 years later ??
(I realize the mission team had specific requirements for fuel margins and the line needed to be drawn somewhere, but I can dream, can't I?)
During my recent explorations into the bowels of Nasa's web servers, I came across this recent mishap of Voyager 2:
Thanks for the info, Pando. Let's hope magnetometer will fully recover!
PS: In the meanwhile, http://voycrs.gsfc.nasa.gov/heliopause/heliopause/recenthist.html confirm that a still increasing regime of low-energy particles and turbulence, compared to 1 year ago...
This is old news from the January 18, 2007, JURAP meeting, but it expands on the news Pando gave:
HYBIC SWAP TEST RESULTS
REDUNDANT HYBIC TEST & TEMPORARY SWAP
-PURPOSE OF THE TEST
. Validate operability of HYBIC 1 and health of celestial sensors
. Refine sun sensor bias offsets between HYBIC 2 and 1
. Gather information in preparation for a permanent swap and calibration or futher study
--The HYBIC test was performed on 11/30/2006, DOY 335/02:32:37 UTC (6:32 PM PST). All events executed as planned. HYBIC 1 functioned properly and the pointing offset data were obtained.
--At the time of the swap, the available power dropped to an unexpected level.
--The MAG instrument status indicated that the Out-Board flipper status had changed and that the flipper is now ON. The instrument temperature increased significantly.
--Our investigation revealed that one of the commands issued to reinstate HYBIC 1 also caused the Out-Board Flipper ON command to be issued. This caused an additional 10.2 watt of power consumption. This similar anomaly happened once before in 1998. The cause was thought to be contamination of the 2N222A transistors in the Power System (power command decoder).
--It's believed that the excessive heat caused the wax pellet actuators that move the flipper back and forth from the "forward" to "reverse" position to melt. Data indicate that the flipper position is "reverse", near O degrees.
--Early indications are that the Out-Board MAG is still functioning. We are awaiting more feedback from the PI's.
--We have formed a team of consultants to investigate the cause of this anomaly.
--The permanent swap has been delayed until this investigation is complete.
I tried to plot the measured magnetic field components reported on the same link where Pando took the warning (note that I sampled only some interesting time windows):
Voyager reports for weeks 03-16-2007 to 07-06-2007 Available
http://voyager.jpl.nasa.gov/mission/weekly-reports/index.htm
http://voyager.jpl.nasa.gov/mission/weekly-reports/2007-03-16.html
to
http://voyager.jpl.nasa.gov/mission/weekly-reports/2007-07-06.html
Fantastic birthday.The Voyagers will outlive them all
http://www.space-travel.com/reports/Pioneering_NASA_Spacecraft_Mark_Thirty_Years_Of_Flight_999.html
"needed less propellant"
For what? V1 hasn't been consuming prop in an attempt to catch up. One could imagine that with 4 required targetting points, V2 would have required more prop for TCM's etc.
However - perhaps V1's trajectory was slightly less optimal than V2's and thus it required more Delta-V for targetting.
Doug
Or maybe one of the spacecraft is inherently more "stable" and hence does less RCS thrusting?
"needed less propellant - for what?"
The extra planetary encounters would also help point the craft to its next destination, thus saving propellant on trajectory changes.
V1 had no 'next destination' after Saturn and therefore needed no trajectory changes. It's been in freefall since Saturn.
Thinking about it, perhaps the answer is a combination of several factors:
1) Maybe voyager operators were more sparing of the propellant for V2 knowing that Uranus (and beyond) were at least 'on the cards' from the off.
2). Different trajectories and speeds of the two spacecraft
3) V1 I think had to make a huge (many minutes) burn to set up for the Titan close encounter.
What I was suggesting is that perhaps V1's trajectory inherantly required significantly more Delta V for..
Post launch TCM
Targetting at Jupiter
Clean up after Jupiter
Targetting for Saturn.
Didn't someone say here a while back that one of the two had an LV underperform a little?
Doug
I was wondering if anyone else had noticed / had any info on what appears to be a mystery about the low and varying signal from Voyager 1? For well over a month now the signal from Voyager 1 (as shown on https://eyes.nasa.gov/dsn/dsn.html) has been varying by approx 3db over the course of an hour or so and its at best about 2db lower than it should be, at worst 5db or 6db lower. With DSS63 out for long term maintenance its currently often being tracked on 2 x 34m dishes at Madrid which are unable to obtain data lock for much of the time. Only DSS14 now seems able to hold data lock.
Tweeting one of the Canberra DSN controllers he confirms this is the case, its not a website anomaly. The mystery is that he tells me the Voyager project apparently are not seeing any problem with the spacecraft?
A month ago, Voyager 2 notes this sort of activity:
Thank you stevesliva. Even though I've only just joined the forum I'm a long time fan of the Voyagers and follow all posts available. The problem was, there was no answer that could be derived online. The condition of Voyager 1 has deteriorated with the signal variations increasing. Personally I was expecting the spacecraft to be lost before too long. I gather today the Voyager team have finally admitted a problem with the Earth pointing of the spacecraft. In a way of confirmation for the first time today I noticed the tracking schedule was not followed as per https://voyager.jpl.nasa.gov/pdf/sfos2019pdf/19_06_27-19_07_22.sfos.pdf with Voyager 1 taking the slot of Voyager 2 on DSS43 and arrayed with DSS34 & DSS35. I understand the team are investigating a yaw error and hope to make corrections shortly. My only fear now is that Voyager 1 is so far off point it may be difficult to upload commands.
On the subject of Voyager 2 I notice from https://voyager.jpl.nasa.gov/pdf/sfos2019pdf/19_06_27-19_07_22.sfos.pdf it looks like Voyager 2 will be swapping attitude control trusters to its TCM thrusters on July 9th, as was done with Voyager 1 in Jan 18.
They've fixed it pretty quickly once they admitted the problem, but it had got very bad. The signal is now stable and the strength is as expected :-) https://eyes.nasa.gov/dsn/dsn.html
Coincidentally posted yesterday:
https://voyager.jpl.nasa.gov/news/details.php?article_id=114
I haven't see anything other than this thread mentioning an anomaly... in any event, amazing what they're still doing with these two.
I don't nitpick often (do I?) but while the Voyagers are perhaps the oldest spacecraft still operating, Vanguard 1 (launched March 17, 1958) is the oldest spacecraft still flying, though it's been dead and inert since 1965.
I've noticed the past couple of days Voyager 1's signal is low again into the DSN, presumably off point again. Tracking times available here https://voyager.jpl.nasa.gov/pdf/sfos2019pdf/19_08_15-19_09_02.sfos.pdf the levels received here https://eyes.nasa.gov/dsn/dsn.html you should see roughly -155db on a 70m and -157db on a 34m dish.
I should have noted that around the 7th September there was a sun sensor calibration and ASCAL which does appear to have fixed the pointing issue and the signal into the DSN has been as expected since then. However as this was the 2nd such incident in the past 6 months I wonder if the sun sensor is having trouble keeping a lock on the sun, now at 147AU.
With info taken from one of the controllers at Canberra on twitter and my observations from https://eyes.nasa.gov/dsn/dsn.html, Voyager 2 was lost for a period of 5-10minutes in the early hours of 26/01/20 while attempting a MAGROL. It looks like an attitude control issue occurred causing the S/C to go off point from the earth. I'm assuming the fault protection kicked in and got the S/C back on point fairly quickly. The S/C was then in engineering mode 40bps rather than the usual 160bps. Even though the Voyager tracking schedule showed no further tracking, later in the day Canberras DSS43 70m dish was tracking and the Canberra website showed a horizon to horizon track was taking place. Its therefore reasonable to assume a spacecraft emergency had been declared with other missions being moved off of DSS43. Checking the most recent track earlier today Voyager 2 remains at 40bps (engineering mode).
My brain is hurting because morning in Canberra is so many hours before here, but it sounds like the most recent tracking was to determine whether the commands sent "later in the day" were received. Round trip time is so slow that this seems fast.
https://twitter.com/NSFVoyager2/status/1222070573536350213?s=20
https://www.jpl.nasa.gov/news/news.php?feature=7587
Canberra still tracking VGR2 at 39bits/sec as of February 2nd, 21h30UT as seen in DSN NOW
Now one week gap of science data.
Any thoughts ?
There was a post from climber in a another thread about the Canberra DSN 70m 11mo shutdown, but this one continues discussion about the V2 anomaly... so, here is an informative now 5-months-old article from the paywalled NY Times:
https://www.nytimes.com/2020/03/04/science/voyager-2-nasa-deep-space-network.html
It is spurred by the DSN maintenance, but has some interesting commentary about the anomaly. Nice to see reporters finding who to ring up, and talking to them.
Voyager 2 is doing well! DSN maintenance has progressed to the point that they have commanded (and heard back) from it after the hiatus.
https://www.nasa.gov/feature/jpl/nasa-contacts-voyager-2-using-upgraded-deep-space-network-dish
According to SFOS schedules, first steps in Voyager energy plan has been executed.
LECP MAIN SUPPLEMENTAL HEATER OFF commands were sent and implemented:
* to Voyager 2 -- on 22/23 Feb 2021;
* to Voyager 1 -- on 16 May 2021.
Confirmed at https://voyager-mac.umd.edu/docs/
Tweet from @NSFVoyager2 on Feb22 confirms it, too:
https://twitter.com/NSFVoyager2/status/1364410980927741952
Voyager 1 sending odd telemetry, acting normally:
https://www.jpl.nasa.gov/news/engineers-investigating-nasas-voyager-1-telemetry-data
Never too late to learn a new language, they say.
https://www.jpl.nasa.gov/news/nasas-voyager-will-do-more-science-with-new-power-strategy
This original report from April, but I've seen it re-reported a few places including another today.
They have bypassed a voltage regulator on V2, and the need to not have headroom for the regulator means that a planned instrument shutoff in 2023 can be postponed until 2026.
Some sort of commanding error means V2 isn't pointing antenna at earth. Should reset pointing October 15th:
https://www.jpl.nasa.gov/news/nasa-mission-update-voyager-2-communications-pause
It looks like DSN has picked up a carrier signal
Voyager II, the best of the best is back on line : https://www.jpl.nasa.gov/news/nasa-mission-update-voyager-2-communications-pause
Thought I should draw attention to the latest SFOS 12th October https://voyager.jpl.nasa.gov/pdf/sfos2023pdf/23_10_12-23_10_30.sfos.pdf Note this is the second time the deadband has been widened in about a month. IE the free drift between thruster firings. It was originally 0.1deg, widened to 0.3deg a few weeks ago, now 0.5deg. The only reason I can see for this is to reduce the thruster firings. So either they've found theres less fuel than expected or the final thrusters (the TCM's) are failing. I hope theres another reason but I cant think of it :-(
Optimistically they are exploring strategies to lessen fuel usage with Voyager 2 (presumably s/c 32 vs. 31), the nearer/slower, before also sending to V1. Same as they are trialing the voltage regulator turnoff first on V2.
Googling says that hydrazine was estimated to runout in ~10 years from now for V2, and the vreg* article (me, above) talks to that date now overlapping the years that science instruments will be active. So it may well just be trying to ensure proactively that hydrazine is not the limit that Pu238 will be.
*vreg --> voltage reguator. Not a scrambling of vger.
Press release regarding various strategies of the team to deal with recent issues:
https://www.jpl.nasa.gov/news/nasas-voyager-team-focuses-on-software-patch-thrusters
https://blogs.nasa.gov/sunspot/
Voyager 1 issue
I would just add. Info from Canberra DSN is that no data is being recovered. Including the engineering channel but they have proved V1 is still responding to commands.
A close parsing of the nasa update says that, too. "No science or engineering data is being sent back."
It is then colored by a lot of what they've deduced. And yes, after reading it yesterday, I did have to remind myself... but they're getting nothing.
It does say they're scrutinizing old documents, and there sure is not a lot out there that I've just discovered in a quick search. The FDS is one unit, no A/B units, though it's redudant internally, I think. It also might be one of the first uses of volatile (presumably SRAM) memory. And that means, this could be an SEU. Whether there have been prior SEU that have done this to either V1 or V2 FDS, I can't discover.
Speaking of SEU. Found in IEEE Spectrum June 1987...
No news that I've heard this year. Anyone else?
From Twitter...... A Voyager update: Engineers are still working to resolve a data issue on Voyager 1. We can talk to the spacecraft, and it can hear us, but it's a slow process given the spacecraft's incredible distance from Earth.
We’ll keep you informed on its status.
Ars Technica did the good ol' fashioned thing and... called the project manager:
https://arstechnica.com/space/2024/02/humanitys-most-distant-space-probe-jeopardized-by-computer-glitch/
Interesting that it's mentioned there are two FDS on V1 and the other "failed in 1981" -- so whatever got me thinking there wasn't originally a spare was wrong.
That FDS history is better than anything else I found a few weeks back. And James Wooddell might be unhappy to hear them say that so many people of import have passed. It appears that he might be turning 91 before the month is out. The text cites a graduate paper from USC in 1974 as having a lot of documentation on the architecture.
That text does make it clear that the bigger issue is... what's the software on there, and if a bit's become stuck, what do you do about it. There, Edgar Blizzard is (maybe) 89. Richard Rice is a more generic name, but I can't see that he's necessarily passed, either.
I did sort of glean that the "registers" might simply be specific words in the memory, and "bad register" might be what means the other FDS was abandoned. Which might make a bad bit in a register harder to work around vs. like it was in a specific word of instruction memory like in the hard error in 1985. Yes, there are other registers, but rewriting that much of the code...
Oh, and it says a full FDS software load took 4hrs in 1984, which was then considered real slow because of lower bandwidth.
I'm not sure why you take issue with the simple claim that a lot of people who worked on the project are dead. And even if they're not, they may not be able to help for other reasons.
I note that the document talks about a software simulator that ran on an Interdata system. The 8/32 has SIMH emulator support (likely due to its use in the early development of Unix) so maybe there's some hope there. https://en.wikipedia.org/wiki/Interdata_7/32_and_8/32
FWIW, I reached out to the Voyager team and mentioned all of these possibilities.
Oh, I'm not taking any truck with it, just wondering about particular players. And actually surprised that I came up 0/3 on, well, obituaries for those three, and I was looking for that mainly to get a sense of when they'd have stopped being involved in things. All retired in the 90s, maybe. But I dunno.
It sort of describes the shape of the problem to me. Because it sure looks like a fix will mean changing to FDS firmware, and how lost to history that process might be. So here's hoping it doesn't.
Oh, and maybe I can square my prior "internal redundancy" comment with the Ars report that 1 of 2 FDS on V1 is dead-- your text does say the processor could address memory in either unit. So the system is semantically two intertwined units, one of which is dead on V1. So some higher level block diagrams might call that intertwined unit "the FDS" which internally has lost some redundancy.
The Voyager FDS was designed several years before the first microprocessors. It has some unusual architectural features, including 128 general-purpose registers (mapped from the main RAM and not as separate logic entities) and a six-clock basic instruction cycle operating on 4-bit values per clock.
I'm not sure how the Voyager team is proceeding. If I were faced with this problem, I would try to build the smallest possible software load that would send useful telemetry to the transmitter. And to support that, I would build a software simulator of the system and make sure the behavior of existing loads was understood. The FDS memory (8K, I think) is loaded through the CCS, so it should be possible to experiment a bit with new FDS loads without the possibility of bricking everything, assuming of course that the CCS keeps working.
What I gathered from Ars was simply that they're going to try to command it into "encounter mode" or some such other modes. Which make sense, try that before software reload, see what happens.
Floyd -- the very long PDF linked by mcaplinger above has lots of details, including that tidbit about the massive amount of registers. The images basically look like the "8K" memory is probably something like 256 discrete CMOS ICs -- making each 32 bits, so if each register is 4bits, maybe there are 16 chips for the "registers" and 240 for the rest of the memory. All that to say -- the reason the registers are mapped from "main" memory is because the MPU itself is a collection of discrete ICs on a huge board with probably hundreds of SRAM ICs... the memory bottlenecks aren't analogous to CPUs.
In any event, if there really were 128 x 4 = 512 bits of registers over 16 separate chips, simple programs probably don't need to use all 128. So I was thinking a bad register would be hypothetically easy to work around, esp since there's not image processing happening. There is some text on page 187 of the PDF about how DMA instructions take the same time that all other instructions take... or I don't quite follow. Probably the biggest distinction between the regs and the other memory words was that the regs were addressable with 7 bits of a 16-bit instruction, while the memory addresses were 4k/16=8bits? So separate instructions were needed to access the "lower 4k" instruction memory vs. upper 4k scratch vs. upper 4k scratch in other unit? Also not clear to me how a 16-bit memory word would load into 4-bit registers, but this is a special ISA, so perhaps 4 registers load at a time. The other thing indicated is that arithmetic would be slow... a several cycle operation because it was doing 4-bits a cycle. Perhaps a more common operation was to simply forward along specific ranges of data, or MSBs, etc.
All that to say, a bit flip in one of the registers shouldn't be more fatal than a bit flip in instruction memory... just even harder to work around at this stage, because you have to change and reload the firmware. And sure, if a 16-bit word has to load into 4 registers, maybe there are effectively 32 and not 128 registers, when it comes to programs loading from memory. But 32 --> 31 should still be manageable.
So, does anyone have a prognosis here? How likely does it seem that we'll get Voyager back?
I wrote a post on the situation here: https://crookedtimber.org/2024/02/19/death-lonely-death/
Written by a non-technical person for a non-technical audience, so please be kind to any errors.
Doug M.
Nice post, but I feel compelled to nit-pick a little, sorry.
What about the Pathfinder models that JPL occasionally trots out for "group portrait" photo ops to show the evolution of Mars rovers from, e.g., Pathfinder to Perseverance? Are those dummies without the electronic innards?
<groan>
Naturally I meant to refer to the Sojourner component of the Pathfinder mission.
A few ... well, quite a few years ago, Doug Ellison put up some interesting posts on twitter showing how he had borrowed a Sojourner-shaped object from JPL and taken it into his home shop, to spruce it up for a JPL open house.
I guess it's unlikely to have been a drivable model.
The only flight-like rover testbeds in the wild are Marie Curie (Sojourner testbed rover, at one point destined to fly on the cancelled 2001 lander) and 'Dusty' ( MER testbed ) which are both now at the Air and Space museum. There were others 'driveable' MER and Sojourner testbeds that were significantly lower fidelity - more like the 'Scarecrow' rover used for Perseverance and Curiosity. The Perseverance and Curiosity Vehicle System Test Beds are both in the garage at the Mars Yard at JPL. Perseverance and Curiosity also share an avionics testbed (MSTB) which is more analogous to what testbeds are usually like for missions that are not rovers/landers - the 'flat-sat' Mike mentions above.
This is Marie Curie, Dusty, and Maggie - the Sojourner, MER and MSL testbeds...
https://mars.nasa.gov/resources/3792/three-generations-of-rovers-in-mars-yard/
I don't know if Voyager has an equivalent to the MSTB functioning right now. I would be surprised if it does. I will say, keeping our testbeds up and running is VERY non-trivial. Having spares to fix them when things break gets harder and harder with age - and can reach a point where it's simply not possible to get the parts to do it. Older missions get parts poached from their testbeds in support of newer missions etc etc. If there IS a Voyager testbed, I suspect in terms of technicians certified to maintain it or operate it, the engineer trained by the engineer trained by the engineer trained by the engineer who built it probably got laid off 2 weeks ago.
mcaplinger the PDF you linked is amazing reading. Thank you for posting.
It might still be garbage (probably is), but DSS-63 Madrid is receiving at 40 bps from Voyager 1 right now.
NPR had some more color on this, yesterday:
https://www.npr.org/2024/03/06/1236033493/nasas-voyager-1-spacecraft-is-talking-nonsense-its-friends-on-earth-are-worried
No news except perhaps reconfirmation that they're trying mild commanding well before anything drastic, so perhaps expect this to take a long long while.
It looks like we have encouraging progress (sorry for the very long link) : https://www.pasadenastarnews.com/2024/03/09/jpls-voyager-team-extremely-hopeful-after-ailing-faraway-craft-shows-signs-of-former-self/?sfnsn=mo&mibextid=VhDh1V&fbclid=IwAR1HLD97SajQYV-Th6UM8EPGaQ3C6Qsg-VQAZ-UGkib39o_t4ctexhFHAQM
From the article: "Today, the Voyager team consists of only 12 full-time employees."
Interesting. Somehow I had gotten the notion that Voyager was no longer a full-time job for anyone, except, of course, when there's a problem. I guess that's incorrect?
Well they still need a permanent team to deal with the dwindling power supply, even if it takes years, the measurements to extend their lifespans are probably extremely well calculated. And you need at least some guys to maintain base knowledge of the probe.
Voyager costs $5-7M per year according to the 2024 NASA budget request https://www.nasa.gov/nasa-fiscal-year-2024-budget-request/
I expect most of that is for labor, not sure how DSN time is accounted for.
Also it's worth noting - the phrase " 12 full time employees " may not mean 12 people who are full time on Voyager. FTE is a unit of currency for work like this. 12 'full time employees' might be......4 people working 100% on Voyager, 8 people working 50% on Voyager, and 16 people working 25% on Voyager.
As an example - I know the power lead for both MSL and M20 also works on Voyager.
Well, this is looking more hopeful than I was expecting. V1 has provided a full memory read out of the FDS, presumably missing framing / sync so it took a bit of work to pull it out of the data stream. https://blogs.nasa.gov/sunspot/?fbclid=IwAR1sVNuIWv7dnvwPKVnh6-Y135n1zw3bK-MbPJJdpQTW3K_Q5h4e4iY3tyc
There's a 2022 documentary https://www.imdb.com/title/tt17658964/ Not sure if anyone else here has watched it, but good for insight into the current goings-on.
I gather a bare bones version of the FDS software was loaded and thats how comms has been restored. I assume its now a case of running tests to find the faulty area of memory.
In some old computer languages, the peek command was used to read a location in memory and a poke command to write to memory. No idea if they poked some addresses in memory to program the peek into memory that was returned????
Thanks Explorer1 for suggesting It's Quieter in the Twilight. It's available for streaming from PBS, and it's an interesting documentary, quite moving, as mcaplinger noted upthread.
And I now recall reading a good New York Times piece (gift link below) about the same team.
My speculation is that the "dump memory" routine was in instruction memory that they decided to command the FDS to try. I would not guess they changed the instruction memory, because then it'd be running the program they expected, not "eureka! it's a core dump." (Commanding might simply be "start program at this instruction address" but it's not really worth clarifying that because it'll come out in time.)
https://arstechnica.com/space/2024/02/humanitys-most-distant-space-probe-jeopardized-by-computer-glitch/
Media reports of a DSN engineer having a eureka moment are a poor game of telephone in that case. But, we've seen that before. Time will tell.
Going by the latest SFOS this looks very hopeful. FDS memory update was sent Friday. Upon the response V1 is getting close to 24hrs coverage! I'll be checking the DSN website to see if we get a data rate of 160bps (cruise mode) :-) https://voyager.jpl.nasa.gov/pdf/sfos2024pdf/24_03_28-24_04_15.sfos.pdf
Thought I'd post an update as clearly Voyager 1 did not return to science / cruise mode following the update. I gather the update however fully restored telemetry, hence the 24hr track afterwards. This article seems to confirm it is a memory failure and they are having to rewrite / move code to avoid the failed area. https://spacenews.com/nasa-optimistic-about-resolving-voyager-1-computer-problem/
This makes me wonder if the "died in 1981" side of the FDS has more working memory words in the lower region they use as instruction memory. In an earlier post, I wondered how well documented what the bad bits or words were from that 1981 failure.
That failure meant that when the FDS glitched this time, they could not just switch to the other side. (And the sides are intertwined, but there's a lower region of memory that's dedicated to the processor logic on either side, while either side can retrieve data from the upper memory of the other side)
Because if the 1981 failure was also bad memory, they're now contemplating what they didn't do then... which is to rewrite the programs to avoid the bad addresses. And that makes me wonder which side is more functional now that they're both bad.
https://blogs.nasa.gov/voyager/2024/04/04/engineers-pinpoint-cause-of-voyager-1-issue-are-working-on-solution/
NASA suspecting entire SRAM chip is bad. Saying 3% of memory, which I suspect is 256/8192=0.03125. That might make one chip 256bytes: 2048bits, 16bit word, 128 addresses. 128x16 is what we'd call it.
If it's instruction memory, that's 6% of instruction memory, but the FDS isn't processing many images these decades.
Very impressive work! Congratulations!
https://blogs.nasa.gov/voyager/2024/04/04/engineers-pinpoint-cause-of-voyager-1-issue-are-working-on-solution/
Maybe you meant to post the link to today's blog post?
https://blogs.nasa.gov/voyager/2024/04/22/nasas-voyager-1-resumes-sending-engineering-updates-to-earth/
Yes, I did.
Incredible to think that they both might actually make it to their 50th anniversary...
Just learnt that Voyager I will be 1 light day from Earth on Nov 18th 2026
Voyager 1 remains in engineering mode but it looks to me that science data is planned to resume Sunday 19th May. See FDS mode & data rate. https://voyager.jpl.nasa.gov/pdf/sfos2024pdf/24_05_16-24_06_03.sfos.pdf
Here we go : https://blogs.nasa.gov/voyager/2024/05/22/voyager-1-resumes-sending-science-data-from-two-instruments/
Ed Stone, Former Director of JPL and Voyager Project Scientist, Dies. https://www.jpl.nasa.gov/news/ed-stone-former-director-of-jpl-and-voyager-project-scientist-dies?utm_source=iContact&utm_medium=email&utm_campaign=nasajpl&utm_content=daily20240611
https://d2pn8kiwq2w21t.cloudfront.net/original_images/1-ed-stone_portrait_voyager_model.webp
Truly a great scientist. He will be missed.
So the machine has now officially outlived the man. Incredible, yet sad
The New York Times has put up its obituary for Ed Stone.
Gift link: https://www.nytimes.com/2024/06/14/science/space/edward-stone-physicist-dead.html?unlocked_article_code=1.z00.ADuh.U0-fL6JqUWOf&smid=url-share
My favorite part of this obituary is a link to a lengthy NYT Magazine Section profile of Stone written 34 years ago, which discusses how Stone “‘revolutionized the world of project science’”
Tom, many thanks for posting that and the links. In posting the NASA link on his passing, I had hoped that members of the forum would add additional links or comments.
https://www.ft.com/content/0f2dce04-ec78-4aeb-808c-cff5b5135699
Everyone interested in the details needs to read the above. Confirmation that the other side of the FDS was totally bad... 3333 was written to every address and 333B comes out. Turns out lots of 333Bs came out after a poke.
And then a young priest and an old priest rewrote enough of the FDS instruction code on paper w/o a emulator to make it work again.
In the last days I checked via DSNOW that V1 was again in the 40b/s mode...I hope its not again a problem like we see already. Anyone aware of this ?
Yes I had also noticed Voyager 1 had been in engineering mode 40bps for several days. Worried I asked a trusted source as to the reason and was told there was nothing to worry about and V1 would return to science mode shortly which she did on 3rd Sept I think. However I wasnt totally convinced and sadly this evening JPL has posted the reason. https://www.jpl.nasa.gov/news/voyager-1-team-accomplishes-tricky-thruster-swap The TCM thrusters were probably pretty clear when they were swapped to in 2018. The fact that they are now totally clogged after 6 years must mean that swapping to the original thrusters which were already partly clogged, they will become totally clogged very quickly and life expectancy cant be very long. Sadly I think we are close to the end :-(
Powered by Invision Power Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)