3D Projecting JunoCam images back onto Jupiter using SPICE Options

[Matt Brealey]

Hi Everyone,

As part of an upcoming VR/VFX-based project, I'm attempting to project the raw images from JunoCam back onto a 3D model of Jupiter, the goal being to do this precisely enough that multiple images from a specific perijove can be viewed on the same model. However, I’m currently having one or two issues calculating the correct JunoCam rotation values, and was hoping someone with more experience might be able to point me in the right direction!

My process for processing a single raw image is as follows :

1. Recombine the raw framelets into multiple exposures, each matching the layout of JunoCam’s CCD/the original exposures.
2. Undistort these original exposures.
3. Get the correct position/rotation of JunoCam using Spiceypy and the relevant kernels.
4. Project the undistorted captured images through the now correctly positioned/rotated 3D camera onto a 3D model of Jupiter
5. Get the position of the sun relative to Jupiter, and use this to remove Jupiter’s specular highlight in the resultant image.

Thanks to Gerald’s fantastic documentation I have reasonably confidently achieved steps 1-2 (thank you very much Gerald!) Using a meta-kernel generated by Boris Semenov along with additional ck/spk kernels as required, I’ve also managed to get the position/rotation data required in steps 3-5. However, my rotation values for JunoCam always seem to be slightly off in a suspiciously consistent way, making me think that I’m perhaps misunderstanding the frames involved.

The frame transformations I'm calculating are currently : J2000 >> IAU_JUPITER >> JUNO_SPACECRAFT >> JUNO_JUNOCAM_CUBE >> JUNO_JUNOCAM.

Here's the snippet of Spiceypy code used :

CODE

#Calculate the rotation offset between J2000 and IAU_JUPITER
rotationMatrix = spice.pxform("J2000", "IAU_JUPITER", exposureTimeET)
rotationAngles = rotationMatrixToEulerAngles(rotationMatrix)
j2000ToIAUJupiterRotations = radiansXYZtoDegreesXYZ(rotationAngles)

#Get the positions of JUNO_SPACECRAFT relative to JUPITER in the IAU_JUPITER frame, ignoring corrections
junoPosition, lt = spice.spkpos('JUNO_SPACECRAFT', exposureTimeET, 'IAU_JUPITER', 'NONE', 'JUPITER')

#Get the rotation of JUNO_SPACECRAFT relative to the IAU_JUPITER frame
rotationMatrix = spice.pxform("IAU_JUPITER", "JUNO_SPACECRAFT", exposureTimeET)
rotationAngles = rotationMatrixToEulerAngles(rotationMatrix)
junoRotations = radiansXYZtoDegreesXYZ(rotationAngles)

#Get the rotation of the JUNO_JUNOCAM_CUBE to the SPACECRAFT, based upon data in juno_v12.tf
rotationMatrix = np.matrix([[-0.0059163, -0.0142817, -0.9998805], [0.0023828, -0.9998954,  0.0142678], [-0.9999797, -0.0022981,  0.0059497]])
rotationAngles = rotationMatrixToEulerAngles(rotationMatrix)
junoCubeRotations = radiansXYZtoDegreesXYZ(rotationAngles)

#Get the rotation of JUNO_JUNOCAM to JUNO_JUNOCAM_CUBE, based upon data in juno_v12.tf
junoCamRotations = [0.583, -0.469, 0.69]

The resultant position coordinates and euler angles are then transferred to a matching hierarchy of objects in 3D. The result is a JUNO_JUNOCAM 3D camera that should in theory be seeing exactly what is shown in the matching original exposure from steps 1-2.

Using the 8th exposure (calculated time 2017-05-19 07:07:11.520680) of this image taken during perijove 6, zooming in on the final projected result gives me the following image :



As you can see, the RGB alignment is close, but not quite perfect. After doing some additional experimentation, I realised that the solution was to add an additional X rotation of -0.195° to the camera for exposure 1, -0.195°*2 for exposure 2, -0.195°*3 for exposure 3 etc. Applying this additional rotation gives me the following image :



Still not perfect as this value is no doubt slightly inaccurate, but it’s certainly a lot closer. The consistency of this additional rotation (plus the fact that the value required changes between different raw images) makes me think that I’m perhaps missing something obvious in my code above! (I am admittedly not yet compensating for planetary rotation but I wouldn't expect that to cause a misalignment of this magnitude).

Problem 2 is that in addition to the alignment offset between exposures, there is another global alignment offset across the entire image-set. Looking again at the 8th exposure, this time in 3D space, we see the following :



The RGB bands are from our single exposure, which has been projected through our 3D JunoCam onto the black sphere, which is my stand-in for Jupiter (sitting in the exact centre of the 3D space). As you can see, there is an alignment mismatch. After yet more experimentation, I found that applying a rotation of approximately [-9.2°, 1.7°, -1.2°] to the entire range of exposures (at the JUNOCAM level) more closely aligns our 8th exposure with it's expected position on Jupiter's surface, however this does introduce a very slight drift in subsequent exposures :



I hope that all makes some sense! I'm (probably fairly obviously) new to SPICE, but after reading multiple tutorials/forum posts elsewhere, I still can't seem to pinpoint the one or two things I'm no doubt misunderstanding!

I'm incredibly excited about what's possible once I iron out the issues in this process, so if anyone has any suggestions, I'd certainly love to hear them!

Thanks very much in advance!

Matt

[Brian Swift]

Question for everyone with a raw pipeline using the juno_junocam_v02.ti lens parameters - what is the magnitude of the color mis-alignment you observe (particularly near the edges of the CCD), sub-pixel, pixel, or several pixels?

[mcaplinger]

Question for everyone with a raw pipeline using the juno_junocam_v02.ti lens parameters - what is the magnitude of the color mis-alignment you observe (particularly near the edges of the CCD), sub-pixel, pixel, or several pixels?

The standard deviation of our model validation efforts to date (based on comparing the observed vs predicted position of the galilean satellites on orbit 1) is about 1 pixel in X and about 2.8 pixels in Y, but that probably doesn't get more than halfway to the edge of the field. The presumption is that the larger Y error is due to timing slop.

Try as we might, I'm not expecting that we will ever come up with a subpixel-accurate model that works across the whole field without some kind of ad hoc adjustment (at a minimum, a per-image timing tweak). I'd love to be proven wrong.

[Brian Swift]

The standard deviation of our model validation efforts to date (based on comparing the observed vs predicted position of the galilean satellites on orbit 1) is about 1 pixel in X and about 2.8 pixels in Y

Thanks. Thats of the order of what I'm seeing with my current lens model which is based on CruisePhase and ORBIT_ 00 "stars".