Solar Sail Transfers under Uncertainties: A Deep Reinforcement Learning Approach

A deep reinforcement learning approach is used to analyze the optimal 3-dimensional interplanetary transfers of a solar sail, accounting for various sources of uncertainty. The propulsive acceleration of the sail is described using an optical thrust model, with nominal optical coefficients derived from recently published experimental measurements. Two primary sources of uncertainty in the solar sail are considered: the imprecise knowledge of the sail’s optical properties, which impacts both the magnitude and direction of the propulsive acceleration, and the presence of wrinkles on the sail due to the folding (prior to launch) and unfolding (after release on orbit) of the ultrathin membrane. The study begins with a minimum-time interplanetary trajectory obtained using an indirect optimization technique in an unperturbed scenario, serving as the reference trajectory for the sail in the presence of model uncertainties. To account for these uncertainties, a proximal policy optimization algorithm is used to train an agent that learns a control policy associating any orbital state with the corresponding sail attitude, minimizing deviations from the reference trajectory. Two distinct scenarios are analyzed, each incorporating the aforementioned sources of uncertainty. The trained control policies are then tested through Monte Carlo simulations to evaluate their effectiveness and robustness. As a case study, a 3-dimensional transfer from Earth’s orbit to Venus’ orbit is examined, demonstrating that the control policy derived from reinforcement learning is capable of guiding the sail to its target with good accuracy, providing real-time control with relatively low computational effort.