Purpose: We present a new method for knowledge-based isocenter selection for treatment planning in radiosurgery. Our objective is to develop a prediction model that can learn from past manually designed treatment plans. We leverage recent advances in deep learning to predict isocenter locations in treatment plans in order to provide a decision support tool. Methods: The proposed method adapts a geometric approach using orthogonal moment expansions as a feature vector for describing the shape of the tumor. Our approach accounts primarily for tumor shape and OAR proximity, the two factors that are known to greatly affect the isocenter placement. We solve the prediction problem by training a residual neural network with skip connections on the formed shape descriptors. Our network was trained on 533 patient cases and was validated on a set of out-of-sample cases. Results: Our method generates heatmap predictions for isocenter locations that are in most cases comparable to the experienced human planners, which shows that the method can be used in treatment planning to guide the users for determining the isocenters. Conclusions: Our numerical experiments indicate a positive predictive value on an independent validation set when compared against a test dataset that was not seen by the model during training.