The purpose of this study was to evaluate and optimize the performance of a multi-atlas based method for the segmentation of prostate in CT scans improving it up to the limit of the inter-observer variability. We assessed and optimized the atlas selection, the Non-Rigid Registration (NRR) and the label fusion steps by introducing new similarity measures based on image features and a multi-scale weighted majority voting. Cross validation results on 45 CT images suggested that the similarity measure based on the local feature histogram of oriented gradients outperformed classical intensity-based metrics for atlas selection. Besides, the NiftyReg optimized in a region of interest was found to be the optimal NRR algorithm. For the label fusion, the multi-scale weighted majority voting outperformed other approaches. All those improvements led to Dice scores of 0.84 +/- 0.03, which are comparable to the inter-observer variability for manual contouring.