It would make sense to use a FrameLayout instead of a LinearLayout with one child view. This change is most likely permanent. From the deleted answer and comments, I see you are using this library and this question is related to this issue on GitHub.
My advice would be to discontinue using that library. You should be using a PreferenceFragment instead of the deprecated PreferenceActivity. If you want to have preferences with material design on pre-L you should use the official v7 preference support library. However, last I checked this had some pitfalls.