When you start observing those patterns, those energies that happen in you, you will see that fear, love and hatred all happen in one region, and it is the same energy which manifests love, hatred or fear.
When you have some passion; if you are passionate about something, fear disappears. Fear appears when you don't have any specific passion in life. Isn't this the experience? Only when you have passion, or strongly hate something, then fear will disappear. Fear will disappear when you either have a strong hatred or strong love towards something. In the absence of either, a little lingering fear comes up. But, again, more hollow and empty meditations, pranayama, engaging in some seva activities; all this will help in eliminating that.