I think you have too much time to sit and think, 'Am I the doer or not'. Just do what you are suppose to do.
Every action has a reward; pleasant, unpleasant, or mixed. There are only three types of rewards. When you act, that action brings you some positive reward, or some negative reward, or some positive and negative reward. The effect of all these rewards stay only for some time. So the wise will not worry about it.
In every action there is something positive and there is something that is negative. An action cannot be 100% perfect. Focus on the positive things that develop out of an action. And don’t worry about the effect of the action, just keep moving.
If you have done a wrong action, that has taught you a lesson not to repeat it next time. If you have done the right action, it encourages you to do many more of the same right actions.
If you find that in the action you did, there are some things that you could improve, then you can always improve on that.
Just move on, don’t say, 'I am the doer', or, 'I am not the doer'. Anyway no one is a doer in one sense, and in another sense, yes, you are the doer, because inside of you there are two.
There is a beautiful story in the Upanishads which says, 'On the tree of this universe, there are two birds sitting, one is enjoying the fruit, the other is just watching, like a witness'. So, there is one aspect of you that is not doing anything, that is much bigger. And there is one aspect in you that is doing everything. Both are present in you.
Now you might ask, 'How?'
On one level, you find that you are changing. Your thoughts are changing, your emotions are changing, your body is changing. Yet at another deeper level, there is something in you that is not changing at all; it is the same. So it is not one or the other, it is both together simultaneously.
When guilt eats you up, or pride takes over your head, then you should understand that you are not the doer. Do you get it?
When you do something very good and you say, 'Oh I did it, I did it', that too makes you unhappy. Doing something good also make you unhappy because you do so much and then you say that nobody recognizes you, nobody listens to you.
So not being recognized for your good work makes you miserable, and if you do something wrong, that too eats up your spirit and creates a sense of guilt. When such things happen, know that you are not the doer. Be a witness then.
People who have committed the biggest crime on the planet, somewhere deep inside, they think, 'Hey, I didn’t do it, how did this happen'.
They feel surprised and shocked, and they feel that they are not the doer, it just happened.
If you talk to the biggest criminals on the planet, you will see that they find themselves being caught by an (criminal) act. Same way, someone who has done some excellent work, they also feel that it just happened; everything fell into place. That non-doership is an experience in itself.
When you have to do things, then don't say, 'Oh, I am not the doer'. Don’t go into that mode. It will only tear you apart. If you have to focus on some work, focus and just do it.
I am not a doer, is an experience, sometimes it just dawns on you. And it is also a reminder to you when you are reeling in guilt, or in arrogance.