This is not relevant, and if it is, all the more reason to believe your motivation for voting was malicious.
If you can call him out and condemn him as scum for passive-aggressive reference to other players, then your own method is scummy for the same reason. As you said, your vote for him stands because you believe he lashed out. Now some things to note:
1) You originally called it lashing out, and then described what he should have done as something which was far more 'lashing out' or aggressive, meaning there was no actual hope for him to be able to get a town read off you.
2) You underplayed your original claim by citing text-book scum plays that applied to him. What you cited, namely the 'lack of aggression' = scum, because scum would come right out and say it flatly contradicts your initial reason for voting him.
3) When you attempted to justify it, you came up with "Passive-aggression." Which is not actually what you were claiming when you were elaborating the reasons previous. Hence you've changed your claims yet again.
4) The reasons which you've cited as evidence against him I'd like to use as a more valid construction for a vote grounds against you. Namely, you are scum because:
a) Instead of 'saying so,' you gave arbitrary and highly ambiguous reads that concluded a vote against Spiffeh was justified.
b) You lashed out against Spiffeh (is equally valid to your original claim, which is to say, not valid at all)
c) You're blatantly performing 'doubt-casting' on Spiffeh, by making a claim against him that can be retracted into: "Oh, it was passive-aggression."
Your play is scummy as fck let's be honest.