Correct use of either in a conversation

Correct use of either in a conversation

In the following conversation:
Person A: I don't like Bob
Person B: No, he's annoying. Mind you I don't like Barry, either.
It is my understanding that the use of the word "either" is appropriate.
As, according to Oxford English dictionary, either can be defined as:
2 [adverb, with negative] used to indicate a similarity or link with a
tatement just made:
You don't like him, do you? I don't either it won't do any harm, but
won't really help either
for that matter; moreover:
I was too tired to go. And I couldn't have paid, either
In this case, either is being used in the "for that matter; moreover"
context.
I am having it argued to me that the use of the word either is not
appropriate here as it is said by Person B in response to person A's
original statement of not liking Bob, and saying "I don't like Barry
either" in direct response to "I don't like Bob" doesn't make sense. I
believe that as Person B has already agreed with Person A's initial
assessment of Bob, the use of the word "either" when referring to Barry is
totally valid. That is that the "either" being used by Person B is in fact
relating to Person B's first sentence and not in direct response to what
Person A originally said.