You can't entirely remove a user from your account as that would affect all the conversations they've been part of.
You can, however, deactivate a user by flipping off the Access Allowed switch in the Account User screen.
