The man has since spoken out – and explained why he thinks he’s ‘done nothing wrong’