Issue of immigration a key concern ahead of Brexit referendum