Fremantle mayor Brad Pettitt realises he has a fight on his hands to get the WA government to part with the prime real estate.