Prime Minister Anthony Albanese spent weeks away from his Sydney residence earlier this year as the government bolstered security and built a blast-resistant safety bunker in Kirribilli House, underscoring anxiety about escalating threats to leaders on both sides of the political divide.