Judge Sir Geoffrey Vos dismissed the appeal on Friday, May 2, adding that the two other judges agreed with his opinion.

Delivering the verdict, Vos said, "The Duke was, in effect, stepping in and out of the cohort of protection provided by RAVEC [the government’s Royal and VIP Executive Committee]. Outside the U.K., he was outside the cohort, but when in the U.K., his security would be considered as appropriate depending on the circumstances. It was impossible, I said in my judgment, to say that this reasoning was illogical or inappropriate. Indeed, it seemed sensible."

"Even if there had been a risk analysis from the Risk Management Board, it would very likely have only confirmed the threat, vulnerability and impact levels which the Duke of Sussex had faced when earlier risk analyses were undertaken," the judge added. "But it would have had nothing to say on the critical features of the changed situation, namely the need for protective security on future uncertain visits and the government's appetite for risk."

The judge called RAVEC's decision on Prince Harry's security "understandable and perhaps predictable."

The Duke of Sussex, 40, traveled from his California home to attend the two-day hearing at the Royal Court of Justice in London on April 8 and 9. For years, he’s been fighting for the restoration of automatic, state-funded security during his visits to the U.K., and as he left court on April 9, he told PEOPLE he was feeling "exhausted and overwhelmed."

For Prince Harry, this legal battle has been described by his legal team as a fight for his life.

In February 2024, Harry lost his initial case against RAVEC’s decision to downgrade his security, which he appealed. A source close to his legal team had previously said they were "cautiously optimistic" about the outcome.

The Duke of Sussex said that his "worst fears have been confirmed by the whole legal disclosure in this case — and that’s really sad." Though he has juggled multiple court cases in a strike back against the British tabloid press, this particular fight is crucial for his future and his family with his wife, Meghan Markle.

"This one has always mattered the most," Prince Harry told PEOPLE.

The Duke and Duchess of Sussex lost their police protection in the U.K. when they stepped back from royal duties in 2020. Harry's lawyers said in court on April 8 that he and Meghan "felt forced to step back from the role of full time official working members of the royal family as they considered they were not being protected by the institution" and had "wished to continue their duties in support of the late Queen as privately funded members of the royal family."

Their security was soon downgraded, but Harry's legal team maintains that RAVEC failed to properly assess the unique risks posed to Harry and his family. Instead, RAVEC gave a case-by-case arrangement based on current needs, which his side says his "inadequate, inappropriate and ineffective."

Prince Harry and Meghan are raising their children, Prince Archie, 5, and Princess Lilibet, 3, in California, and his lawyers have said previously that he "does not feel safe" bringing his family to his home country without official police protection.

Exacerbating the fracture, the Duke of Sussex has long maintained that as ceremonial head of state, his father, King Charles, could help restore the security he is fighting for (an idea the palace denies).

Amid the courtroom clash, a source close to the palace tells PEOPLE there's still distance between the father and son: "I don’t think there is any rapprochement. Nothing has changed."