London judge Jonathan Swift rejected Kingsley Kanu's application that the UK foreign ministry should rule on whether his brother, Nnamdi Kanu, was extraordinarily rendered. and if so found, call for his release.
Swift rejected the brother's argument that the foreign secretary had "acted irrationally" by not doing so, and said the minister had the right to determine what was in the UK's diplomatic interests.
Nnamdi Kanu, a dual British-Nigerian citizen, heads the Indigenous People of Biafra (IPOB) movement, which wants a separate state for the Igbo people in southeast Nigeria.
The Court of Appeal in Abuja ruled in October last year that he was abducted, ill-treated and "illegally moved" from Kenya to Nigeria to face treason and terrorism charges.
The judges dismissed the criminal case but Nigerian prosecutors have appealed and Kanu, who is in his mid-50s, remains in custody.
The Kanu family lawyers have argued that the case should lead to a reassessment of what official help is given to British nationals held abroad, particularly when there are suspected human rights violations.
Kingsley Kanu welcomed the fact that the court had recognized that his brother was subject to extraordinary rendition.
"But I find it very disappointing that in spite of this, the High Court has held that the British authorities are not required to reach a decisive view on his rendition and can refuse to confront the mass of evidence of what has occurred," he added.
"Until the British authorities reach such a view, it is impossible for them to properly consider what steps should be taken to assist my brother."
He added: "The ruling sets a dangerous limitation on the obligations the British authorities have to any British national who has been detained by a rogue state abroad."
The family is now hoping to appeal, he added.
Nnamdi Kanu, a former London estate agent who also runs the outlawed Radio Biafra station, was first arrested in 2015 but jumped bail two years later, reappearing in the UK and Israel.
The Nigerian government has banned IPOB as a terrorist organization, accusing it of stoking ethnic tensions by claiming genocide against Igbos.
In 1967, a unilateral declaration of an independent republic of Biafra led to a brutal 30-month civil war that left more than one million people dead, most of them Igbos.