While Charlie did not have concrete proof that the three elders from the Elder Council of the Qing Eliminating Society were waiting for him in Nigeria, his instincts told him that they were there.
After all, Fleur would have understood that she faced an unknown but formidable adversary after losing the Four Great Earls. At this point, they couldn't afford to split up their forces, but were more likely to concentrate them in one place for a higher chance of victory.
In fact, Fleur must hate him