I thought that no-for-real-I'm-not-this-Fujitora-fellow guy would end up not confronting Luffy because he wouldn't be able to "see" the corrupt/rotten "presence" common of many other pirates (a la Boa not being able to turn Luffy to stone because he didn't lust after her), so he would never know that was Luffy. Now I don't think he will confront the Strawhats because he has found more important business to take care of in Dressrosa. I'm sure even Akainu could be happy knowing that I've-never-even-heard-of-a-Fuji'd-Torah settled a bigger matter/fish for the government.
Also, glad they separated, less all of them forget that they are supposed to be destroying a factory and rescuing a samurai. Zoro, Sanji, and Kinemon shall do this. Luffy and Franky shall handle the arena and Ace's fruit.