TBH, I don't see enough of a problem to worry yourself with shaving every little bit. How does the flange look on the turbine?
I've also noticed that the merge collector diameter is quite small.... It looks like lovefab manifolds are like this and still make good power....... i don't think that porting the turbo flange will solve the problem, as on the manifolds I have seen, also the merge collector was smaller than the turbine housing....