Hmm... problem here lies in algorithm area - not in Direct3D.

Photoshop while blending layers (in normal mode) is using different function than SrcAlpha / InvSrcAlpha. I'm not sure is Photoshop function is commutative, but SrcAlpha / InvSrcAlpha definetly is NOT!

So, while blending surfaces S1, S2 and S3: (S1 (S2 S3)) <> ((S1 S2) S3).
You can figure it out by yourself if you take a piece of paper and try to do all calculations by pen

So - you can't - you have to workaround