I believe Nadella was referring to a more general sentiment in the machine learning community about our lack of theoretical understanding of deep neural nets. I'd suspect that the network still learns some higher level features that exist across languages which is why it could become better in English after learning German.