Give up phil.
If you were talking to a Bengali, then he/she will know exactly what you mean by gravy.
It's not the "knowing"; it's the "being able to replicate". Of course he/she will know what "gravy" means, if we are discussing BIR food as opposed to roast beef; but if I insist on pronouncing it "gravy" when I know that he/she is going to have difficulty with replicating that pronunciation, then I am indeed in danger of giving offence, because I am tacitly saying "listen to me, I can say the word properly", whereas if I adapt to his/her pronunciation then I am trying to ease the conversation and make it easier for both of us.
And is that the only word in the whole sentence that you're going to change? And are you going to speak with a Bengali accent as well? Come on. :
I can't speak with a Bengali accent because I am not sufficiently familiar with the language; but when (for example) I order "two chapati", I do my best to bring the sound as close as I can to to the sound that I know the waiter will make when he repeats my order, and I avoid the English plural "s" because I know that Bengali plurals are not formed that way. It's no different (to my mind) to trying to get the tones right when ordering dim sum in a Chinese restaurant; it shows respect to my hosts, whose language it is that I am trying to speak. OK, "garabi" is not a native Bengali word, but the English word "gravy" has been adopted by Bengali speakers and adapted to make it better fit Bengali phonological patterns, just as we have adopted thousands (literally) of words from the sub-continent and adapted them to better conform to English phonology (tiffin, veranda, bungalow, kedgeree, dungarees, gymkhana, pyjamas, coolie, chutney, bazaar, thug, chokey, amok, loot, ...) [see "Hobson Jobson" for thousands more].
p.s. I hate being called Fronk! 
Spell your name with an "i" ("Frinc") and the French will happily call you "Frank"

** Phil.