The "automatic google search after not finding results" (Try anyways mode) seems to be incompatible with Xon's "Search Improvements" addon. It only worked after I disabled Xon's addon.
Also should mention here that this addon seems to work on 2.3 Betas. There are some minor issues at my end. Like you cannot select Google tab on the forum's search result page.