I also have problems with the different domain setup. On the moment I have two languages, dutch (default) and english. On home switching between domains goes like it should. If you visit a page and switch from default to another language all is fine as well. But when you try to switch from a page (non default language) to the default language, the anchor points to the right slug, but the current domain instead of the target domain. It seems the like the permalink filter processes the value.
I tried the widget, added it to the navigation and tried the development version from above. After a little bit of digging everything works by adding this:
include/switcher.php:68 (added after url apply filters)
$url = str_replace(get_home_url(), $language->home_url, $url);
Maybe it's not the right solution, but I couldn't find the appropiate filters.
My setup: I have a WordPress Multisite setup and am using the WordPress MU Domain Mapping plugin to add the second domain as an alias to the (sub)site where Polylang is active.
Can you please look into it? Thank you in advance, I really like your plugin so far!