webbrowser.open with file: URLs may launch editor instead of browser

[minrk]

Bug report

Transferred to https://discuss.python.org/t/support-for-file-urls-in-webbrowser-open/81612 until consensus is reached.

Bug description:

In the code:

import webbrowser
from pathlib import Path

index_html = Path.cwd() / "index.html"
webbrowser.open(index_html.as_uri())

It is common for xdg-open (on linux) or open location (on mac) or os.startfile (on windows) to be selected as the default browser. This is great and simple for http[s] urls, but when these are passed file:// urls, all of the above select the default opener for HTML files, which may not be a browser at all (commonly a text editor for developer systems).

I don't know if there's a way to indicate that Browser entries can handle given URL schemes, but when selecting a browser from the try list, ideally these generic 'open' mechanisms should be skipped for file: URLs. Even better, use standard APIs to lookup the default browsers for http[s] and invoke that explicitly, rather than relying on an interpretation of "open URL" which is only valid for http URLs.

I believe URLForApplicationToOpenURL is the current macOS API to look up the application for http:, which could be used to launch explicitly with a browser. I guess xdg-settings get is already the equivalent for most linux situations and preferred if available, which is great. I'm not sure there's a more general way to discover what xdg-open will do with http://, but we have occasional reports where xdg-settings get default-web-browser doesn't work and xdg-open is invoked and doesn't launch a browser (or launches a different browser). I've no idea what the Windows call would be, but this looks like the same question.

This appears to have been opened long ago as #37540 and erroneously closed as an Apple bug (all platforms exhibit this behavior, and I think the bug is pretty clearly in webbrowser's assumption that "open file://path.html" means "open file://path.html with a web browser" which does not appear to be what any of the platforms mean by the chosen API).

CPython versions tested on:

3.9, 3.10, 3.11, 3.12

Operating systems tested on:

Linux, macOS, Windows

Linked PRs

• gh-128540: Fixed webbrowser.open with file: URLs may launching editor instead of browser #130031
• gh-128540: lookup default webbrowser on macOS #130535
• gh-128540: lookup default webbrowser on Windows #130538
• gh-128540: launch default browser more often on Linux #130541

[Wulian233]

I made a PR #130031

[ned-deily]

Thanks for the issue. And for the PR. But I don't think the issue correctly describes the current behavior and I'm not sure how we could change it without creating incompatibilities.

I am most familiar with the macOS behavior. As an example, the Python IDLE app on macOS uses webbrowser.open to open file:// urls that are documentation html files and which are intended to be opened in the user's default browser. By default, that works fine. On modern macOS systems, I believe it is the case that what application is used to open a URL is not just determined by the protocol part of the URL but also by hints in the path itself such as an extension or lack thereof. That is, by default, from the command line:

open "file:///path/to/index.html" # opens in a browser
open "file:///path/to/file" # opens in the default editor

And, of course, the user has some control over LaunchServices file association settings via the Finder.

The macOS open command, which is what is used by webbrowser.open, does have a -t option to open a file with the user's default text editor and also has a -a to specify the application to use to open the file but does not have an "open with default browser" option. And, while there are hacky ways to find the user's current default browser, I think that would be overkill for this.

I would imagine there are similar gotcha's on other platforms.

The current documentation for webbrowser.open states:

Note that on some platforms, trying to open a filename using this function, may work and start the operating system’s associated program. However, this is neither supported nor portable.

So there are definitely programs out there expecting to use file:// URLs with webbrowser.open and have been for many years. And it seems that webbrowser.open is behaving as documented. Changing the behavior at this point seems to me to be ill-advised especially since it is not always clear what behavior the programmer expects versus what the operating system will do in a particular user environment. I think it would be best to leave it up to users of webbrowser.open to decide on and program for the desired behavior, for example, by checking user-supplied URLs before calling webbrowser.open.

Other opinions?

[minrk]

We at least have many years of people being confused and surprised and frustrated by the current behavior in Jupyter. Plus, there is the factual inaccuracy in the name "webbrowser.open" doesn't open a webbrowser when you give it a URL.

I think the linked note in the docs mentions "filename", not "file:// url", which to me is a different thing, but I could be wrong. If indeed this case is really that file:// URLs are "unsupported" as that bit of the docs suggests, then consider this a feature request to support them.

[ned-deily]

I agree that the documentation is ambiguous: I assumed that filename meant a file:// URL but you are right that it could be read to mean a plain OS file path. I doubt that is what was intended and a file path doesn't seem to work with webbrowser.open on macOS at least.

I also agree that we should explicitly support file:// URLs. In fact, that is why I commented here because the proposed PR would disallow such URLs. So we shouldn't do that.

As far as whether we should or realistically can ensure that webbrowser.open does open a web browser across all supported platforms, I don't have as strong an opinion. In principle, it sounds like a good thing to do if it can be done without too many hacks. I don't have any particular insights into how that could be done on all platforms. If someone wants to propose a design and an implementation, that would be great; perhaps it is something that could be discussed on discuss.python.org to get more input about the various platforms. In the meantime, I think we should limit a PR at this point to updating the documentation.

[minrk]

I opened a discussion and implemented a prototype that works on all 3 platforms, at least in my tests.

I think it may still be correct and preferable to disallow file: URLs on the openers that are known to launch default applications which may not be browsers (mac open, xdg-open, startfile, etc.), allowing the existing fallbacks to select unambiguously Browser openers.

[ned-deily]

I opened a discussion and implemented a prototype that works on all 3 platforms, at least in my tests.

Thanks for doing both. @ronaldoussoren would be the best person to review the macOS part of your prototype.

[ronaldoussoren]

I've looked at the Mac bit of the prototype, it uses the NSWorkspace class to find the default browser of the user, and that uses the MacOSXOSAScript opener with that browser.

A way to to this while doing all work in AppleScript:

diff --git a/Lib/webbrowser.py b/Lib/webbrowser.py
index d2efc72113a..6eeabe2587b 100644
--- a/Lib/webbrowser.py
+++ b/Lib/webbrowser.py
@@ -598,6 +598,25 @@ def open(self, url, new=0, autoraise=True):
             url = url.replace('"', '%22')
             if self.name == 'default':
-                script = f'open location "{url}"'  # opens in default browser
+                script = f'''
+                    use framework "AppKit"
+                    use AppleScript version "2.4"
+                    use scripting additions
+
+                    property NSWorkspace : a reference to current application's NSWorkspace
+                    property NSURL : a reference to current application's NSURL
+
+                    set wurl to NSURL's URLWithString:"https://www.apple.com"
+                    set thisBrowser to (NSWorkspace's sharedWorkspace)'s ¬
+                        URLForApplicationToOpenURL:wurl
+                    set appname to (thisBrowser's absoluteString)'s lastPathComponent()'s ¬
+                        stringByDeletingPathExtension() as text
+
+                    tell application appname
+                       activate
+                       open location "{url}"
+                   end tell
+                   '''
             else:
                 script = f'''
                    tell application "{self.name}"

This has two (minor) advantages over the prototype:

1. The prototype uses a system framework, which in the past has proven to be problematic in code using os.fork without calling os.exec (see old closed issues about _scproxy in programs using multiprocessing).
2. The code above fits cleaner into the current code for for MacOSXOSAScript (in the eye of this beholder)

BTW. I would have preferred either a PR or an explanation of what the prototype does and why.

[minrk]

Thanks! I didn't know you could make the appkit calls directly from AppleScript, I'll update it, that is indeed far simpler.

FWIW, the full description of the prototype is in the Python-ideas post, as best I could do it. My impression was that was preferred before a PR.