Bug: Starting server V2 results in "Address already in use" error, when its not in use.

[Nurb4000]

Contact Details

No response

What happened?

Latest release on github. If i try to start with "--server --v2" it throws error "127.0.0.1:8080: Address already in use"

Port is not in use. Using default v1 server works fine. Tried changing the port on the command line, still same result, just shows the 'new' port and claims its in use.

v2 server worked fine for me on the previous release and i prefer it over the v1 server. But i do realize its still in development.

This happens both via the 'standalone' executable, as well as if i do a 'convert' of a GGUF into a single file.

Debian Linux. Latest updates of this week.

Version

llamafile-0.9.1

What operating system are you seeing the problem on?

Linux

Relevant log output

[corebonts]

Have you also tried it with -l localhost:8080 or -l 0.0.0.0:8080? Should not matter but who knows? :)

[Nurb4000]

Have you also tried it with -l localhost:8080 or -l 0.0.0.0:8080? Should not matter but who knows? :)

I did try changing port to a few random ones, with no luck, before i posted, however i did not try specifying the listen address as well. Adding that did help ( went with 0.0.0.0 as i want this to be visible on another machine, it runs headless ). Not required in v1 server as that 'just works', but hey, if that is just the way v2 works now, as long as its documented somewhere eventually....

i do suspect this is still a bug, since the previous release ( 0.9.0 ) didn't need this and i didn't see it documented ( yet at least ).

thanks for the suggestion.

[corebonts]

So this is how it's documented on the help (--help):

   -l HOSTPORT, --listen HOSTPORT
           Specifies the local [HOST:]PORT on which the HTTP server should
           listen.  By default this is 0.0.0.0:8080 which means llamafiler
           will bind to port 8080 on every locally available IPv4  network
           interface. This option may currently only be specified once.

The problem is that as I see the code, this is definitely not what is actually happening. The default seems to be:

• 127.0.0.1:8080 if nothing is defined (I think this is an older change)
• if something is defined without the colon, that will be the address and not the port (so the port is optional in the code, not the host, as the help suggests, this is new in 0.9.1)

But anyway, this does not explain the exact error what you have.
What I can imagine that you really have something on that port, but only for 127.0.0.1, not for your external address.
You may check it with ss -tulnp sport = :8080

[Nurb4000]

Oh, i didn't mean the parameter wasn't documented, that it wasn't documented that it was required to do so when using v2 server.

And yes i am sure nothing else was running that was listening on that port. When it first failed i also tried other random ports just to see if v2 didn't like the 'default' port for some odd reason. Besides, if there was something else that had taken that port already, v1 server should have crashed with the same result.

But with specifying the address, it works.. so its not totally broke... just not 'right'.

(oh and going back to 0.9.0 it still works like before.. i did that too, just to be sure it was a 0.9.1 v2 thing and nothing else was at play )

[Nurb4000]

Had a few spare minutes, built a vm to test. same result, but same workaround works. If its just me for some strange reason, sorry about that.