Skip to content
This repository was archived by the owner on Dec 17, 2024. It is now read-only.

unhealthy selenoid-ui if recreate selenoid container separately #277

Closed
patsevanton opened this issue Dec 27, 2019 · 7 comments
Closed

unhealthy selenoid-ui if recreate selenoid container separately #277

patsevanton opened this issue Dec 27, 2019 · 7 comments
Labels
question
Milestone
1.9.2

Comments

@patsevanton
Copy link

unhealthy selenoid-ui if use not standart port (8080)

run:

docker pull aerokube/selenoid-ui && aerokube-cm selenoid-ui start --port 8090 --registry xxxx

image
@vania-pooh vania-pooh added this to the 1.9.2 milestone Dec 27, 2019
@vania-pooh vania-pooh added the bug label Dec 27, 2019
@lanwen
Copy link
Collaborator

lanwen commented Dec 31, 2019 •
edited
Loading

@patsevanton That's weird, maybe its really broken?

$ docker pull aerokube/selenoid-ui && ./cm selenoid-ui start --port 8090

$ docker ps
CONTAINER ID        IMAGE                         COMMAND             CREATED             STATUS                   PORTS                    NAMES
c61f6190a6b1        aerokube/selenoid-ui:latest   "/selenoid-ui "     3 minutes ago       Up 3 minutes (healthy)   0.0.0.0:8090->8080/tcp   selenoid-ui

what /ping handle returns?

$ curl -v 'http://localhost:8090/ping'
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8090 (#0)
> GET /ping HTTP/1.1
> Host: localhost:8090
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Tue, 31 Dec 2019 11:14:26 GMT
< Content-Length: 56
<
{"uptime":"6m0.3394946s","version":"1.9.1-10-g124665d"}

@lanwen lanwen added question and removed bug labels Dec 31, 2019
@lanwen lanwen changed the title unhealthy selenoid-ui if use not standart port (8080) unhealthy selenoid-ui if use non-standart port (8080) Dec 31, 2019
@patsevanton
Copy link
Author

patsevanton commented Jan 1, 2020 •
edited
Loading

image

curl -v 'http://localhost:8090/ping'
* About to connect() to localhost port 8090 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 8090 (#0)
> GET /ping HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:8090
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Wed, 01 Jan 2020 11:33:11 GMT
< Content-Length: 59
< 
{"uptime":"2m37.204081273s","version":"1.9.1-10-g124665d"}

docker logs 08b975a15cd6
2020/01/01 11:30:34 [INIT] [Listening on :8080]
2020/01/01 11:30:36 Client added. 1 registered clients

@lanwen
Copy link
Collaborator

lanwen commented Jan 1, 2020

@patsevanton thank you! So it's healthy now. Did you change smth?

@patsevanton
Copy link
Author

Kill all docker container and images, run puppet again for run selenoid-ui and selenoid.

Now is unhealthy

image

@lanwen
Copy link
Collaborator

lanwen commented Jan 2, 2020

@patsevanton can you share more details about the way to kill the container and images, as well as puppet script?

Also, can you share docker inspect on the ui container?
It could bring some light on the unhealthy reason

For me it looks like this even after several kills

"State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 38065,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-01-02T22:17:02.4128462Z",
            "FinishedAt": "2020-01-02T22:16:31.0007902Z",
            "Health": {
                "Status": "healthy",
                "FailingStreak": 0,
                "Log": [
                    {
                        "Start": "2020-01-02T22:17:15.5054821Z",
                        "End": "2020-01-02T22:17:15.7307333Z",
                        "ExitCode": 0,
                        "Output": "2020/01/02 22:17:15 OK\n"
                    },
                    {
                        "Start": "2020-01-02T22:17:16.7425251Z",
                        "End": "2020-01-02T22:17:16.9609314Z",
                        "ExitCode": 0,
                        "Output": "2020/01/02 22:17:16 OK\n"
                    },
                    {
                        "Start": "2020-01-02T22:17:17.9836102Z",
                        "End": "2020-01-02T22:17:18.1618982Z",
                        "ExitCode": 0,
                        "Output": "2020/01/02 22:17:18 OK\n"
                    },
                    {
                        "Start": "2020-01-02T22:17:19.1747518Z",
                        "End": "2020-01-02T22:17:19.3433762Z",
                        "ExitCode": 0,
                        "Output": "2020/01/02 22:17:19 OK\n"
                    },
                    {
                        "Start": "2020-01-02T22:17:20.3571213Z",
                        "End": "2020-01-02T22:17:20.55466Z",
                        "ExitCode": 0,
                        "Output": "2020/01/02 22:17:20 OK\n"
                    }
                ]
            }
        },

@patsevanton
Copy link
Author

patsevanton commented Jan 5, 2020 •
edited
Loading

docker ps

CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS                  PORTS                    NAMES
d767ca5f0479        aerokube/selenoid:latest   "/usr/bin/selenoid -…"   14 minutes ago      Up 14 minutes           0.0.0.0:4444->4444/tcp   selenoid
08b975a15cd6        78cf25678d14               "/selenoid-ui --sele…"   3 days ago          Up 3 days (unhealthy)   0.0.0.0:8090->8080/tcp   selenoid-ui

docker logs 08b975a15cd6

2020/01/01 11:30:34 [INIT] [Listening on :8080]
2020/01/01 11:30:36 Client added. 1 registered clients
2020/01/01 11:35:49 Removed client. 0 registered clients
2020/01/05 03:50:04 Client added. 1 registered clients
2020/01/05 03:50:04 [ERROR] [Can't get status: Get http://selenoid:4444/status: dial tcp 172.17.0.3:4444: connect: connection refused]
2020/01/05 03:50:04 [ERROR] [Can't get status: Get http://selenoid:4444/status: dial tcp 172.17.0.3:4444: connect: connection refused]

docker inspect 08b975a15cd6

[
    {
        "Id": "08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f",
        "Created": "2020-01-01T11:30:33.892306112Z",
        "Path": "/selenoid-ui",
        "Args": [
            "--selenoid-uri=http://selenoid:4444"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 14222,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-01-01T11:30:34.498123292Z",
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 316820,
                "Log": [
                    {
                        "Start": "2020-01-05T06:55:34.007223371+03:00",
                        "End": "2020-01-05T06:55:34.007368729+03:00",
                        "ExitCode": -1,
                        "Output": "Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid"
                    },
                    {
                        "Start": "2020-01-05T06:55:35.010135805+03:00",
                        "End": "2020-01-05T06:55:35.010260817+03:00",
                        "ExitCode": -1,
                        "Output": "Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid"
                    },
                    {
                        "Start": "2020-01-05T06:55:36.013333409+03:00",
                        "End": "2020-01-05T06:55:36.013478435+03:00",
                        "ExitCode": -1,
                        "Output": "Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid"
                    },
                    {
                        "Start": "2020-01-05T06:55:37.016242113+03:00",
                        "End": "2020-01-05T06:55:37.01640904+03:00",
                        "ExitCode": -1,
                        "Output": "Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid"
                    },
                    {
                        "Start": "2020-01-05T06:55:38.018997621+03:00",
                        "End": "2020-01-05T06:55:38.019118335+03:00",
                        "ExitCode": -1,
                        "Output": "Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid"
                    }
                ]
            }
        },
        "Image": "sha256:78cf25678d1470cc6fdeece81a65a060286fd0ad6d086904af265d4ea4787eb4",
        "ResolvConfPath": "/var/lib/docker/containers/08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f/hostname",
        "HostsPath": "/var/lib/docker/containers/08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f/hosts",
        "LogPath": "/var/lib/docker/containers/08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f/08b975a15cd6f8fd53d9aa9b0d03cacf93241dcf775f2408385ba5670a8cef0f-json.log",
        "Name": "/selenoid-ui",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "8080/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8090"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": [
                "/selenoid:/selenoid-ui/selenoid"
            ],
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/ad7f773caf28acca05a0e07242d668a41af9cf675bc85563263911042f0a35a7-init/diff:/var/lib/docker/overlay2/f5a986f9e839df83b368499bb023219a04076f21ab8eb79d3c24e118fd4bddb9/diff:/var/lib/docker/overlay2/c7677edb9da2e3413941965df51a46a093cbb0def30d29778b14b5e21c7b4686/diff:/var/lib/docker/overlay2/534a46368b3551bff302247c86fac7edb2795fb217a4a5af62af2a1851ecff52/diff",
                "MergedDir": "/var/lib/docker/overlay2/ad7f773caf28acca05a0e07242d668a41af9cf675bc85563263911042f0a35a7/merged",
                "UpperDir": "/var/lib/docker/overlay2/ad7f773caf28acca05a0e07242d668a41af9cf675bc85563263911042f0a35a7/diff",
                "WorkDir": "/var/lib/docker/overlay2/ad7f773caf28acca05a0e07242d668a41af9cf675bc85563263911042f0a35a7/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "localhost",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "XDG_SESSION_ID=1",
                "HOSTNAME=d01selenapp02t",
                "TERM=xterm-256color",
                "SHELL=/bin/bash",
                "HISTSIZE=2000",
                "HISTFILESIZE=2000",
                "LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:",
                "PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
                "MAIL=/var/spool/mail/root",
                "PWD=/root",
                "LANG=en_US.UTF-8",
                "HISTCONTROL=ignoredups",
                "SHLVL=2",
                "APT_LISTCHANGES_FRONTEND=none",
                "DEBIAN_FRONTEND=noninteractive",
                "LESSOPEN=||/usr/bin/lesspipe.sh %s",
                "APT_LISTBUGS_FRONTEND=none",
                "PROMPT_COMMAND=history -a",
                "HISTTIMEFORMAT=%Y/%m/%d-%H:%M:%S ",
                "_=/usr/local/bin/aerokube-cm",
                "DOCKER_API_VERSION=1.39",
                "TZ=Local"
            ],
            "Cmd": [
                "--selenoid-uri=http://selenoid:4444"
            ],
            "Healthcheck": {
                "Test": [
                    "CMD",
                    "/health-check"
                ],
                "Interval": 1000000000,
                "Timeout": 1000000000,
                "StartPeriod": 2000000000,
                "Retries": 3
            },
            "Image": "aerokube/selenoid-ui:latest",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/selenoid-ui"
            ],
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "e8e94d68c89fce8280b13384ee562bf006755048566c26fa10d2dc922ace0aab",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8080/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8090"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/e8e94d68c89f",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "df28fa5b7ef6e2d3ceb37518b8e7ff4448fc5391cca016740d69d8f593fba110",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.4",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:04",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "be16dd71d1ae2b70c0a1edadb9404e206484ecc3ce9b93cc2ed2b72ba9f1b4cb",
                    "EndpointID": "df28fa5b7ef6e2d3ceb37518b8e7ff4448fc5391cca016740d69d8f593fba110",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.4",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:04",
                    "DriverOpts": null
                }
            }
        }
    }
]

@lanwen lanwen mentioned this issue Jan 5, 2020
Closed
@lanwen
Copy link
Collaborator

lanwen commented Jan 5, 2020

@patsevanton so as you can see it's complaining on Cannot link to a non running container: /selenoid AS /selenoid-ui/selenoid since you recreated the selenoid container. You can't do that with links mechanics of docker - you have to recreate selenoid-ui as well then with a proper link. That's a nice finding tho, and actually, we should switch to networks instead of links. Will open proper issue in cm:

aerokube/cm#257

@lanwen lanwen closed this as completed Jan 5, 2020
@lanwen lanwen changed the title unhealthy selenoid-ui if use non-standart port (8080) unhealthy selenoid-ui if recreate selenoid container separately Jan 5, 2020
@lanwen lanwen mentioned this issue Feb 8, 2020
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
1.9.2
Development

No branches or pull requests
3 participants
@vania-pooh @lanwen @patsevanton