Update DockerfilePytorch and fix CICD files

[zigzagcai]

1. add spark lib in pytorch env
2. add recsys support (lightgbm, xgboost, transformers)
3. base image change from oneapi-aikit to ubuntu
4. fix cicd scripts

[xuechendi]

@zigzagcai , I successfully built my docker using this new Dockerfile with one additional argument as below:
docker build -t e2eaiok-pytorch -f DockerfilePytorch . --build-arg http_proxy --build-arg https_proxy

I think we'd better add this info to README to guide users who also needs proxy when building docker, below is my BKM

1. config ~/.docker/config.json with proxy info

{
        ...
        "proxies": {
                "default": {
                        "httpProxy": "http://${proxy_host}:${proxy_port}",
                        "httpsProxy": "http://${proxy_host}:${proxy_port}",
                        "noProxy": "localhost,::1,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
                }
        }
}

3. When call docker build, add below cmd when using proxy

--build-arg http_proxy --build-arg https_proxy
# Example:
docker build -t e2eaiok-pytorch -f DockerfilePytorch . --build-arg http_proxy --build-arg https_proxy

[xuechendi]

I realized you only installed spark but no hadoop, which leads to we can only use local filesystem instead of HDFS as backend, personally I think it should be OK for now, but we may see this as a potential issue if we need to do distributed data processing.

[zigzagcai]

Okay, Thanks.

[zigzagcai]

I have tested the refined PyTorch Dockerfile locally, and it works fine with DLRM

[zigzagcai]

@zigzagcai , I successfully built my docker using this new Dockerfile with one additional argument as below: docker build -t e2eaiok-pytorch -f DockerfilePytorch . --build-arg http_proxy --build-arg https_proxy

I think we'd better add this info to README to guide users who also needs proxy when building docker, below is my BKM

1. config ~/.docker/config.json with proxy info

{
        ...
        "proxies": {
                "default": {
                        "httpProxy": "http://${proxy_host}:${proxy_port}",
                        "httpsProxy": "http://${proxy_host}:${proxy_port}",
                        "noProxy": "localhost,::1,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
                }
        }
}

3. When call docker build, add below cmd when using proxy

--build-arg http_proxy --build-arg https_proxy
# Example:
docker build -t e2eaiok-pytorch -f DockerfilePytorch . --build-arg http_proxy --build-arg https_proxy

Thanks. README.md has been updated.

[zigzagcai]

@Jian-Zhang @xuechendi Please help to merge this PR, so that @csdingbin can trigger DLRM CICD tests with the updated Dockerfile.
Thanks!