Centos7 docker 容器内root身份应用自启动 /usr/sbin/init 问题

Centos7 docker 容器内root身份应用自启动 & /usr/sbin/init 问题

环境：我在一个 docker 容器内手动安装了 mysql、nginx、autotestsystem（自己的服务）；
mysql 和 nginx 都做了服务脚本：mysqld.service、nginx.service，并设置了开机自启动：
systemctl enable mysqld.service
systemctl enable nginx.service

vi mysqld.service

vi nginx.service

自己的应用位置：
/opt/AutoTestSystem/ui/ #前端，前端用的nginx
/opt/AutoTestSystem/server/ #后端，后端用的python flask
/opt/AutoTestSystem/server/start.sh #后端服务启动脚本
/opt/AutoTestSystem/server/stop.sh #后端服务停止脚本
ln -s /opt/AutoTestSystem/server/start.sh /usr/bin/autotestsystem-start
ln -s /opt/AutoTestSystem/server/stop.sh /usr/bin/autotestsystem-stop

# cat /opt/AutoTestSystem/server/start.sh
#!/bin/sh

pid=$(ps aux |grep "python3 main.py" |grep -v "grep" | awk '{print $2}')
echo $pid
if [ -n "$pid" ]; then
echo "WARNNING: main.py already is running, need not run again!"
exit
fi

# start server
pushd /opt/AutoTestSystem/server 2>&1 >/dev/null
$(nohup python3 main.py >> log &)
sleep 2

pid=$(ps aux |grep "python3 main.py" |grep -v "grep" | awk '{print $2}')
if [ -z "$pid" ]; then
echo "ERROR: start failed!"
else
echo "SUCCESS: start success!"
fi
popd 2>&1 >/dev/null

# cat /opt/AutoTestSystem/server/stop.sh
#!/bin/sh

#pid=$(ps aux |grep "python3 main.py" |grep -v "grep" | awk '{print $2}')
pid=$(ps aux |grep "/opt/AutoTestSystem/server/main.py" |grep -v "grep" | awk '{print $2}')

if [ -z "$pid" ]; then
echo "INFO: main.py is not running"
exit
fi

echo "INFO: to stop main.py, pid is: $pid"
$(kill -9 $pid)
sleep 3

pid2=$(ps aux |grep "python3 main.py" |grep -v "grep" | awk '{print $2}')
if [ -n "$pid2" ]; then
echo "ERROR: stop $pid2 failed!"
else
echo "SUCCESS: stop $pid2 success!"
fi

执行docker commit保存持久化：

docker commit -a "hrf" -m "autotestsystem 1.0" 478394f89173 autotestsystem:1.0

注意：docker commit 时如果指定的 REPOSITORY:TAG 变化则会生成新的image，如果不变则不会生成新的image、只会合并，docker image查看会多出 REPOSITORY和TAG未空 <none> 的 IMAGE ID，这种是可以删除的（docker rmi IMAGE_ID）。

创建build目录 & Dockerfile & init_ats启动脚本，执行 docker build 把自启动脚本打包到容器里面去并设置自启动：

cd build

vi Dockerfile
FROM autotestsystem:1.0
WORKDIR /opt/AutoTestSystem
COPY init_ats /usr/bin/
RUN chmod +x /usr/bin/init_ats
ENTRYPOINT init_ats
LABEL user="hrf"
USER root

vi init_ats
#!/bin/bash
/usr/sbin/init
sleep 2
/usr/bin/autotestsystem-start

docker build -t autotestsystem:1.0 .
docker images
docker run -idt --name autotest2 -p 29090:8080 -p 23306:3306 -p 29000:80 --privileged --cap-add SYS_ADMIN --restart=always autotestsystem:1.0

也尝试过docker run是传参替换：
docker run -idt --name autotest2 -p 29090:8080 -p 23306:3306 -p 29000:80 --privileged --cap-add SYS_ADMIN --restart=always autotestsystem:1.0 /usr/sbin/init
或
docker run -idt --name autotest2 -p 29090:8080 -p 23306:3306 -p 29000:80 --privileged --cap-add SYS_ADMIN --restart=always autotestsystem:1.0 init_ats
或
docker run -idt --name autotest2 -p 29090:8080 -p 23306:3306 -p 29000:80 --privileged --cap-add SYS_ADMIN --restart=always autotestsystem:1.0 autotestsystem-start
都不行。

执行脚本，还尝试了多种，都不行：
ENTRYPOINT init_ats
ENTRYPOINT ["init_ats"]
ENTRYPOINT ["sh","init_ats"]
ENTRYPOINT ["sh","-c","init_ats"]
ENTRYPOINT autotestsystem-start
CMD init_ats
CMD autotestsystem-start

网上找了很多资料，原因是 root 身份运行docker容器，需要特权，需要加 /usr/sbin/init 运行容器。可是编写 Dockerfile 执行 docker build 的方式，无法支持启动时执行多个脚本或命令，只能执行一个或一条命令，即使写在一个shell脚本中也不行。

最后，换了一个思路，既然mysqld和nginx做成了service服务自启动可以（systemctl enable xxx），那我也尝创建自己应用的.service脚本，做成系统服务自启动。

查看当前容器：
[root@localhost build]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
478394f89173 centos:7 "/usr/sbin/init" 33 hours ago Up 33 hours 0.0.0.0:9000->80/tcp, :::9000->80/tcp, 0.0.0.0:13306->3306/tcp, :::13306->3306/tcp, 0.0.0.0:9090->8080/tcp, :::9090->8080/tcp autotest

查看镜像：
[root@localhost build]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
autotestsystem 1.0 a781ff5a15b8 46 minutes ago 1.72GB
centos 7 eeb6ee3f44bd 2 years ago 204MB

容器 478394f89173 是以root运行的，里面的mysqld服务和nginx服务自启正常，但自己服务编写的自启动脚本 init_ats 自启不生效。

接下来，我尝试把我的 autotestsystem 做成系统服务：

进入docker容器：
docker exec -it 478394f89173 /bin/bash

创建.service文件：
[root@478394f89173 system]# vi /lib/systemd/system/autotestsystem.service

[Unit]
Description=autotestsystem
After=network.target

[Service]
Type=forking
ExecStart=/opt/AutoTestSystem/server/start.sh
ExecReload=/opt/AutoTestSystem/server/start.sh
ExecStop=/opt/AutoTestSystem/server/stop.sh
PrivateTmp=true
User=root
#Group=root
WorkingDirectory=/opt/AutoTestSystem/server/
RestartSec=2s

[Install]
WantedBy=multi-user.target

[root@478394f89173 system]# chmod 754 /lib/systemd/system/autotestsystem.service

[root@478394f89173 system]# systemctl daemon-reload # 每次修改service后需要刷新生效

[root@478394f89173 system]# systemctl start autotestsystem.service
[root@478394f89173 system]# systemctl enable autotestsystem.service
Created symlink from /etc/systemd/system/multi-user.target.wants/autotestsystem.service to /usr/lib/systemd/system/autotestsystem.service.
[root@478394f89173 system]# ps aux |grep main.py #启动服务成功

退出容器、停止容器、启动容器，进入容器，再查看 autotestsystem 能正常自启动：
exit
docker stop 478394f89173
docker start 478394f89173
docker exec -it 478394f89173 /bin/bash
ps aux |grep main.py

退出容器，在宿主机下重新 docker build 构建/保存镜像（image），即持久化：
做成.service系统服务了，用不到docker启动脚本，可以直接docker commit持久化、并删除none镜像：
docker commit -a "hrf" -m "autotestsystem 1.0" 478394f89173 autotestsystem:1.0
docker rmi `docker images | grep '<none>' | awk '{print $3}'`

当然如果要使用docker build方式持久化镜像也是可以的。

如果是在宿主机上build目录下直接vi创建和编写的autotestsystem.service服务器脚本（而不是在容器内创建服务脚本和设置自启动），则可以使用docker build方式构建镜像、打脚本包到容器里面去：

cd build

vi autotestsystem.service

vi Dockerfile

FROM autotestsystem:1.0
WORKDIR /opt/AutoTestSystem
#COPY init_ats /usr/bin/
#RUN chmod +x /usr/bin/init_ats
#ENTRYPOINT ["init_ats"]
COPY autotestsystem.service /lib/systemd/system/
COPY autotestsystem.service /etc/systemd/system/multi-user.target.wants/
RUN chmod 754 /lib/systemd/system/autotestsystem.service
RUN chmod 754 /etc/systemd/system/multi-user.target.wants/autotestsystem.service
ENTRYPOINT ["/usr/sbin/init"]
LABEL user="hrf"
USER root

说明：
build目录下创建和编写下面2个文件：autotestsystem.service、Dockerfile。
docker build构建时会执行Dockerfile中的COPY命名，COPY是构件阶段执行的。RUN、CMD、ENTRYPOINT命令则是在容器启动阶段执行的。
COPY autotestsystem.service /lib/systemd/system/ 这条命令会在docker build构件阶段把build目录下的utotestsystem.service文件拷贝到容器内的/lib/systemd/system/这个目录下。

重新构建：
docker build -t autotestsystem:1.0 .

[root@localhost build]# docker build -t autotestsystem:1.0 .
[+] Building 22.0s (11/11) FINISHED docker:default
=> [internal] load .dockerignore 0.7s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 1.8s
=> => transferring dockerfile: 550B 0.0s
=> [internal] load metadata for docker.io/library/autotestsystem:1.0 0.0s
=> [1/6] FROM docker.io/library/autotestsystem:1.0 6.6s
=> [internal] load build context 0.8s
=> => transferring context: 461B 0.0s
=> [2/6] WORKDIR /opt/AutoTestSystem 0.7s
=> [3/6] COPY autotestsystem.service /lib/systemd/system/ 1.4s
=> [4/6] COPY autotestsystem.service /etc/systemd/system/multi-user.target.wants/ 1.3s
=> [5/6] RUN chmod 754 /lib/systemd/system/autotestsystem.service 2.3s
=> [6/6] RUN chmod 754 /etc/systemd/system/multi-user.target.wants/autotestsystem.service 2.3s
=> exporting to image 2.9s
=> => exporting layers 2.9s
=> => writing image sha256:a1750b9abb21ba65b122edfdf21c37b81896ec68ffbe2f510bfff7ce97e4a95b 0.0s
=> => naming to docker.io/library/autotestsystem:1.0 0.1s
[root@localhost build]#
[root@localhost build]#
[root@localhost build]#
[root@localhost build]# docker run -idt --name autotest2 -p 29090:8080 -p 23306:3306 -p 29000:80 --privileged --cap-add SYS_ADMIN --restart=always autotestsystem:1.0
06ce37919cf068e20fdd676fa51ce1988e3a1d40ba7424d645fd5f561b758a27

[root@localhost build]#
[root@localhost build]#
[root@localhost build]#
[root@localhost build]#
[root@localhost build]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
06ce37919cf0 autotestsystem:1.0 "/usr/sbin/init" 7 seconds ago Up 4 seconds 0.0.0.0:29000->80/tcp, :::29000->80/tcp, 0.0.0.0:23306->3306/tcp, :::23306->3306/tcp, 0.0.0.0:29090->8080/tcp, :::29090->8080/tcp autotest2
478394f89173 centos:7 "/usr/sbin/init" 33 hours ago Up 33 hours 0.0.0.0:9000->80/tcp, :::9000->80/tcp, 0.0.0.0:13306->3306/tcp, :::13306->3306/tcp, 0.0.0.0:9090->8080/tcp, :::9090->8080/tcp autotest
[root@localhost build]#

最后导出、备份镜像文件：

cd release
docker save -o autotestsystem-1.0.tar autotestsystem:1.0