Some pitfalls using docker to host Nginx and php-fpm

I'm deploying my WordPress on AWS with Lambda function, all the services are coerced in one container. (I was intended to separate into different services, e.g. MySQL, Nginx, lambda-php-server, but not necessary) And I would get into 504 for Nginx.

My frontend Nginx do a proxy_pass to the backend Lambda function which always shows timeout. I was to think the performance of the chassis is bad, or some cgroup stuff is cast on the service. The log from behind Nginx shows there's setrlimit(RLIMIT_NOFILE, 51200) failed (1: Operation not permitted) I realize I have to give privilege to the Docker runtime. It will setrlimit to the host machine.

docker run -c 1024 --blkio-weight 600  -d -e https_proxy= -e http_proxy= -p 8022:22  -p 8443:443 -p 888:888 -p 3306:3306 -p 8888:8888 yangyw12345/wordpress_mainpage sh -c "while true;do echo hello world; sleep 100000;done"

CentOS 8 startup repo issue

You may jump into issues like

[[email protected] ~]# dnf update
CentOS-8 - AppStream                                                                    128  B/s |  38  B     00:00
Failed to download metadata for repo 'AppStream'
Error: Failed to download metadata for repo 'AppStream'

you need to

Last login: Mon Feb 28 02:01:55 2022 from
(reverse-i-search)`': ^C
[[email protected] ~]# nan o^C
[[email protected] ~]# curl -O
curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
[[email protected] ~]# curl -O
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22216  100 22216    0     0  15279      0  0:00:01  0:00:01 --:--:-- 15268curl
[[email protected] ~]# curl -O
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 19892  100 19892    0     0  16521      0  0:00:01  0:00:01 --:--:-- 16521lur
[[email protected] ~]# curl -O
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12552  100 12552    0     0  10330      0  0:00:01  0:00:01 --:--:-- 10330
[[email protected] ~]# rpm -i ./
anaconda-ks.cfg                             centos-stream-release-8.5-2.el8.noarch.rpm
.bash_history                               centos-stream-repos-8-3.el8.noarch.rpm
.bash_logout                                .cshrc
.bash_profile                               original-ks.cfg
.bashrc                                     .ssh/
centos-gpg-keys-8-3.el8.noarch.rpm          .tcshrc
[[email protected] ~]# rpm -i ./
anaconda-ks.cfg                             centos-stream-release-8.5-2.el8.noarch.rpm
.bash_history                               centos-stream-repos-8-3.el8.noarch.rpm
.bash_logout                                .cshrc
.bash_profile                               original-ks.cfg
.bashrc                                     .ssh/
centos-gpg-keys-8-3.el8.noarch.rpm          .tcshrc
[[email protected] ~]# rpm -i ./centos-gpg-keys-8-3.el8.noarch.rpm
warning: ./centos-gpg-keys-8-3.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
[[email protected] ~]# rpm -i ./centos-stream-repos-8-3.el8.noarch.rpm
warning: ./centos-stream-repos-8-3.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
error: Failed dependencies:
	centos-repos(8) conflicts with centos-stream-repos-8-3.el8.noarch
[[email protected] ~]# nf re^C
[[email protected] ~]# dnf remove centos-repos
Dependencies resolved.
 Package                         Architecture         Version                             Repository               Size
 centos-repos                    x86_64               8.1-1.1911.0.8.el8                  @anaconda               8.7 k
Removing dependent packages:
 initscripts                     x86_64               10.00.4-1.el8                       @anaconda               1.0 M
 nfs-utils                       x86_64               1:2.3.3-26.el8                      @anaconda               1.8 M
 setup                           noarch               2.12.2-2.el8                        @anaconda               707 k
 shadow-utils                    x86_64               2:4.6-8.el8                         @anaconda               5.1 M
Removing unused dependencies:
 centos-gpg-keys                 noarch               8.1-1.1911.0.8.el8                  @anaconda               3.3 k
 centos-release                  x86_64               8.1-1.1911.0.8.el8                  @anaconda                25 k
 gssproxy                        x86_64               0.8.0-14.el8                        @anaconda               393 k
 keyutils                        x86_64               1.5.10-6.el8                        @anaconda               115 k
 libverto-libevent               x86_64               0.3.0-5.el8                         @anaconda                12 k
 quota                           x86_64               1:4.04-10.el8                       @anaconda               949 k
 quota-nls                       noarch               1:4.04-10.el8                       @anaconda               277 k
 rpcbind                         x86_64               1.2.5-4.el8                         @anaconda               137 k

Transaction Summary
Remove  13 Packages

Freed space: 11 M
Is this ok [y/N]: y
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                1/1
  Running scriptlet: nfs-utils-1:2.3.3-26.el8.x86_64                                                                1/1
  Running scriptlet: nfs-utils-1:2.3.3-26.el8.x86_64                                                               1/13
  Erasing          : nfs-utils-1:2.3.3-26.el8.x86_64                                                               1/13
warning: file /var/lib/nfs/v4recovery: remove failed: No such file or directory
warning: file /var/lib/nfs/statd/sm.bak: remove failed: No such file or directory
warning: file /var/lib/nfs/statd/sm: remove failed: No such file or directory
warning: file /var/lib/nfs/statd: remove failed: No such file or directory
warning: directory /var/lib/nfs/rpc_pipefs: remove failed: Device or resource busy

  Running scriptlet: nfs-utils-1:2.3.3-26.el8.x86_64                                                               1/13
  Running scriptlet: rpcbind-1.2.5-4.el8.x86_64                                                                    2/13
  Erasing          : rpcbind-1.2.5-4.el8.x86_64                                                                    2/13
  Running scriptlet: rpcbind-1.2.5-4.el8.x86_64                                                                    2/13
  Running scriptlet: initscripts-10.00.4-1.el8.x86_64                                                              3/13
  Erasing          : initscripts-10.00.4-1.el8.x86_64                                                              3/13
  Running scriptlet: initscripts-10.00.4-1.el8.x86_64                                                              3/13
  Erasing          : shadow-utils-2:4.6-8.el8.x86_64                                                               4/13
  Erasing          : setup-2.12.2-2.el8.noarch                                                                     5/13
warning: /etc/shadow saved as /etc/shadow.rpmsave
warning: /etc/passwd saved as /etc/passwd.rpmsave
warning: /etc/gshadow saved as /etc/gshadow.rpmsave
warning: /etc/group saved as /etc/group.rpmsave

  Erasing          : centos-release-8.1-1.1911.0.8.el8.x86_64                                                      6/13
  Running scriptlet: centos-release-8.1-1.1911.0.8.el8.x86_64                                                      6/13
  Erasing          : centos-repos-8.1-1.1911.0.8.el8.x86_64                                                        7/13
  Running scriptlet: gssproxy-0.8.0-14.el8.x86_64                                                                  8/13
  Erasing          : gssproxy-0.8.0-14.el8.x86_64                                                                  8/13
  Running scriptlet: gssproxy-0.8.0-14.el8.x86_64                                                                  8/13
  Erasing          : quota-1:4.04-10.el8.x86_64                                                                    9/13
  Erasing          : quota-nls-1:4.04-10.el8.noarch                                                               10/13
  Erasing          : centos-gpg-keys-8.1-1.1911.0.8.el8.noarch                                                    11/13
  Erasing          : libverto-libevent-0.3.0-5.el8.x86_64                                                         12/13
  Erasing          : keyutils-1.5.10-6.el8.x86_64                                                                 13/13
  Running scriptlet: keyutils-1.5.10-6.el8.x86_64                                                                 13/13
  Verifying        : centos-gpg-keys-8.1-1.1911.0.8.el8.noarch                                                     1/13
  Verifying        : centos-release-8.1-1.1911.0.8.el8.x86_64                                                      2/13
  Verifying        : centos-repos-8.1-1.1911.0.8.el8.x86_64                                                        3/13
  Verifying        : gssproxy-0.8.0-14.el8.x86_64                                                                  4/13
  Verifying        : initscripts-10.00.4-1.el8.x86_64                                                              5/13
  Verifying        : keyutils-1.5.10-6.el8.x86_64                                                                  6/13
  Verifying        : libverto-libevent-0.3.0-5.el8.x86_64                                                          7/13
  Verifying        : nfs-utils-1:2.3.3-26.el8.x86_64                                                               8/13
  Verifying        : quota-1:4.04-10.el8.x86_64                                                                    9/13
  Verifying        : quota-nls-1:4.04-10.el8.noarch                                                               10/13
  Verifying        : rpcbind-1.2.5-4.el8.x86_64                                                                   11/13
  Verifying        : setup-2.12.2-2.el8.noarch                                                                    12/13
  Verifying        : shadow-utils-2:4.6-8.el8.x86_64                                                              13/13

  centos-repos-8.1-1.1911.0.8.el8.x86_64    initscripts-10.00.4-1.el8.x86_64  nfs-utils-1:2.3.3-26.el8.x86_64
  setup-2.12.2-2.el8.noarch                 shadow-utils-2:4.6-8.el8.x86_64   centos-gpg-keys-8.1-1.1911.0.8.el8.noarch
  centos-release-8.1-1.1911.0.8.el8.x86_64  gssproxy-0.8.0-14.el8.x86_64      keyutils-1.5.10-6.el8.x86_64
  libverto-libevent-0.3.0-5.el8.x86_64      quota-1:4.04-10.el8.x86_64        quota-nls-1:4.04-10.el8.noarch

[[email protected] ~]# dnf remove centos-repos^C
[[email protected] ~]# rpm -i ./
anaconda-ks.cfg                             centos-stream-release-8.5-2.el8.noarch.rpm
.bash_history                               centos-stream-repos-8-3.el8.noarch.rpm
.bash_logout                                .cshrc
.bash_profile                               original-ks.cfg
.bashrc                                     .ssh/
centos-gpg-keys-8-3.el8.noarch.rpm          .tcshrc
[[email protected] ~]# rpm -i ./centos-gpg-keys-8-3.el8.noarch.rpm
warning: ./centos-gpg-keys-8-3.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
	package centos-gpg-keys-1:8-3.el8.noarch is already installed
[[email protected] ~]# rpm -i ./centos-stream-release-8.5-2.el8.noarch.rpm
warning: ./centos-stream-release-8.5-2.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
[[email protected] ~]# rpm -i ./centos-stream-repos-8-3.el8.noarch.rpm
warning: ./centos-stream-repos-8-3.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 8483c65d: NOKEY
[[email protected] ~]# yum update
CentOS Stream 8 - AppStream                                                             6.0 MB/s |  20 MB     00:03
CentOS Stream 8 - BaseOS                                                                6.1 MB/s |  19 MB     00:03
CentOS Stream 8 - Extras                                                                 15 kB/s |  18 kB     00:01
Dependencies resolved.
 Package                              Architecture    Version                                  Repository          Size
 kernel                               x86_64          4.18.0-365.el8                           baseos             7.7 M
 kernel-core                          x86_64          4.18.0-365.el8                           baseos              39 M
 kernel-modules                       x86_64          4.18.0-365.el8                           baseos              31 M
 authselect-compat                    x86_64          1.2.2-3.el8                              appstream           38 k
 libfastjson                          x86_64          0.99.9-1.el8                             appstream           38 k
 libmaxminddb                         x86_64          1.2.0-10.el8                             appstream           33 k
 libxkbcommon                         x86_64          0.9.1-1.el8                              appstream          116 k
 oddjob                               x86_64          0.34.7-1.el8                             appstream           80 k
 oddjob-mkhomedir                     x86_64          0.34.7-1.el8                             appstream           49 k
 plymouth                             x86_64          0.9.4-10.20200615git1e36e30.el8          appstream          127 k
 plymouth-core-libs                   x86_64          0.9.4-10.20200615git1e36e30.el8          appstream          122 k
 plymouth-scripts                     x86_64          0.9.4-10.20200615git1e36e30.el8          appstream           44 k
 python3-newt                         x86_64          0.52.20-11.el8                           appstream           64 k
 python3-pyOpenSSL                    noarch          19.0.0-1.el8                             appstream          103 k
 python3-unbound                      x86_64          1.7.3-17.el8                             appstream          119 k
 rsyslog                              x86_64          8.2102.0-7.el8                           appstream          752 k
 unbound-libs                         x86_64          1.7.3-17.el8                             appstream          503 k
 xkeyboard-config                     noarch          2.28-1.el8                               appstream          782 k
 NetworkManager                       x86_64          1:1.36.0-0.9.el8                         baseos             2.3 M
 NetworkManager-libnm                 x86_64          1:1.36.0-0.9.el8                         baseos             1.8 M
 NetworkManager-team                  x86_64          1:1.36.0-0.9.el8                         baseos             152 k
 NetworkManager-tui                   x86_64          1:1.36.0-0.9.el8                         baseos             345 k
 audit                                x86_64          3.0.7-1.el8                              baseos             263 k
 audit-libs                           x86_64          3.0.7-1.el8                              baseos             122 k
 authselect                           x86_64          1.2.2-3.el8                              baseos             133 k
 authselect-libs                      x86_64          1.2.2-3.el8                              baseos             222 k
 bash                                 x86_64          4.4.20-3.el8                             baseos             1.5 M
 bind-export-libs                     x86_64          32:9.11.36-2.el8                         baseos             1.1 M
 brotli                               x86_64          1.0.6-3.el8                              baseos             323 k
 c-ares                               x86_64          1.13.0-6.el8                             baseos              93 k
 ca-certificates                      noarch          2021.2.50-82.el8                         baseos             390 k
 centos-gpg-keys                      noarch          1:8-4.el8                                baseos              12 k
 centos-stream-release                noarch          8.6-1.el8                                baseos              22 k
 centos-stream-repos                  noarch          8-4.el8                                  baseos              20 k
 chkconfig                            x86_64          1.19.1-1.el8                             baseos             198 k
 chrony                               x86_64          4.1-1.el8                                baseos             327 k
 coreutils                            x86_64          8.30-12.el8                              baseos             1.2 M
 coreutils-common                     x86_64          8.30-12.el8                              baseos             2.0 M
 cpio                                 x86_64          2.12-11.el8                              baseos             266 k
 cronie                               x86_64          1.5.2-6.el8                              baseos             118 k
 cronie-anacron                       x86_64          1.5.2-6.el8                              baseos              42 k
 crontabs                             noarch          1.11-17.20190603git.el8                  baseos              25 k
 crypto-policies                      noarch          20211116-1.gitae470d6.el8                baseos              64 k
 cryptsetup-libs                      x86_64          2.3.7-1.el8                              baseos             487 k
 curl                                 x86_64          7.61.1-22.el8                            baseos             351 k
 cyrus-sasl-lib                       x86_64          2.1.27-5.el8                             baseos             123 k
 dbus                                 x86_64          1:1.12.8-18.el8                          baseos              41 k
 dbus-common                          noarch          1:1.12.8-18.el8                          baseos              46 k
 dbus-daemon                          x86_64          1:1.12.8-18.el8                          baseos             240 k
 dbus-libs                            x86_64          1:1.12.8-18.el8                          baseos             184 k
 dbus-tools                           x86_64          1:1.12.8-18.el8                          baseos              85 k
 device-mapper                        x86_64          8:1.02.181-3.el8                         baseos             377 k
 device-mapper-libs                   x86_64          8:1.02.181-3.el8                         baseos             410 k
 dhcp-client                          x86_64          12:4.3.6-47.el8.0.1                      baseos             318 k
 dhcp-common                          noarch          12:4.3.6-47.el8.0.1                      baseos             207 k
 dhcp-libs                            x86_64          12:4.3.6-47.el8.0.1                      baseos             148 k
 diffutils                            x86_64          3.6-6.el8                                baseos             358 k
 dmidecode                            x86_64          1:3.3-1.el8                              baseos              92 k
 dnf                                  noarch          4.7.0-7.el8                              baseos             544 k
 dnf-data                             noarch          4.7.0-7.el8                              baseos             155 k
 dnf-plugins-core                     noarch          4.0.21-10.el8                            baseos              71 k
 dracut                               x86_64          049-201.git20220131.el8                  baseos             376 k
 dracut-config-rescue                 x86_64          049-201.git20220131.el8                  baseos              61 k
 dracut-network                       x86_64          049-201.git20220131.el8                  baseos             109 k
 dracut-squash                        x86_64          049-201.git20220131.el8                  baseos              62 k
 e2fsprogs                            x86_64          1.45.6-3.el8                             baseos             1.0 M
 e2fsprogs-libs                       x86_64          1.45.6-3.el8                             baseos             233 k
 elfutils-default-yama-scope          noarch          0.186-1.el8                              baseos              50 k
 elfutils-libelf                      x86_64          0.186-1.el8                              baseos             229 k
 elfutils-libs                        x86_64          0.186-1.el8                              baseos             295 k
 ethtool                              x86_64          2:5.13-1.el8                             baseos             219 k
 expat                                x86_64          2.2.5-5.el8                              baseos             112 k
 file                                 x86_64          5.33-20.el8                              baseos              77 k
 file-libs                            x86_64          5.33-20.el8                              baseos             543 k
 filesystem                           x86_64          3.8-6.el8                                baseos             1.1 M
 firewalld                            noarch          0.9.3-11.el8                             baseos             503 k
 firewalld-filesystem                 noarch          0.9.3-11.el8                             baseos              78 k
 freetype                             x86_64          2.9.1-4.el8_3.1                          baseos             394 k
 fuse-libs                            x86_64          2.9.7-14.el8                             baseos             102 k
 gawk                                 x86_64          4.2.1-2.el8                              baseos             1.1 M
 glib2                                x86_64          2.56.4-158.el8                           baseos             2.5 M
 glibc                                x86_64          2.28-189.el8                             baseos             2.2 M
 glibc-common                         x86_64          2.28-189.el8                             baseos             1.3 M
 glibc-langpack-en                    x86_64          2.28-189.el8                             baseos             834 k
 gnupg2                               x86_64          2.2.20-2.el8                             baseos             2.4 M
 gnupg2-smime                         x86_64          2.2.20-2.el8                             baseos             283 k
 gnutls                               x86_64          3.6.16-4.el8                             baseos             1.0 M
 gpgme                                x86_64          1.13.1-11.el8                            baseos             336 k
 grub2-common                         noarch          1:2.02-106.el8                           baseos             891 k
 grub2-pc                             x86_64          1:2.02-106.el8                           baseos              42 k
 grub2-pc-modules                     noarch          1:2.02-106.el8                           baseos             916 k
 grub2-tools                          x86_64          1:2.02-106.el8                           baseos             2.0 M
 grub2-tools-extra                    x86_64          1:2.02-106.el8                           baseos             1.1 M
 grub2-tools-minimal                  x86_64          1:2.02-106.el8                           baseos             210 k
 grubby                               x86_64          8.40-42.el8                              baseos              49 k
 gzip                                 x86_64          1.9-12.el8                               baseos             167 k
 hdparm                               x86_64          9.54-4.el8                               baseos             100 k
 hostname                             x86_64          3.20-7.el8.0.1                           baseos              32 k
 hwdata                               noarch          0.314-8.12.el8                           baseos             1.7 M
 ima-evm-utils                        x86_64          1.3.2-12.el8                             baseos              64 k
 info                                 x86_64          6.5-7.el8_5                              baseos             198 k
 ipcalc                               x86_64          0.2.4-4.el8                              baseos              38 k
 iproute                              x86_64          5.15.0-3.el8                             baseos             796 k
 iprutils                             x86_64          2.4.19-1.el8                             baseos             255 k
 iptables                             x86_64          1.8.4-22.el8                             baseos             584 k
 iptables-ebtables                    x86_64          1.8.4-22.el8                             baseos              72 k
 iptables-libs                        x86_64          1.8.4-22.el8                             baseos             108 k
 iputils                              x86_64          20180629-9.el8                           baseos             148 k
 irqbalance                           x86_64          2:1.4.0-6.el8                            baseos              56 k
 iwl100-firmware                      noarch                      baseos             174 k
 iwl1000-firmware                     noarch          1:                    baseos             237 k
 iwl105-firmware                      noarch                     baseos             258 k
 iwl135-firmware                      noarch                     baseos             267 k
 iwl2000-firmware                     noarch                     baseos             261 k
 iwl2030-firmware                     noarch                     baseos             269 k
 iwl3160-firmware                     noarch          1:                   baseos             1.7 M
 iwl3945-firmware                     noarch                      baseos             112 k
 iwl4965-firmware                     noarch                    baseos             125 k
 iwl5000-firmware                     noarch                     baseos             318 k
 iwl5150-firmware                     noarch                       baseos             170 k
 iwl6000-firmware                     noarch                      baseos             191 k
 iwl6000g2a-firmware                  noarch                     baseos             334 k
 iwl6050-firmware                     noarch                      baseos             267 k
 iwl7260-firmware                     noarch          1:                   baseos              23 M
 jansson                              x86_64          2.14-1.el8                               baseos              47 k
 json-c                               x86_64          0.13.1-3.el8                             baseos              41 k
 kbd                                  x86_64          2.0.4-10.el8                             baseos             390 k
 kbd-legacy                           noarch          2.0.4-10.el8                             baseos             481 k
 kbd-misc                             noarch          2.0.4-10.el8                             baseos             1.5 M
 kernel-tools                         x86_64          4.18.0-365.el8                           baseos             7.9 M
 kernel-tools-libs                    x86_64          4.18.0-365.el8                           baseos             7.7 M
 kexec-tools                          x86_64          2.0.20-68.el8                            baseos             523 k
 keyutils-libs                        x86_64          1.5.10-9.el8                             baseos              34 k
 kmod                                 x86_64          25-19.el8                                baseos             126 k
 kmod-libs                            x86_64          25-19.el8                                baseos              68 k
 kpartx                               x86_64          0.8.4-22.el8                             baseos             114 k
 krb5-libs                            x86_64          1.18.2-14.el8                            baseos             840 k
 libarchive                           x86_64          3.3.3-3.el8_5                            baseos             360 k
 libblkid                             x86_64          2.32.1-32.el8                            baseos             218 k
 libcap                               x86_64          2.48-2.el8                               baseos              74 k
 libcap-ng                            x86_64          0.7.11-1.el8                             baseos              33 k
 libcom_err                           x86_64          1.45.6-3.el8                             baseos              49 k
 libcomps                             x86_64          0.1.18-1.el8                             baseos              82 k
 libcroco                             x86_64          0.6.12-4.el8_2.1                         baseos             113 k
 libcurl                              x86_64          7.61.1-22.el8                            baseos             301 k
 libdb                                x86_64          5.3.28-42.el8_4                          baseos             751 k
 libdb-utils                          x86_64          5.3.28-42.el8_4                          baseos             150 k
 libdnf                               x86_64          0.63.0-7.el8                             baseos             701 k
 libfdisk                             x86_64          2.32.1-32.el8                            baseos             252 k
 libffi                               x86_64          3.1-23.el8                               baseos              37 k
 libgcc                               x86_64          8.5.0-10.el8                             baseos              80 k
 libgcrypt                            x86_64          1.8.5-6.el8                              baseos             463 k
 libgomp                              x86_64          8.5.0-10.el8                             baseos             207 k
 libkcapi                             x86_64          1.2.0-2.el8                              baseos              48 k
 libkcapi-hmaccalc                    x86_64          1.2.0-2.el8                              baseos              31 k
 libldb                               x86_64          2.4.1-1.el8                              baseos             188 k
 libmodulemd1                         x86_64          1.8.16-                        baseos             176 k
 libmount                             x86_64          2.32.1-32.el8                            baseos             235 k
 libndp                               x86_64          1.7-6.el8                                baseos              40 k
 libnfsidmap                          x86_64          1:2.3.3-50.el8                           baseos             121 k
 libnftnl                             x86_64          1.1.5-5.el8                              baseos              83 k
 libnghttp2                           x86_64          1.33.0-3.el8_2.1                         baseos              77 k
 libnl3                               x86_64          3.5.0-1.el8                              baseos             320 k
 libnl3-cli                           x86_64          3.5.0-1.el8                              baseos             193 k
 libpcap                              x86_64          14:1.9.1-5.el8                           baseos             169 k
 libpsl                               x86_64          0.20.2-6.el8                             baseos              61 k
 libpwquality                         x86_64          1.4.4-3.el8                              baseos             107 k
 librepo                              x86_64          1.14.2-1.el8                             baseos              93 k
 libreport-filesystem                 x86_64          2.9.5-15.el8                             baseos              21 k
 libseccomp                           x86_64          2.5.2-1.el8                              baseos              71 k
 libselinux                           x86_64          2.9-5.el8                                baseos             165 k
 libselinux-utils                     x86_64          2.9-5.el8                                baseos             243 k
 libsemanage                          x86_64          2.9-6.el8                                baseos             165 k
 libsepol                             x86_64          2.9-3.el8                                baseos             340 k
 libsmartcols                         x86_64          2.32.1-32.el8                            baseos             178 k
 libsolv                              x86_64          0.7.20-1.el8                             baseos             375 k
 libss                                x86_64          1.45.6-3.el8                             baseos              54 k
 libssh                               x86_64          0.9.6-3.el8                              baseos             216 k
 libssh-config                        noarch          0.9.6-3.el8                              baseos              19 k
 libsss_autofs                        x86_64          2.6.1-2.el8                              baseos             119 k
 libsss_certmap                       x86_64          2.6.1-2.el8                              baseos             162 k
 libsss_idmap                         x86_64          2.6.1-2.el8                              baseos             121 k
 libsss_nss_idmap                     x86_64          2.6.1-2.el8                              baseos             128 k
 libsss_sudo                          x86_64          2.6.1-2.el8                              baseos             117 k
 libstdc++                            x86_64          8.5.0-10.el8                             baseos             453 k
 libsysfs                             x86_64          2.1.0-25.el8                             baseos              53 k
 libtalloc                            x86_64          2.3.3-1.el8                              baseos              49 k
 libtdb                               x86_64          1.4.4-1.el8                              baseos              59 k
 libteam                              x86_64          1.31-2.el8                               baseos              64 k
 libtevent                            x86_64          0.11.0-0.el8                             baseos              50 k
 libtirpc                             x86_64          1.1.4-6.el8                              baseos             113 k
 libusbx                              x86_64          1.0.23-4.el8                             baseos              74 k
 libuser                              x86_64          0.62-24.el8                              baseos             414 k
 libuuid                              x86_64          2.32.1-32.el8                            baseos              97 k
 libxcrypt                            x86_64          4.1.1-6.el8                              baseos              73 k
 libxml2                              x86_64          2.9.7-11.el8                             baseos             696 k
 linux-firmware                       noarch          20220210-106.git6342082c.el8             baseos             194 M
 logrotate                            x86_64          3.14.0-4.el8                             baseos              86 k
 lshw                                 x86_64          B.02.19.2-6.el8                          baseos             341 k
 lsscsi                               x86_64          0.32-3.el8                               baseos              71 k
 lua-libs                             x86_64          5.3.4-12.el8                             baseos             118 k
 lz4-libs                             x86_64          1.8.3-3.el8_4                            baseos              66 k
 man-db                               x86_64                           baseos             887 k
 microcode_ctl                        x86_64          4:20220207-1.el8                         baseos             5.5 M
 mozjs60                              x86_64          60.9.0-4.el8                             baseos             6.6 M
 ncurses                              x86_64          6.1-9.20180224.el8                       baseos             387 k
 ncurses-base                         noarch          6.1-9.20180224.el8                       baseos              81 k
 ncurses-libs                         x86_64          6.1-9.20180224.el8                       baseos             334 k
 net-tools                            x86_64          2.0-0.52.20160912git.el8                 baseos             322 k
 nettle                               x86_64          3.4.1-7.el8                              baseos             301 k
 newt                                 x86_64          0.52.20-11.el8                           baseos             121 k
 nftables                             x86_64          1:0.9.3-24.el8                           baseos             323 k
 numactl-libs                         x86_64          2.0.12-13.el8                            baseos              36 k
 openldap                             x86_64          2.4.46-18.el8                            baseos             352 k
 openssh                              x86_64          8.0p1-12.el8                             baseos             522 k
 openssh-clients                      x86_64          8.0p1-12.el8                             baseos             668 k
 openssh-server                       x86_64          8.0p1-12.el8                             baseos             491 k
 openssl                              x86_64          1:1.1.1k-5.el8_5                         baseos             709 k
 openssl-libs                         x86_64          1:1.1.1k-5.el8_5                         baseos             1.5 M
 openssl-pkcs11                       x86_64          0.4.10-2.el8                             baseos              66 k
 os-prober                            x86_64          1.74-9.el8                               baseos              51 k
 p11-kit                              x86_64          0.23.22-1.el8                            baseos             324 k
 p11-kit-trust                        x86_64          0.23.22-1.el8                            baseos             137 k
 pam                                  x86_64          1.3.1-16.el8                             baseos             739 k
 parted                               x86_64          3.2-39.el8                               baseos             555 k
 passwd                               x86_64          0.80-4.el8                               baseos             115 k
 pciutils-libs                        x86_64          3.7.0-1.el8                              baseos              54 k
 pcre                                 x86_64          8.42-6.el8                               baseos             211 k
 pcre2                                x86_64          10.32-2.el8                              baseos             246 k
 pigz                                 x86_64          2.4-4.el8                                baseos              79 k
 platform-python                      x86_64          3.6.8-45.el8                             baseos              85 k
 platform-python-pip                  noarch          9.0.3-22.el8                             baseos             1.6 M
 platform-python-setuptools           noarch          39.2.0-6.el8                             baseos             632 k
 policycoreutils                      x86_64          2.9-18.el8                               baseos             375 k
 polkit                               x86_64          0.115-13.el8_5.1                         baseos             154 k
 polkit-libs                          x86_64          0.115-13.el8_5.1                         baseos              76 k
 popt                                 x86_64          1.18-1.el8                               baseos              61 k
 procps-ng                            x86_64          3.3.15-6.el8                             baseos             329 k
 psmisc                               x86_64          23.1-5.el8                               baseos             151 k
 python3-cryptography                 x86_64          3.2.1-5.el8                              baseos             559 k
 python3-dnf                          noarch          4.7.0-7.el8                              baseos             545 k
 python3-dnf-plugins-core             noarch          4.0.21-10.el8                            baseos             230 k
 python3-firewall                     noarch          0.9.3-11.el8                             baseos             433 k
 python3-gobject-base                 x86_64          3.28.3-2.el8                             baseos             313 k
 python3-gpg                          x86_64          1.13.1-11.el8                            baseos             244 k
 python3-hawkey                       x86_64          0.63.0-7.el8                             baseos             116 k
 python3-libcomps                     x86_64          0.1.18-1.el8                             baseos              52 k
 python3-libdnf                       x86_64          0.63.0-7.el8                             baseos             778 k
 python3-librepo                      x86_64          1.14.2-1.el8                             baseos              53 k
 python3-libs                         x86_64          3.6.8-45.el8                             baseos             7.8 M
 python3-libselinux                   x86_64          2.9-5.el8                                baseos             283 k
 python3-libxml2                      x86_64          2.9.7-11.el8                             baseos             237 k
 python3-linux-procfs                 noarch          0.7.0-1.el8                              baseos              42 k
 python3-perf                         x86_64          4.18.0-365.el8                           baseos             7.9 M
 python3-pip-wheel                    noarch          9.0.3-22.el8                             baseos             895 k
 python3-ply                          noarch          3.9-9.el8                                baseos             111 k
 python3-rpm                          x86_64          4.14.3-22.el8                            baseos             155 k
 python3-setuptools-wheel             noarch          39.2.0-6.el8                             baseos             289 k
 python3-syspurpose                   x86_64          1.28.25-1.el8                            baseos             324 k
 rng-tools                            x86_64          6.14-4.git.b2b7934e.el8                  baseos              72 k
 rpm                                  x86_64          4.14.3-22.el8                            baseos             543 k
 rpm-build-libs                       x86_64          4.14.3-22.el8                            baseos             157 k
 rpm-libs                             x86_64          4.14.3-22.el8                            baseos             345 k
 rpm-plugin-selinux                   x86_64          4.14.3-22.el8                            baseos              77 k
 rpm-plugin-systemd-inhibit           x86_64          4.14.3-22.el8                            baseos              79 k
 sed                                  x86_64          4.5-5.el8                                baseos             298 k
 selinux-policy                       noarch          3.14.3-92.el8                            baseos             643 k
 selinux-policy-targeted              noarch          3.14.3-92.el8                            baseos              15 M
 sg3_utils                            x86_64          1.44-5.el8                               baseos             917 k
 sg3_utils-libs                       x86_64          1.44-5.el8                               baseos              99 k
 snappy                               x86_64          1.1.8-3.el8                              baseos              37 k
 sqlite-libs                          x86_64          3.26.0-15.el8                            baseos             581 k
 squashfs-tools                       x86_64          4.3-20.el8                               baseos             165 k
 sssd-client                          x86_64          2.6.1-2.el8                              baseos             224 k
 sssd-common                          x86_64          2.6.1-2.el8                              baseos             1.6 M
 sssd-kcm                             x86_64          2.6.1-2.el8                              baseos             250 k
 sssd-nfs-idmap                       x86_64          2.6.1-2.el8                              baseos             118 k
 sudo                                 x86_64          1.8.29-8.el8                             baseos             925 k
 systemd                              x86_64          239-58.el8                               baseos             3.6 M
 systemd-libs                         x86_64          239-58.el8                               baseos             1.1 M
 systemd-pam                          x86_64          239-58.el8                               baseos             483 k
 systemd-udev                         x86_64          239-58.el8                               baseos             1.6 M
 teamd                                x86_64          1.31-2.el8                               baseos             130 k
 trousers                             x86_64          0.3.15-1.el8                             baseos             152 k
 trousers-lib                         x86_64          0.3.15-1.el8                             baseos             168 k
 tuned                                noarch          2.18.0-2.el8                             baseos             316 k
 tzdata                               noarch          2021e-1.el8                              baseos             474 k
 util-linux                           x86_64          2.32.1-32.el8                            baseos             2.5 M
 vim-minimal                          x86_64          2:8.0.1763-16.el8_5.12                   baseos             575 k
 virt-what                            x86_64          1.18-13.el8                              baseos              36 k
 which                                x86_64          2.21-17.el8                              baseos              49 k
 xfsprogs                             x86_64          5.0.0-10.el8                             baseos             1.1 M
 yum                                  noarch          4.7.0-7.el8                              baseos             201 k
 zlib                                 x86_64          1.2.11-17.el8                            baseos             102 k
Installing dependencies:
 grub2-tools-efi                      x86_64          1:2.02-106.el8                           baseos             474 k
 initscripts                          x86_64          10.00.17-1.el8                           baseos             340 k
 libbpf                               x86_64          0.4.0-3.el8                              baseos             125 k
 libibverbs                           x86_64          37.2-1.el8                               baseos             384 k
 libmodulemd                          x86_64          2.13.0-1.el8                             baseos             233 k
 libzstd                              x86_64          1.4.4-1.el8                              baseos             266 k
 lmdb-libs                            x86_64          0.9.24-1.el8                             baseos              58 k
 python3-nftables                     x86_64          1:0.9.3-24.el8                           baseos              29 k
 setup                                noarch          2.12.2-6.el8                             baseos             181 k
 shadow-utils                         x86_64          2:4.6-16.el8                             baseos             1.2 M
 tpm2-tss                             x86_64          2.3.2-4.el8                              baseos             275 k
Installing weak dependencies:
 glibc-gconv-extra                    x86_64          2.28-189.el8                             appstream          1.5 M
 crypto-policies-scripts              noarch          20211116-1.gitae470d6.el8                baseos              83 k
 elfutils-debuginfod-client           x86_64          0.186-1.el8                              baseos              71 k
 memstrack                            x86_64          0.1.11-1.el8                             baseos              48 k

Transaction Summary
Install   18 Packages
Upgrade  291 Packages

Total download size: 463 M
Is this ok [y/N]: y
Downloading Packages:

Everything goes fine

centos7/8 podman kernel not support overlayfs

Sounds like I'm messed up the mount of overlay hostPath that stores the containers when updating from centos7 to 8-stream. I also noticed that 9-stream is in beta.

[[email protected] ~]# podman ps
Error: kernel does not support overlay fs: 'overlay' is not supported over extfs at "/var/lib/containers/storage/overlay": backing file system is unsupported for this graph driver

For Kubernetes, we automatically apply the pod using yaml file like

  name: vo-hostpath-pod
  - name: filebeat
    image: ikubernetes/filebeat:5.6.7-alpine
    - name: REDIS_HOST              
    - name: LOG_LEVEL               
      value: info                   
    - name: varlog            
      mountPath: /var/log   
    - name: socket                
      mountPath: /var/run/docker.sock
    - name: varlibdockercontainers 
      mountPath: /var/lib/docker/containers
      readOnly: true    
  - name: varlog  
      path: /var/log   
      type: DirectoryOrCreate 
  - name: varlibdockercontainers
      path: /var/lib/docker/containers
      type: Directory
  - name: socket
      path: /var/run/docker.sock
      type: Socket              

Debbug the command using strace

newfstatat(AT_FDCWD, "/root/bin/crun", 0xc00019e378, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/usr/bin/runc", {st_mode=S_IFREG|0755, st_size=11889776, ...}, 0) = 0
openat(AT_FDCWD, "/etc/selinux/refpolicy/contexts/lxc_contexts", O_RDONLY|O_CLOEXEC) = 9
epoll_ctl(4, EPOLL_CTL_ADD, 9, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data={u32=1719281968, u64=140043422935344}}) = 0
fcntl(9, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fstat(9, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0

The open at /etc/selinux/refpolicy/contexts/lxc_contexts is wierd so I think there's sth about the selinux, so I remvoe container-selinux and everythin works fine.

[[email protected] ~]# docker ps
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
WARN[0000] Error validating CNI config file /etc/cni/net.d/10-flannel.conflist: [failed to find plugin "flannel" in path [/usr/local/libexec/cni /usr/libexec/cni /usr/local/lib/cni /usr/lib/cni /opt/cni/bin]]

Also after upgrade to CentOS 8, don't forget to disable firealld and selinux because it'll update the settings. I debug it through self ping success and couldn't get anything from browser.

[[email protected] ~]# sudo ss -tulpn
Netid                 State                  Recv-Q                 Send-Q                                                    Local Address:Port                                  Peer Address:Port                Process
udp                   UNCONN                 0                      0                                                                                  *                    users:(("systemd-resolve",pid=1045,fd=12))
udp                   UNCONN                 0                      0                                                                              *                    users:(("systemd-resolve",pid=1045,fd=18))
udp                   UNCONN                 0                      0                                                                             *                    users:(("ntpd",pid=640,fd=21))
udp                   UNCONN                 0                      0                                                                                 *                    users:(("ntpd",pid=640,fd=18))
udp                   UNCONN                 0                      0                                                                                   *                    users:(("ntpd",pid=640,fd=16))
udp                   UNCONN                 0                      0                                                                  [::]:5355                                          [::]:*                    users:(("systemd-resolve",pid=1045,fd=14))
udp                   UNCONN                 0                      0                                      [fe80::f816:3eff:fe8b:cbe7]%eth0:123                                           [::]:*                    users:(("ntpd",pid=640,fd=22))
udp                   UNCONN                 0                      0                                                                 [::1]:123                                           [::]:*                    users:(("ntpd",pid=640,fd=19))
udp                   UNCONN                 0                      0                                                                  [::]:123                                           [::]:*                    users:(("ntpd",pid=640,fd=17))
tcp                   LISTEN                 0                      9                                                                                    *                    users:(("pure-ftpd",pid=601,fd=5))
tcp                   LISTEN                 0                      511                                                                                 *                    users:(("nginx",pid=2970,fd=20),("nginx",pid=2969,fd=20),("nginx",pid=2792,fd=20))
tcp                   LISTEN                 0                      128                                                                                *                    users:(("BT-Panel",pid=764,fd=6))
tcp                   LISTEN                 0                      100                                                                                *                    users:(("master",pid=1030,fd=16))
tcp                   LISTEN                 0                      511                                                                                 *                    users:(("nginx",pid=2970,fd=22),("nginx",pid=2969,fd=22),("nginx",pid=2792,fd=22))
tcp                   LISTEN                 0                      511                                                                              *                    users:(("redis-server",pid=1706,fd=6))
tcp                   LISTEN                 0                      128                                                                                *                    users:(("systemd-resolve",pid=1045,fd=13))
tcp                   LISTEN                 0                      1024                                                                            *                    users:(("memcached",pid=734,fd=28))
tcp                   LISTEN                 0                      128                                                                                *                    users:(("sshd",pid=1244,fd=5))
tcp                   LISTEN                 0                      511                                                                                  *                    users:(("nginx",pid=2970,fd=21),("nginx",pid=2969,fd=21),("nginx",pid=2792,fd=21))
tcp                   LISTEN                 0                      9                                                                  [::]:21                                            [::]:*                    users:(("pure-ftpd",pid=601,fd=6))
tcp                   LISTEN                 0                      100                                                               [::1]:25                                            [::]:*                    users:(("master",pid=1030,fd=17))
tcp                   LISTEN                 0                      150                                                                   *:3306                                             *:*                    users:(("mysqld",pid=2166,fd=19))
tcp                   LISTEN                 0                      128                                                                [::]:5355                                          [::]:*                    users:(("systemd-resolve",pid=1045,fd=15))
tcp                   LISTEN                 0                      1024                                                              [::1]:11211                                         [::]:*                    users:(("memcached",pid=734,fd=29))
tcp                   LISTEN                 0                      128                                                                [::]:9999                                          [::]:*                    users:(("sshd",pid=1244,fd=6))
[[email protected] ~]#

如何在Python logging.Formatter 格式化


[    test_log    ][    DEBUG]  test (
[    test_log    ][     INFO]  test (
[    test_log    ][  WARNING]  test (
[    test_log    ][    ERROR]  test (
[    test_log    ][ CRITICAL]  test (


[__main__][DEBUG]  test (
[__main__][INFO]  test (
[__main__][WARNING]  test (
[__main__][ERROR]  test (
[__main__][CRITICAL]  test (


  • funcName 而不是 name

  • 得考虑右对齐和居中


import logging


#The background is set with 40 plus the number of the color, and the foreground with 30
#These are the sequences need to get colored ouput
RESET_SEQ = "\033[0m"
COLOR_SEQ = "\033[1;%dm"
BOLD_SEQ = "\033[1m"

def formatter_message(message, use_color = True):
    if use_color:
        message = message.replace("$RESET", RESET_SEQ).replace("$BOLD", BOLD_SEQ)
        message = message.replace("$RESET", "").replace("$BOLD", "")
    return message

    'INFO': WHITE,
    'DEBUG': BLUE,
    'ERROR': RED

class ColoredFormatter(logging.Formatter):
    def __init__(self, msg, use_color = True):
        logging.Formatter.__init__(self, msg)
        self.use_color = use_color

    def format(self, record):
        levelname = record.levelname
        if self.use_color and levelname in COLORS:
            levelname_color = COLOR_SEQ % (30 + COLORS[levelname]) + levelname + RESET_SEQ
            record.levelname = levelname_color
        return logging.Formatter.format(self, record)

# Custom logger class with multiple destinations
class ColoredLogger(logging.Logger):
    FORMAT = "[$BOLD" + "%(funcName)s".center(20," ")+"$RESET]["+ "%(levelname)20s" +"]  %(message)s ($BOLD%(filename)s$RESET:%(lineno)d)"
    COLOR_FORMAT = formatter_message(FORMAT, True)
    def __init__(self, name):
        logging.Logger.__init__(self, name, logging.DEBUG)                

        color_formatter = ColoredFormatter(self.COLOR_FORMAT)

        console = logging.StreamHandler()





看了历史占用 CPU100% 这次比上次多了个储存也被占满了,用 find / -type f -size +10G 以为是/proc/kcore 可hexdump 一下却发现是纸老虎。实际不占用空间,只是个内存 的映射物,linux提供了几个args的参数。如果复用的话可能会达到128T。

这次因为ssh密码换了,而且有个后台自动修改替换,走我的华为云的自动开机重置密码无济于事。这次立功的bt-panel。 我直接把我的私钥换掉root/.ssh/authorized_keys。

ssh上去之后,crontab -e删了,删完自动又有。不过发掘是 /root/.tmp00/bash 在作恶,ps -ef | grep .tmp00 | grep -v grep | awk '{print $2}' | xargs kill -9 (注意不能删bash,会奔溃,估计脚本制作着就是这么想的)


struct kcore_list {
struct kcore_list *next;
unsigned long addr;
size_t size;
struct vmcore {
struct list_head list;
unsigned long long paddr;
unsigned long long size;
loff_t offset;
可 以看到vmcore比较复杂,事实上也正是如此,因此它的操作比较复杂,而且使用环境也是很复杂的,涉及到kexec和kdump机制,也许就是这个原因 它使用了内核中最普遍的list_head结构,但是对于kcore,它的结构十分简单,目的就是为了遍历整个内存,也不需要查找,删除等操作,因此它用 了自己的next字段来组成链表,如此一来可以节省一个指针的空间。
在系统初始化的时候,mem_init函数中将整个物理内存和vmalloc的动态内存都加入了kcore_list中,这样的话,这个链表中就最起码有 了两个元素,一个是物理内存,另一个是vmalloc动态内存。注意这里所说的物理内存就是一一映射的内存,其实也可以不是,你完全可以自己实现一个映射 方法代替这里的一一映射,linux内核默认的什么highmem,vmalloc_start等等还有一一映射抑或高端映射等等机制都只是一个更底层的 机制一些策略,这个更底层的机制就是linux内核的内存映射,因此在这个机制提出的约束上你可以实现很多种策略,区分物理一一映射和高端映射只是其中之 一罢了:
void __init mem_init(void)

kclist_add(&kcore_mem, __va(0), max_low_pfn << PAGE_SHIFT); kclist_add(&kcore_vmalloc, (void *)VMALLOC_START, VMALLOC_END-VMALLOC_START); … } void kclist_add(struct kcore_list *new, void *addr, size_t size) { new->addr = (unsigned long)addr;
new->size = size;
new->next = kclist;
kclist = new;
static size_t get_kcore_size(int *nphdr, size_t *elf_buflen)
size_t try, size;
struct kcore_list *m;
*nphdr = 1;
size = 0;
for (m=kclist; m; m=m->next) { //找到最大的地址值加上长度后就是最后的结果,依据就是linux内核空间的映射方式
try = kc_vaddr_to_offset((size_t)m->addr + m->size);
if (try > size)
size = try;
*nphdr = *nphdr + 1;
*elf_buflen = sizeof(struct elfhdr) +
*elf_buflen = PAGE_ALIGN(*elf_buflen);
return size + *elf_buflen; //总的长度就是实际内存大小长度加上额外的头部的长度
procfs 是一个文件系统,是文件系统的话就要有一个file_operations结构体来实现这个文件系统的操作,可是在procfs文件系统中,每一个文件可 以有不同的操作回调函数,也就是说,procfs首先是一个文件系统,在它是文件系统的意义的基础之上,它又是另一种机制,它提供了一个内核导出信息的口 子,就是说,procfs作为文件系统的意义仅仅在于信息的导出,它里面的文件从来都不是真实的文件,但是确实有文件的接口,比如你在ls -l命令发出给/proc/kcore文件时,它给出了文件的“大小”,实际上并不会占据那么大的空间而仅仅是一个数字,该数字是从上面的 get_kcore_size中得到的。在procfs文件系统中,每个文件都是一个proc_dir_entry,这才是它真正要表达的,套在标准文件 系统之上的那一层东西,该结构中的proc_fops就是该结构代表文件的file_operations结构体,如果这么理解的话,procfs文件系 统下的每一个文件都可以有自己的file_operations了而不必统一用整个procfs的一个file_operations,就像 ext2/ext3等传统的真实文件系统一样,从OO的角度来看,procfs继承了vfs文件系统,在文件系统的基础上实现了自己的特性(其实每一个具 体文件系统都有自己的特性,都继承并实现了vfs这个抽象类,不过本文就是说procfs的一个文件的,因此它显得比较特殊)。就好像前几篇文章中所描述 的seqfile一样,它就是专门为procfs提供一个串行化读取的接口函数机制而不是一个独立的机制,它可以被用在procfs的 file_operations中,当然也可以被用到别处。我们接下来看看read_kcore,它就是/proc/kcore这个proc文件的 proc_fops即file_operations的read回调函数:
static ssize_t read_kcore(struct file *file, char __user *buffer, size_t buflen, loff_t *fpos)
ssize_t acc = 0;
size_t size, tsz;
size_t elf_buflen;
int nphdr;
unsigned long start;
proc_root_kcore->size = size = get_kcore_size(&nphdr, ⪙f_buflen);
if (buflen == 0 || *fpos >= size) {
return 0;
if (buflen > size - *fpos)
buflen = size - *fpos;
start = kc_offset_to_vaddr(*fpos - elf_buflen); //物理地址到虚拟地址的转换,其实对于一一映射就是加上一个PAGE_OFFSET偏移量,这也是默认情况,当然也可以提供别的转换方式。
if ((tsz = (PAGE_SIZE - (start & ~PAGE_MASK))) > buflen)
tsz = buflen;
while (buflen) {
struct kcore_list *m;
for (m=kclist; m; m=m->next) { //寻找这个地址所属的kcore_list
if (start >= m->addr && start < (m->addr+m->size))
} else if ((start >= VMALLOC_START) && (start < VMALLOC_END)) { //在这种情况下,说明用户要读取的是vmalloc空间的内存映像,那么很简单,就是遍历vmalloc空间的vm_struct结构体们,然后将之上 的数据取出来。 char * elf_buf; struct vm_struct *m; unsigned long curstart = start; unsigned long cursize = tsz; elf_buf = kmalloc(tsz, GFP_KERNEL); if (!elf_buf) return -ENOMEM; memset(elf_buf, 0, tsz); read_lock(&vmlist_lock); for (m=vmlist; m && cursize; m=m->next) {
unsigned long vmstart;
unsigned long vmsize;
unsigned long msize = m->size - PAGE_SIZE;
vmstart = (curstart < (unsigned long)m->addr ?
(unsigned long)m->addr : curstart);
if (((unsigned long)m->addr + msize) > (curstart + cursize))
vmsize = curstart + cursize - vmstart;
vmsize = (unsigned long)m->addr + msize - vmstart;
memcpy(elf_buf + (vmstart - start), (char *)vmstart, vmsize);
if (copy_to_user(buffer, elf_buf, tsz)) //向用户拷贝内存数据

} else { //最后一种情况就是读取物理内存了,其实也不一定,要看体系结构了,在x86上而且内核编译flatmem的情形下,这就是读取物理内存。
if (kern_addr_valid(start)) {
unsigned long n;
n = copy_to_user(buffer, (char *)start, tsz);
return acc;
read 函数完毕之后,整个内存就被读出来了,存到一个地方保存那么这就是当时的内存运行快照,这里不得不说的是,这个信息可以用于调试,但是对于module的 调试就不是那么简单了,虽然kcore文件可以dump出整个内存,但是对于调试来说,这些信息是不够的,我们通过这些信息只能得到它当前是什么,而不能 得到它应该是什么,要想得到它应该是什么就必须有了原始的副本,幸运的是,linux的物理内存一一映射使得这个问题简化,linux内核vmlinuz 或者用于调试的vmlinux本身就是一个elf文件,-g选项编译的内核还有很多调试信息,elf连接脚本上写了符号加载的位置,以及elf的code 节,data节等等elf的要素,一一映射使得内核连接脚本的编写很简单,而且使得该脚本连接得到的内核载入内核时很容易的映射到了很简单的虚拟内存位 置,就是一个地址加上偏移。但是简单也就到此为止了,试想一下可加载的内核模块(LKM),在sys_init_module系统调用实现函数里发现模块 都是被映射到了vmalloc动态内存空间,包括它的代码,数据等等,如此一来,module的elf文件中写的节的载入地址在linux内核映射策略面 前成了一堆废物,即使你用module的原始副本来调试从/proc/kcore导出的映像也会发现很多的调试信息对不上,因此如何调试模块也就成了一个 大问题,linux的内核开发者也在着手解决这个问题…

linux编译出现 (.text+0x20):对‘main’未定义的引用 collect2: 错误:ld 返回 1 | (.text+0x20): undefined reference to `main' and undefined reference to function 解决方法

操作系统:ubuntu18.04 hpbook2000 gcc版本7.3.0

首先借鉴了国内网站,但大多说的是Makefile 很奇怪,我以为是源码编译的时候的问题但大家都是用”apt install gcc && apt install g++“来安装的,按说是不会出现这种错误的。我那时比较幼稚,就傻吼吼的等了一天源码编译gcc。

搞定了也没用,照样提示以上错误。那怎么办,听网上说只要把main 改成_start 或者不是main的函数名就ok了。又有人说最最主要的原因是Scrt1.o的main函数名定义在了使用它的前面所以报错。好吧,我就算知道也没法改.o文件怎么办。

网上又说有个可以痛改前非的办法,在gcc&g++加个-nostdlib或-nostartfile参数就可以。编译完说内核缺失,错的更离谱了。还有segmentation fault 搞笑。


cd /usr/lib/ && cp crt1.o Scrt1.o