'Permission Denied' Errors in vfunction-mysql Container running in a Podman environment
Overview of the Issue
This issue occurs in the following circumstances:
- An organization installs Podman on a Linux VM for the vFunction Server
- The organization configures a custom location on the filesystem for the Database and Measurement Volumes
- The organization adds the custom locations to the server-installation.yaml’s storage.db and storage.measurements fields. For example
storage:
### Leave empty to use an internal docker volume for database storage
database: /vfunction/podman/db
### Leave empty to use an internal docker volume for measurements storage
measurements: /vfunction/podman/measurements
- The organization adds the Keep-ID flag to the, “Podman Run” Workflow in the server-installation.yaml’s security.custom_security_options. For example
security:
### If empty, the default options would be: "--security-opt no-new-privileges --cap-drop=all"
custom_security_options: "--security-opt no-new-privileges --cap-drop=all --userns=keep-id"
- The organization successfully installs the vFunction Server
- The permissions are changed on the custom locations for the Database Volume
- The organization runs an upgrade in the environment
- Unexpectedly, the upgrade displays that the vfunction-mysql Container is stopped
Upgrading mysql...
Running docker with command: podman run --runtime runc --name vfunction-mysql --network=vfunction --ulimit nofile=65536:65536 --env-file /vfunction/podman/vfunction/etc/sysconfig/vfunction/mysql/automated-custom-env.list -e OFFLINE=true --add-host=vfunction-vfapi-organizations:10.89.0.1 -v /vfunction/podman/db:/opt/vfunction -p 3306:3306 --restart=unless-stopped --user 27:27 --security-opt no-new-privileges --cap-drop=all --userns=keep-id -d vfunction/vfunction-mysql-ubi:v3.6.1436
84faef998f1dce8ffb258b3301e409bf712a11a0e5cbadf30116e8c3d510383c
Backing up and upgrading mysql... (This may take a while depending on the DB size)
vfunction-mysql: stopped
vfunction-mysql: stopped
vfunction-mysql: stopped
vfunction-mysql: stopped
vfunction-mysql: stopped
- The output of, “podman logs vfunction-mysql” displays a, “Permission Denied” error trying to write or access files on the Database Volume
+ set -x
+ [[ -v DEBUG_IGNORE_SCRIPT_FAILURES ]]
++ cgroup-limits
Warning: Can't detect cpu quota from cgroups
Warning: Can't detect cpuset size from cgroups, will use nproc
+ export_vars='MAX_MEMORY_LIMIT_IN_BYTES=9223372036854775807
MEMORY_LIMIT_IN_BYTES=9223372036854775807
NUMBER_OF_CORES=2
NO_MEMORY_LIMIT=true'
+ export MAX_MEMORY_LIMIT_IN_BYTES=9223372036854775807 MEMORY_LIMIT_IN_BYTES=9223372036854775807 NUMBER_OF_CORES=2 NO_MEMORY_LIMIT=true
+ MAX_MEMORY_LIMIT_IN_BYTES=9223372036854775807
+ MEMORY_LIMIT_IN_BYTES=9223372036854775807
+ NUMBER_OF_CORES=2
+ NO_MEMORY_LIMIT=true
+ source /usr/share/container-scripts/mysql/common.sh
++ source /usr/share/container-scripts/mysql/helpers.sh
++ export MYSQL_DATADIR=/var/lib/mysql/data
++ MYSQL_DATADIR=/var/lib/mysql/data
++ export MYSQL_LOCAL_SOCKET=/tmp/mysql.sock
++ MYSQL_LOCAL_SOCKET=/tmp/mysql.sock
++ export MYSQL_DEFAULTS_FILE=/etc/my.cnf
++ MYSQL_DEFAULTS_FILE=/etc/my.cnf
++ export MYSQL_DATADIR_FIRST_INIT=false
++ MYSQL_DATADIR_FIRST_INIT=false
++ mysql_identifier_regex='^[a-zA-Z0-9_]+$'
++ mysql_password_regex='^[a-zA-Z0-9_~!@#$%^&*()-=<>,.?;:|]+$'
++ mysql_flags='-u root --socket=/tmp/mysql.sock'
++ admin_flags='--defaults-file=/etc/my.cnf -u root --socket=/tmp/mysql.sock'
+ source /usr/local/bin/backup.sh
++ set -x
++ _is_sourced /usr/bin/run-mysqld
+++ basename /usr/bin/run-mysqld
++ script_name=run-mysqld
+++ basename /usr/local/bin/backup.sh
++ this_script=backup.sh
++ '[' run-mysqld '!=' backup.sh ']'
+ [[ -z v3.6.1436 ]]
+ mkdir -p /opt/vfunction/logs
+ mkdir -p /opt/vfunction/errors
++ date +%Y%m%d%H%M%S
+ error_file_name=/opt/vfunction/errors/error-startup-20240814200146-v3.6.1436.log
+ touch /opt/vfunction/errors/error-startup-20240814200146-v3.6.1436.log
touch: cannot touch '/opt/vfunction/errors/error-startup-20240814200146-v3.6.1436.log': Permission denied
Solution
The following steps can be used to resolve this issue:
- Confirm that SELinux is not set to, “enforcing”. If it is, change to, “permissive”.
grep 'enforcing' /etc/sysconfig/selinux
- Get the starting Subuid for the User performing the Installation
Find-and-replace $INSTALLING_USER with actual Username
grep $INSTALLING_USER /etc/subuid
- Get the starting Subgid for the Group the User is in
Find-and-replace $INSTALLING_GROUP with actual Group Name
grep $INSTALLING_GROUP /etc/subgid
- Gather the MySQL Subuid and Subgid to be used for the ownership of the Database Volume. For example:
grep 'vfunction' /etc/subuid
vfunction:17000:6536
grep 'vfunction' /etc/subgid
vfunction:17000:6536
First Subuid = 17000
First Subgid = 17000
### Add 27 (the MySQL User ID in a UBI Image) to the First Subuid and First Subgid
MySQL Subuid = 17027
MySQL Subgid = 17027
- Change Ownership to the files and folders below the Database Volume to match the MySQL Subuid and MySQL Subgid. For example
sudo chown -R 17027:17027 /vfunction/podman/db
- Change Ownership of the Database Volume itself to be owned by the Installing User. For example
sudo chown vfunction:vfunction /vfunction/podman/db
- Run the upgrade script again to complete the upgrade process