General Troubleshooting - Java Agent running on a Linux Host
Confirm Permissions
The vFunction folder structure (where the vFunction Agent reads from and writes to) should be owned by the user that runs the Java Application that is being profiled. So, for example, if the Application runs on a Tomcat Application Server and the “tomcat” user and “tomcat” group run the Application, then the “tomcat” user and “tomcat” group should own the vFunction Base Installation Directory and all files / folders below this Base Directory.
Take the following steps to confirm and, if required, change the permissions of the vFunction Base Directory:
- Confirm the user running the Java Application
### Find the running Java process
ps -ef | grep java
### Find the location of a relevant file from the process output and check the user and group ownership
ls -l /path/to/relevant/java/application/files
- Stop the Java Application
- Change the ownership of the vFunction directory to be owned by the user running the Java Application
### Modify the $USER, $GROUP and $BASE_DIR with the relevant values
chown -R $USER:$GROUP $BASE_DIR/vfunction
- Rerun the vFunction Installation Script
### Modify the $USER, $GROUP and $BASE_DIR with the relevant values
bash $BASE_DIR/vfunction/opt/vfunction/controller-installation/install.sh -i default-java
- Restart the Application
Confirm Startup Parameters
Based on the Agent Key Values in the vFunction installation.yaml or Environment Variables, a distinct set of Startup Parameters and steps to set these Parameters are displayed at the end of the output of the install.sh script. The Agent Keys and Values that can change the Startup Parameters are the agent.application_server and agent.application_java_version. Additionally, the type of Installation package (Sudo vs Sudoless) will also change the Startup Parameters used.
To find the Startup Parameters based on the installation.yaml Keys and Values for your application, open the folder $BASE_DIR/vfunction/etc/sysconfig/vfunction/agent/instances/$INSTANCE_NAME/vmargs-examples/.
To find generic Startup Parameters that do not include any information about your installation, open the folder $BASE_DIR/vfunction/opt/vfunction/controller-installation/stuff/agent/java/vmargs-examples.
Confirm Java Version and relevant Parameters
The vFunction Agents and Java versions can be considered in three distinct blocks with regards to how the Agent runs and the Startup Parameters needed:
- Java 6 / 7
- Java 8
- Java 11 / 17 / 21
Java 6 / 7
If the Application being profiled runs using Java 6 / 7, the vFunction Key agent:use_java_6_or_7_jre needs to be set to Value true. This Key and Value can be set either via Environment Variable VF_GLOBAL_AGENT_USE_JAVA_6_OR_7_JRE=true or via the $BASE_DIR/vfunction/etc/sysconfig/vfunction/installation/global.yaml before the install.sh / upgrade.sh script is run.
Java 8
If the Application being profiled runs using Java 8, no additional Java Parameters need to be added to the Startup Parameters.
Java 11 / 17 / 21
If the Application being profiled runs using Java 11 / 17 / 21, an additional Java Parameter needs to be added to the Startup Parameters. Assuming that the Environment Variable **VF_CONTROLLER_$INSTANCENAME_AGENT_APPLICATION_JAVA_VERSION** is set to 11 / 17 / 21 or the installation.yaml’s **agent.application_java_version** is set to 11 / 17 /21, the **$BASE_DIR/vfunction/etc/sysconfig/vfunction/agent/instances/$INSTANCE_NAME/vmargs-examples/** that is displayed at the end of the install.sh output will include the required line:
--add-opens java.base/java.lang=ALL-UNNAMED
This line needs to be in the Startup Parameters for a Java 11 / 17 / 21 for the vFunction Agent to profile the Application successfully.
Add Startup Parameters to a relevant location
vFunction recommends to add the Startup Parameters to a generic, Application Server-specific location in the $BASE_DIR/vfunction/etc/sysconfig/vfunction/agent/instances/$INSTANCE_NAME/vmargs-examples/. For some organizations, this may not be the appropriate location for the Startup Parameters. To find where other Application Server Startup Parameters are set, take the following steps:
- Find the relevant Java process for the Application
ps -ef | grep java
- Look for Java Options that would be explicitly set through a configuration file or script. For example, values that start with the string -X, -D, or -P.
- Search the Base Directory of the Application for one of the strings found in the Java process output. For example:
grep -irnw --color /opt/tomcat -e '-Xmx'
- Format the vFunction Startup Parameters correctly and add them to the location found in Step #3. For example:
- If the file is formatted as an XML, modify the vFunction values to match the XML-based formatting
- If the file uses a JAVA_OPTS to set the parameters, create a VFUN_OPTS Key with the full string of vFunction Startup Parameters and add the VFUN_OPTS to the JAVA_OPTS
- Restart the Application
Gather Logs
- Run the vFunction Script to Gather Logs
Root / Sudo Installation
bash /opt/vfunction/controller-installation/collect-logs.sh
Sudoless Installation
### Replace $BASE_DIRECTORY with the location of the vFunction Installation
bash $BASE_DIRECTORY/vfunction/opt/vfunction/controller-installation/collect-logs.sh
- Unpack the vFunction logs TGZ created in the /tmp directory
### Replace $VERSION with the version of the vFunction Server, such as v3.6.1436
### Replace $TIMESTAMP with the current timestamp, such as 20240821143907
cd /tmp
ls -l *.tgz
tar -xvzf v3.6.1436-20240821143907-controller-logs.tgz
- Review the vFunction logs and look for errors that give obvious next steps
- Send symptoms and logs to support@vfunction.com for additional support
Add Debugging Flags to Java Options
- Open the location where the current vFunction Java Options are set
- Add the flag to turn the Logging Level to DEBUG
-Dcom.vfunction.logging.level=DEBUG
- If an issue with Beans is suspected, add the flag to turn Beans Tracking Level to DEBUG
-Dcom.vfunction.logging.level.beanstracking=DEBUG
- If an issue with the DB Tracking is suspected, add the flag to turn the DB Tracking Level to DEBUG
-Dcom.vfunction.logging.level.dbtracking=DEBUG
- Restart the Application
- Gather Logs
- Remove the DEBUG flags