Configurable Settings - .NET Distributed Applications on Linux

No OpenTelemetry Collector
Using an OpenTelemetry Collector
Agent
Collector
N/A

Overview

vFunction Distributed Applications use the .NET Auto-instrumentation OpenTelemetry Agent to generate traces sent to the vFunction Server for Analysis. The OpenTelemetry Agent accepts Environment Variables to set configurable settings for the Agent.

If sending OpenTelemetry Traces only to the vFunction Server, these Environment Variables can be used to define the vFunction Server as the Endpoint for Traces and to eliminate all other types of unneeded OpenTelemetry traffic. In this scenario, all OpenTelemetry configurable settings are defined alongside the running application.

Alternatively, vFunction can be one of multiple OpenTelemetry vendors that receive information from the OpenTelemetry Agent. In this scenario, an OpenTelemetry Collector can be used to define some OpenTelemetry configurable settings and there are very few configurable settings to set alongside the running application.


No OpenTelemetry Collector


Required Agent Settings with no Collector

How to apply changes to the Required Agent Settings

Environment- or Process-level Environment Variables can be used for the Configurable Settings of the OpenTelemetry Agent. The addition of or changes to these Environment Variables will be picked up when the .NET Application is restarted.


Required Fields

Quick links to specific required fields:

Explanations of each required field:

Variable Name
Sample Value
Explanation of the Variable and Key Values
  • COR_PROFILER_PATH
  • CORECLR_PROFILER_PATH
Full path to OpenTelemetry.AutoInstrumentation.Native.so such as in /opt/otel/linux-glibc-x64/
  • The Variable Name that starts as COR_ is used by .NET Framework 4.x Applications. The Variable Name that starts as CORECLR_ is used by .NET Core and Standard 3.x, 6.x, 8.x, 9.x and 10.x Applications
  • The OpenTelemetry Agent can be installed by a number of methods. If installing manually by using the OpenTelemetry AutoInstrumentation ZIP, you will need to explicitly call out the path to this DLL. If using a Bash Script or PowerShell Module, this Environment Variable will be set as part of the script
  • COR_PROFILER
  • CORECLR_PROFILER
{918728DD-259F-4A6A-AC2B-B85E1B658318}
  • Each .NET Agent / Vendor has its own unique UUID used to validate the authenticity of the Agent. This UUID is that unchanging value for the OpenTelemetry .NET Agent
  • COR_ENABLE_PROFILING
  • CORECLR_ENABLE_PROFILING
0 or 1
  • With the value set to 0, the Application will not be hooked by the .NET Agent. With the value set to 1, the Application will be hooked
  • Modifying this value is the only required step to disable the OpenTelemetry Agent if, for example, debugging an issue on an Application that is believed to be associated with the OpenTelemetry Agent
  • OTEL_EXPORTER_OTLP_ENDPOINT
https://vfunction.mycompany.com/api/unauth/otlp
  • The api/unauth/otlp is the URL used on a vFunction Server to receive OpenTelemetry Traces
  • The URL will vary for each environment depending on the vFunction Server Address set during the installation process
  • OTEL_EXPORTER_OTLP_HEADERS
X-VF-APP=e1e1b85d-0000-0000-0000-aeced0597832
  • This value can be retrieved by logging into the vFunction Server UI, creating a Distributed Application, selecting Learning from the top-center menu bar, and clicking Installation Instructions on the left-hand sidebar
  • This value will be unique for each vFunction Distributed Application and must include, as part of the Value, the X-VF-APP Keyname

Sample Environment Variables used during the Application’s Startup
export CORECLR_PROFILER="{918728DD-259F-4A6A-AC2B-B85E1B658318}"
export CORECLR_PROFILER_PATH="/opt/otel/linux-glibc-x64/OpenTelemetry.AutoInstrumentation.Native.so"
export CORECLR_ENABLE_PROFILING=1
export OTEL_EXPORTER_OTLP_ENDPOINT="https://vfunction.mycompany.com/api/unauth/otlp"
export OTEL_EXPORTER_OTLP_HEADERS="X-VF-APP=e1e1b85d-0000-0000-0000-aeced0597832"
dotnet run

Optional Agent Settings with no Collector

A full list of configurable settings can be accessed on the OpenTelemetry website. The list below pertain to how vFunction uses the OpenTelemetry Agent.

Quick links to all optional fields:

Explanations of each optional field:

Variable Name
Sample Value
Explanation of the Variable and Key Values
  • DOTNET_ADDITIONAL_DEPS
$INSTALL_DIR/AdditionalDeps
  • Like DOTNET_SHARED_STORE, DOTNET_ADDITIONAL_DEPS is used to mitigate assembly version conflicts in .NET
  • DOTNET_SHARED_STORE
#INSTALL_DIR/store
  • Like DOTNET_ADDITIONAL_DEPS, DOTNET_SHARED_STORE is used to mitigate assembly version conflicts in .NET
  • DOTNET_STARTUP_HOOKS
$INSTALL_DIR/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
  • Required if the .NET CLR Profiler is not used
  • OTEL_BSP_SCHEDULE_DELAY
2000
  • Delay interval (in milliseconds) between two consecutive exports
  • Default value is 5000
  • OTEL_BSP_EXPORT_TIMEOUT
60000
  • Maximum allowed time (in milliseconds) to export data
  • Default value is 30000
  • OTEL_BSP_MAX_QUEUE_SIZE
4096
  • Maximum queue size
  • Default value is 2048
  • OTEL_BSP_MAX_EXPORT_BATCH_SIZE
1024
  • Maximum batch size. Must be less than or equal to OTEL_BSP_MAX_QUEUE_SIZE
  • Default value is 512
  • OTEL_DOTNET_AUTO_HOME
$INSTALL_DIR
  • Installation Location
  • OTEL_DOTNET_AUTO_EXCLUDE_PROCESSES
/opt/app/reservedProcess
  • Names of the executable files that the profiler cannot instrument. Supports multiple comma-separated values, for example: ReservedProcess.exe,powershell.exe. If unset, the profiler attaches to all processes by default
  • OTEL_DOTNET_AUTO_FAIL_FAST_ENABLED
true
  • Enables possibility to fail process when automatic instrumentation cannot be executed. It is designed for debugging purposes. It should not be used in production environment
  • OTEL_DOTNET_AUTO_ORACLEMDA_SET_DBSTATEMENT_FOR_TEXT
true
  • Used to show DB Table interactions when a .NET Service interacts with an Oracle Database
  • Defines whether or not the Oracle Client instrumentation can pass SQL statements through the db.statement attribute. If set to false, db.statement is recorded only for executing stored procedures.
  • OTEL_DOTNET_AUTO_SQLCLIENT_SET_DBSTATEMENT_FOR_TEXT
true
  • Used to show DB Table interactions when a .NET Service interacts with an Microsoft SQL Server Database
  • Defines whether or not the SQL Client instrumentation can pass SQL statements through the db.statement attribute. If set to false, db.statement is recorded only for executing stored procedures.
  • OTEL_EXPORTER_OTLP_CERTIFICATE
/opt/app/ca-cert.pem
  • Path to the CA certificate file (PEM format) used to verify the server’s TLS certificate
  • OTEL_EXPORTER_OTLP_HEADERS
X-VF-TAG=qa
  • Comma-separated list of additional HTTP headers sent with each export
  • The X-VF-APP header is required for a vFunction Distributed App
  • In addition to the X-VF-APP header, the X-VF-TAG header can be used to define tags to be associated to Services, such as environment tags like QA, DEV or Service Groupings
  • OTEL_EXPORTER_OTLP_PROTOCOL
http/protobuf
  • OTLP exporter transport protocol. Supported values are grpc, http/protobuf
  • The OpenTelemetry .NET Automatic Instrumentation defaults to http/protobuf, which differs from the OpenTelemetry .NET SDK default value of grpc
  • On .NET Framework, the grpc OTLP exporter protocol is not supported
  • OTEL_EXPORTER_OTLP_TIMEOUT
5000
  • The max waiting time (in milliseconds) for the backend to process each batch
  • Default value is 10000
  • OTEL_LOG_LEVEL
debug
  • SDK log level. (supported values: none, error, warn, info, debug)
  • Default value is "info"
  • OTEL_LOGS_EXPORTER
none
  • Comma-separated list of exporters. Supported options: otlp, console, none
  • The vFunction Distributed Application does not support Log Exports, so the value can be set to "none"
  • OTEL_METRICS_EXPORTER
none
  • Comma-separated list of exporters. Supported options: otlp, prometheus, console, none
  • The vFunction Distributed Application does not support Metrics Exports, so the value can be set to "none"
  • OTEL_SERVICE_NAME
  • OpenTelemetry tries to auto-detect the Service Name. For example, if the application is hosted on IIS in .NET Framework this will be SiteName\VirtualPath ex: MySite\MyApp
  • Explicitly adding the OTEL_SERVICE_NAME can be useful to ensure that Names match C4 Exports and vFunction Distributed Application Analysis
  • OTEL_TRACES_EXPORTER
otlp
  • Comma-separated list of exporters. Supported options: otlp, zipkin, console, none
  • Default value is "otlp"
  • The vFunction Distributed Application accepts the "otlp" traces exporter

Using an OpenTelemetry Collector


Required Agent Settings with an OpenTelemetry Collector

How to apply changes to the Required Agent Settings

Environment- or Process-level Environment Variables can be used for the Configurable Settings of the OpenTelemetry Agent. The addition of or changes to these Environment Variables will be picked up when the .NET Application is restarted.


Required Fields

Quick links to specific required fields:

Explanations of each required field:

Variable Name
Sample Value
Explanation of the Variable and Key Values
  • COR_PROFILER_PATH
  • CORECLR_PROFILER_PATH
Full path to OpenTelemetry.AutoInstrumentation.Native.so such as in /opt/otel/linux-glibc-x64/
  • The Variable Name that starts as COR_ is used by .NET Framework 4.x Applications. The Variable Name that starts as CORECLR_ is used by .NET Core and Standard 3.x, 6.x, 8.x, 9.x and 10.x Applications
  • The OpenTelemetry Agent can be installed by a number of methods. If installing manually by using the OpenTelemetry AutoInstrumentation ZIP, you will need to explicitly call out the path to this DLL. If using a Bash Script or PowerShell Module, this Environment Variable will be set as part of the script
  • COR_PROFILER
  • CORECLR_PROFILER
{918728DD-259F-4A6A-AC2B-B85E1B658318}
  • Each .NET Agent / Vendor has its own unique UUID used to validate the authenticity of the Agent. This UUID is that unchanging value for the OpenTelemetry .NET Agent
  • COR_ENABLE_PROFILING
  • CORECLR_ENABLE_PROFILING
0 or 1
  • With the value set to 0, the Application will not be hooked by the .NET Agent. With the value set to 1, the Application will be hooked
  • Modifying this value is the only required step to disable the OpenTelemetry Agent if, for example, debugging an issue on an Application that is believed to be associated with the OpenTelemetry Agent
  • OTEL_EXPORTER_OTLP_ENDPOINT
http://otelcollector.mycompany.com:4318
  • Target endpoint for the OTLP exporter. In this scenario, the OpenTelemetry Collector address
  • If using "http/protobuf" as the OTEL_TRACES_OTLP_PROTOCOL, then the default value is "http://localhost:4318"

Sample Environment Variables used during the Application’s Startup
export CORECLR_PROFILER="{918728DD-259F-4A6A-AC2B-B85E1B658318}"
export CORECLR_PROFILER_PATH="/opt/otel/linux-glibc-x64/OpenTelemetry.AutoInstrumentation.Native.so"
export CORECLR_ENABLE_PROFILING=1
export OTEL_EXPORTER_OTLP_ENDPOINT="http://otelcollector.mycompany.com:4318"
dotnet run

Optional Agent Settings with an OpenTelemetry Collector

A full list of configurable settings can be accessed on the OpenTelemetry website. The list below pertain to how vFunction uses the OpenTelemetry Agent.

Quick links to all optional fields:

Explanations of each optional field:

Variable Name
Sample Value
Explanation of the Variable and Key Values
  • DOTNET_ADDITIONAL_DEPS
$INSTALL_DIR/AdditionalDeps
  • Like DOTNET_SHARED_STORE, DOTNET_ADDITIONAL_DEPS is used to mitigate assembly version conflicts in .NET
  • DOTNET_SHARED_STORE
#INSTALL_DIR/store
  • Like DOTNET_ADDITIONAL_DEPS, DOTNET_SHARED_STORE is used to mitigate assembly version conflicts in .NET
  • DOTNET_STARTUP_HOOKS
$INSTALL_DIR/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
  • Required if the .NET CLR Profiler is not used
  • OTEL_BSP_SCHEDULE_DELAY
2000
  • Delay interval (in milliseconds) between two consecutive exports
  • Default value is 5000
  • OTEL_BSP_EXPORT_TIMEOUT
60000
  • Maximum allowed time (in milliseconds) to export data
  • Default value is 30000
  • OTEL_BSP_MAX_QUEUE_SIZE
4096
  • Maximum queue size
  • Default value is 2048
  • OTEL_BSP_MAX_EXPORT_BATCH_SIZE
1024
  • Maximum batch size. Must be less than or equal to OTEL_BSP_MAX_QUEUE_SIZE
  • Default value is 512
  • OTEL_DOTNET_AUTO_HOME
$INSTALL_DIR
  • Installation Location
  • OTEL_DOTNET_AUTO_EXCLUDE_PROCESSES
/opt/app/reservedProcess
  • Names of the executable files that the profiler cannot instrument. Supports multiple comma-separated values, for example: ReservedProcess.exe,powershell.exe. If unset, the profiler attaches to all processes by default
  • OTEL_DOTNET_AUTO_FAIL_FAST_ENABLED
true
  • Enables possibility to fail process when automatic instrumentation cannot be executed. It is designed for debugging purposes. It should not be used in production environment
  • OTEL_DOTNET_AUTO_ORACLEMDA_SET_DBSTATEMENT_FOR_TEXT
true
  • Used to show DB Table interactions when a .NET Service interacts with an Oracle Database
  • Defines whether or not the Oracle Client instrumentation can pass SQL statements through the db.statement attribute. If set to false, db.statement is recorded only for executing stored procedures.
  • OTEL_DOTNET_AUTO_SQLCLIENT_SET_DBSTATEMENT_FOR_TEXT
true
  • Used to show DB Table interactions when a .NET Service interacts with an Microsoft SQL Server Database
  • Defines whether or not the SQL Client instrumentation can pass SQL statements through the db.statement attribute. If set to false, db.statement is recorded only for executing stored procedures.
  • OTEL_EXPORTER_OTLP_CERTIFICATE
/opt/app/ca-cert.pem
  • Path to the CA certificate file (PEM format) used to verify the server’s TLS certificate
  • OTEL_EXPORTER_OTLP_HEADERS
X-VF-TAG=qa
  • Comma-separated list of additional HTTP headers sent with each export
  • The X-VF-APP header is required for a vFunction Distributed App
  • In addition to the X-VF-APP header, the X-VF-TAG header can be used to define tags to be associated to Services, such as environment tags like QA, DEV or Service Groupings
  • OTEL_EXPORTER_OTLP_PROTOCOL
http/protobuf
  • OTLP exporter transport protocol. Supported values are grpc, http/protobuf
  • The OpenTelemetry .NET Automatic Instrumentation defaults to http/protobuf, which differs from the OpenTelemetry .NET SDK default value of grpc
  • On .NET Framework, the grpc OTLP exporter protocol is not supported
  • OTEL_EXPORTER_OTLP_TIMEOUT
5000
  • The max waiting time (in milliseconds) for the backend to process each batch
  • Default value is 10000
  • OTEL_LOG_LEVEL
debug
  • SDK log level. (supported values: none, error, warn, info, debug)
  • Default value is "info"
  • OTEL_LOGS_EXPORTER
none
  • Comma-separated list of exporters. Supported options: otlp, console, none
  • The vFunction Distributed Application does not support Log Exports, so the value can be set to "none"
  • OTEL_METRICS_EXPORTER
none
  • Comma-separated list of exporters. Supported options: otlp, prometheus, console, none
  • The vFunction Distributed Application does not support Metrics Exports, so the value can be set to "none"
  • OTEL_SERVICE_NAME
  • OpenTelemetry tries to auto-detect the Service Name. For example, if the application is hosted on IIS in .NET Framework this will be SiteName\VirtualPath ex: MySite\MyApp
  • Explicitly adding the OTEL_SERVICE_NAME can be useful to ensure that Names match C4 Exports and vFunction Distributed Application Analysis
  • OTEL_TRACES_EXPORTER
otlp
  • Comma-separated list of exporters. Supported options: otlp, zipkin, console, none
  • Default value is "otlp"
  • The vFunction Distributed Application accepts the "otlp" traces exporter

Required Collector Settings with an OpenTelemetry Collector

Quick links to specific required fields:

Explanations of each required field:

Variable Name
Explanation of the Variable and Key Values
exporters:
   otlphttp/vf:
      endpoint: https://vfunction.mycompany.com/api/unauth/otlp
      headers:
         X-VF-APP: e1e1b85d-0000-0000-0000-aeced0597832
  • Responsible for sending processed OpenTelemetry data to vFunction
  • The api/unauth/otlp is the URL used on a vFunction Server to receive OpenTelemetry Traces
  • The URL will vary for each environment depending on the vFunction Server Address set during the installation process
  • The X-VF-APP Headers value can be retrieved by logging into the vFunction Server UI, creating a Distributed Application, selecting Learning from the top-center menu bar, and clicking Installation Instructions on the left-hand sidebar
  • This value will be unique for each vFunction Distributed Application and must include, as part of the Value, the X-VF-APP Keyname
service:
   pipelines:
      exporters: [otlphttp/vf]
  • Responsible for defining how OpenTelemetry exports OpenTelemetry data to vFunction

Optional Collector Settings with an OpenTelemetry Collector

A full list of configurable settings can be accessed on the OpenTelemetry website. The list below pertain to how vFunction uses the OpenTelemetry Agent.

Quick links to all optional fields:

Explanations of each optional field:

Variable Name
Explanation of the Variable and Key Values
exporters:
   otlphttp/vf:
      retry_on_failure:
         initial_interval: 10
         max_interval: 60
  • See OpenTelemetry documentation for full details
  • Retry_on_failure is enabled by default. Using this field allows you to disable the feature or modify the behavior of the default settings: enabled (true default), interval_interval (5s default), max_interval (30s default), max_elapsed_time (300s default), or multiplier (1.5 default)
exporters:
   otlphttp/vf:
      sending_queue:
         num_consumers: 20
         sizer: bytes
  • See OpenTelemetry documentation for full details
  • The sending_queue is enabled by default. Using this field allows you to disable this feature or modify the behavior of the default settings: enabled (true default), num_consumers (10 default), wait_for_result (false default), block_on_overflow (false default), sizer (requests)
exporters:
   otlphttp/vf:
      tls:
         insecure_skip_verify: true
  • See OpenTelemetry documentation for full details
  • By default, TLS is enabled with server certificate verification using the system root CAs. If using HTTP instead of HTTPS, set insecure: true. If using self-signed certificate for HTTPS, set insecure_skip_verify: true
  • If using internally signed certificates, use ca_file: /path/to/ca.crt
processors:
   probabilistic_sampling:
      sampling_percentage: 50
      mode: "proportional"
service:
   pipelines:
      traces:
         processors: [probabilistic_sampler]
  • See OpenTelemetry documentation for full details
  • An application with heavy usage may create a significant amount of OpenTelemetry traffic that prevents the OpenTelemetry Endpoint from processing this data without using a significant amount of resources. Probabilistic Sampling allows the OpenTelemetry Collector to receive the Application’s OpenTelemetry Traces, but to only send some percentage of this traffic onto the vFunction OpenTelemetry Endpoint for processing.