Integrating Telemetry Plugin with OpenTelemetry Collector

In a NitroEnclaveDeployment, telemetry is handled by a dedicated telemetry plugin that runs as a special container inside the enclave. This plugin is responsible for collecting internal metrics and logs, and then pushing them to an OpenTelemetry Collector. Once the telemetry data reaches the collector, it can be processed and exported to any backend you choose (for example, Prometheus for metrics or Loki for logs).

To install OpenTelemetry Collector, check out the official documentation.

Below is a sample manifest that demonstrates how to configure the telemetry plugin for a NitroEnclaveDeployment. In this example, the telemetry plugin pushes data to an OTEL collector running at the endpoint http://otel-collector.monitoring.svc.cluster.local:4318.

apiVersion: k8s.oblv.com/v1alpha1
kind: NitroEnclaveDeployment
metadata:
  name: hello-fastapi
  namespace: default
spec:
  userPlugins:
  - name: fastapi
    image: public.ecr.aws/oblivious-ai/oblv-sample-fastapi:latest
    ports:
    - containerPort: 8001
      hostPort: 4455
    command:
    - "python"
    - "/app/uvicorn_runner.py"
  plugins:
    telemetry:
      image: public.ecr.aws/oblivious-ai/oblv-telemetry-dev:dev
      name: telemetry-plugin
      volumes:
        - containerPath: /etc/oblv
          readOnly: true
          source:
            configMap:
              name: fastapi-telemetry-configmap
              items:
                - key: config.yaml
                  path: config.yaml
          name: telemetry-config-vol
      env:
        - name: HOST_PORT
          value: '8100'
        - name: EXPORTER_URL
          value: "http://otel-collector.monitoring.svc.cluster.local:4318"
      outboundConnections:
        - fqdn:
            value: otel-collector.monitoring.svc.cluster.local
          port: 4318
          tls: false
          redirects: false
  replicas: 1
  serviceAccount: enclave-pod
  hugepages-1Gi: 12Gi
  enclaveCpuCount: 2
  ingress:
    enabled: true
    internetFacing: true
    dnsHostName: monitoring-test.oblv.com
    ingressTlsCertificate: oblv-ingress-tls
    ports:
    - port: 4455
      targetPort: 4455
  caCertDetails:
    enclaveCertType: ENCLAVE_GENERATED

Additionally, the telemetry plugin’s configuration is provided via a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: fastapi-telemetry-configmap
  namespace: default
data:
  config.yaml: 'scrape_configs: []'

Here is a sample metric for k8s_node_memory_usage_bytes:

{
  job="oblv_telemetry_service",
  k8s_node_name="hello-fastapi-788791ab48-58c6dd584c-bsmds",
  oblv_deployment_name="hello-fastapi"
}
2.113527808e+09 1745181459055

Note

In this example, the label k8s_node_name is not the name of the EKS Node hosting the enclave. It is the name of enclave itself where the static pods are running. It gets the same name as the NitroEnclave pod.

Telemetry

Integrating Telemetry Plugin with OpenTelemetry Collector​

Integrating Telemetry Plugin with OpenTelemetry Collector