Error from mlflow traces in Databricks notebook

[rafaelgildin]

I'm unable to log my traces in my mlflow experiments.

The goal is to have the tracing from my genAI calls like described in Tutorial: Connect your development environment to MLflow .

So I'm able to create my experiment run, but they are every time empty, as shown below.

By adding the logging for showing debugging infos I was able to get the following error.

Overriding of current TracerProvider is not allowed 2025/11/27 16:16:35 DEBUG mlflow.tracing.utils: Failed to get attribute mlflow.traceRequestId with from span NonRecordingSpan(SpanContext(trace_id=0x00000000000000000000000000000000, span_id=0x0000000000000000, trace_flags=0x00, trace_state=[], is_remote=False)). Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-7eff4f88-fa35-45e5-8cad-fdcd71df8df3/lib/python3.12/site-packages/mlflow/tracing/utils/__init__.py", line 240, in get_otel_attribute attribute_value = span.attributes.get(key) ^^^^^^^^^^^^^^^

 What I've tried to do for solving it.

1. Recreating my compute resource to check if the error persist. And it does. 
2. Adding multiple layers for open telemetry and trying to change the packages versions.

Which none worked out. Also this is very strange, since I've people from my team that's able to run it with ease.

Current code I'm using with my notebook in databricks.

# -------------------------------- cell 1
%pip install uv
%uv pip install -U 'mlflow[databricks]>=3.1' openai "flask<3.0.0" "opentelemetry-exporter-otlp-proto-grpc"
dbutils.library.restartPython()

# -------------------------------- cell 2
import os, logging, mlflow
from dotenv import load_dotenv
from databricks.sdk.runtime import dbutils

# Initialize OpenTelemetry
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor

os.environ["MLFLOW_ENABLE_TRACES"] = "true"
os.environ["MLFLOW_TRACKING_URI"] = "databricks"

# Set up the tracer provider
resource = Resource.create({"service.name": "mlflow-tracing"})
tracer_provider = TracerProvider(resource=resource)
trace._TRACER_PROVIDER = None 
trace.set_tracer_provider(tracer_provider)

logging.getLogger().setLevel(logging.DEBUG)
mlflow.openai.autolog()
mlflow.set_experiment(experiment_id=EXPERIMENT_ID)

os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

from openai import OpenAI
client = OpenAI()

messages = [
  {
    "role": "system", 
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "Hello!"
  }
]

client.chat.completions.create(model="gpt-4o-mini-2024-07-18", messages=messages)

[rafaelgildin]

Also, I've created this issue on Mlflow Github: https://github.com/mlflow/mlflow/issues/19092

[iyashk-DB]

@rafaelgildin , I see that you have raised an Issue in the OSS mlflow repo, while the issue that you are encountering is on Databricks. The issue here is that you are explicitly adding trace.set_tracer_provider(tracer_provider). Databricks installs its own OpenTelemetry TracerProvider by default. 

You can remove the part and let autolog() generate the spans naturally.

[rafaelgildin]

@iyashk-DB thanks for the response.
But I keep geeting the same error.
should I change something else pls?


2025/11/27 21:24:59 DEBUG mlflow.tracing.utils: Failed to get attribute mlflow.traceRequestId with from span NonRecordingSpan(SpanContext(trace_id=0x00000000000000000000000000000000, span_id=0x0000000000000000, trace_flags=0x00, trace_state=[], is_remote=False)). Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-c56af6ef-cbd4-4a46-8da6-ba9e8059213d/lib/python3.12/site-packages/mlflow/tracing/utils/__init__.py", line 240, in get_otel_attribute attribute_value = span.attributes.get(key) ^^^^^^^^^^^^^^^ AttributeError: 'NonRecordingSpan' object has no attribute 'attributes'. Did you mean: 'set_attributes'? 2025/11/27 21:24:59 WARNING mlflow.tracing.fluent: Failed to start span Completions: 'NonRecordingSpan' object has no attribute 'context'. For full traceback, set logging level to debug. Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-c56af6ef-cbd4-4a46-8da6-ba9e8059213d/lib/python3.12/site-packages/mlflow/tracing/fluent.py", line 592, in start_span_no_context trace_id, encode_span_id(otel_span.context.span_id) ^^^^^^^^^^^^^^^^^ AttributeError: 'NonRecordingSpan' object has no attribute 'context'. Did you mean: '_context'? ChatCompletion(id='chatcmpl-Cgdlz7DUK2qHR4Qiu24n8ZYW2wFLc', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Hello! How can I assist you today?', refusal=None, role='assistant', annotations=[], audio=None, function_call=None, tool_calls=None))], created=1764278699, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier='default', system_fingerprint='fp_eca0ce8298', usage=CompletionUsage(completion_tokens=9, prompt_tokens=19, total_tokens=28, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0)))