topic Re: Error from mlflow traces in Databricks notebook in Generative AI

Error from mlflow traces in Databricks notebook

rafaelgildin — Thu, 27 Nov 2025 16:40:58 GMT

I'm unable to log my traces in my mlflow experiments.

The goal is to have the tracing from my genAI calls like described in Tutorial: Connect your development environment to MLflow .

So I'm able to create my experiment run, but they are every time empty, as shown below.

By adding the logging for showing debugging infos I was able to get the following error.

Overriding of current TracerProvider is not allowed 2025/11/27 16:16:35 DEBUG mlflow.tracing.utils: Failed to get attribute mlflow.traceRequestId with from span NonRecordingSpan(SpanContext(trace_id=0x00000000000000000000000000000000, span_id=0x0000000000000000, trace_flags=0x00, trace_state=[], is_remote=False)). Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-7eff4f88-fa35-45e5-8cad-fdcd71df8df3/lib/python3.12/site-packages/mlflow/tracing/utils/__init__.py", line 240, in get_otel_attribute attribute_value = span.attributes.get(key) ^^^^^^^^^^^^^^^

What I've tried to do for solving it.

Recreating my compute resource to check if the error persist. And it does.
Adding multiple layers for open telemetry and trying to change the packages versions.

Which none worked out. Also this is very strange, since I've people from my team that's able to run it with ease.

Current code I'm using with my notebook in databricks.

# -------------------------------- cell 1 %pip install uv %uv pip install -U 'mlflow[databricks]>=3.1' openai "flask<3.0.0" "opentelemetry-exporter-otlp-proto-grpc" dbutils.library.restartPython() # -------------------------------- cell 2 import os, logging, mlflow from dotenv import load_dotenv from databricks.sdk.runtime import dbutils # Initialize OpenTelemetry from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace.export import BatchSpanProcessor os.environ["MLFLOW_ENABLE_TRACES"] = "true" os.environ["MLFLOW_TRACKING_URI"] = "databricks" # Set up the tracer provider resource = Resource.create({"service.name": "mlflow-tracing"}) tracer_provider = TracerProvider(resource=resource) trace._TRACER_PROVIDER = None trace.set_tracer_provider(tracer_provider) logging.getLogger().setLevel(logging.DEBUG) mlflow.openai.autolog() mlflow.set_experiment(experiment_id=EXPERIMENT_ID) os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY from openai import OpenAI client = OpenAI() messages = [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "Hello!" } ] client.chat.completions.create(model="gpt-4o-mini-2024-07-18", messages=messages)

Re: Error from mlflow traces in Databricks notebook

rafaelgildin — Thu, 27 Nov 2025 17:09:20 GMT

Also, I've created this issue on Mlflow Github: https://github.com/mlflow/mlflow/issues/19092

Re: Error from mlflow traces in Databricks notebook

iyashk-DB — Thu, 27 Nov 2025 18:41:16 GMT

@rafaelgildin , I see that you have raised an Issue in the OSS mlflow repo, while the issue that you are encountering is on Databricks. The issue here is that you are explicitly adding trace.set_tracer_provider(tracer_provider). Databricks installs its own OpenTelemetry TracerProvider by default.

You can remove the part and let autolog() generate the spans naturally.

Re: Error from mlflow traces in Databricks notebook

rafaelgildin — Thu, 27 Nov 2025 21:27:39 GMT

@iyashk-DB thanks for the response.
But I keep geeting the same error.
should I change something else pls?

2025/11/27 21:24:59 DEBUG mlflow.tracing.utils: Failed to get attribute mlflow.traceRequestId with from span NonRecordingSpan(SpanContext(trace_id=0x00000000000000000000000000000000, span_id=0x0000000000000000, trace_flags=0x00, trace_state=[], is_remote=False)). Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-c56af6ef-cbd4-4a46-8da6-ba9e8059213d/lib/python3.12/site-packages/mlflow/tracing/utils/__init__.py", line 240, in get_otel_attribute attribute_value = span.attributes.get(key) ^^^^^^^^^^^^^^^ AttributeError: 'NonRecordingSpan' object has no attribute 'attributes'. Did you mean: 'set_attributes'? 2025/11/27 21:24:59 WARNING mlflow.tracing.fluent: Failed to start span Completions: 'NonRecordingSpan' object has no attribute 'context'. For full traceback, set logging level to debug. Traceback (most recent call last): File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-c56af6ef-cbd4-4a46-8da6-ba9e8059213d/lib/python3.12/site-packages/mlflow/tracing/fluent.py", line 592, in start_span_no_context trace_id, encode_span_id(otel_span.context.span_id) ^^^^^^^^^^^^^^^^^ AttributeError: 'NonRecordingSpan' object has no attribute 'context'. Did you mean: '_context'? ChatCompletion(id='chatcmpl-Cgdlz7DUK2qHR4Qiu24n8ZYW2wFLc', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Hello! How can I assist you today?', refusal=None, role='assistant', annotations=[], audio=None, function_call=None, tool_calls=None))], created=1764278699, model='gpt-4o-mini-2024-07-18', object='chat.completion', service_tier='default', system_fingerprint='fp_eca0ce8298', usage=CompletionUsage(completion_tokens=9, prompt_tokens=19, total_tokens=28, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0)))