topic Re: Langchain SQLDatabase not fetching column names from table in Unity Catalog in Generative AI

Langchain SQLDatabase not fetching column names from table in Unity Catalog

ericy — Sun, 06 Apr 2025 01:44:14 GMT

I am building a text-to-sql agent using Langchain API.

I created a SQLDatabase using:

db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")

When I call db.get_table_info() I get the following error:

sqlalchemy.exc.DatabaseError: (databricks.sql.exc.ServerOperationError) [UNRESOLVED_COLUMN.WITHOUT_SUGGESTION] A column, variable, or function parameter with name `FROM` cannot be resolved. SQLSTATE: 42703; line 2 pos 0
[SQL: SELECT
FROM my_table
LIMIT %(param_1)s]
[parameters: {'param_1': 3}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)

Stepping through the code I found that SQLDatabase is not fetching the table columns. However, if I execute the following command using a connection from db._engine.connect(), I can retrieve the table column definitions:

schema_query = """
SELECT 
    table_name,
    column_name,
    data_type
FROM `my-catalog-name`.information_schema.columns 
WHERE table_schema = 'my_schema'
ORDER BY table_name, ordinal_position
"""

Not sure what I am missing.

Any help would be greatly appreciated.

Re: Langchain SQLDatabase not fetching column names from table in Unity Catalog

sakshi_in_loops — Mon, 05 May 2025 17:01:23 GMT

Hi there,

I am facing the similar issue. Were you able to fix this?

Re: Langchain SQLDatabase not fetching column names from table in Unity Catalog

mark_ott — Sat, 08 Nov 2025 13:09:26 GMT

The error occurs because the underlying query used by Langchain's SQLDatabase.get_table_info() method does not properly specify columns in the SELECT statement, resulting in a malformed query:
SELECT FROM my_table LIMIT %(param_1)s
which is missing column names and thus generates the "UNRESOLVED_COLUMN.WITHOUT_SUGGESTION" exception from Databricks SQL.

Root Cause

The Langchain SQLDatabase relies on SQLAlchemy's metadata inspection to fetch table and column information.
Databricks' SQL dialect or driver may not fully support SQLAlchemy’s reflection methods, or they require explicit column listing in the query.
Your manual query works because it directly queries the information_schema.columns view, which is the correct way to retrieve schema details for Databricks.

Solutions

1. Use Custom Schema Queries

Since your manual query works, override or replace the schema-fetching logic in your agent with your custom query to the information_schema.columns view:

python

schema_query = """
SELECT 
    table_name,
    column_name,
    data_type
FROM `my-catalog-name`.information_schema.columns 
WHERE table_schema = 'my_schema'
ORDER BY table_name, ordinal_position
"""
with db._engine.connect() as conn:
    results = conn.execute(schema_query).fetchall()
    # Transform results to desired format

Use these results to inform your agent’s schema construction.

2. Manually Construct Table and Column Metadata

You can manually instantiate table metadata for Langchain by building a dictionary from your results:

python

from langchain.sql_database import SQLDatabase

table_info = {
    table: [col for col in cols]  # Build table: columns mapping from the query results
}

# Inject this info into your agent where schema details are needed, or use for prompt construction.

This bypasses reflection issues.

3. Check SQLAlchemy + Databricks Compatibility

Ensure you have the latest versions of sqlalchemy and Databricks connectors.
Reflection with Databricks SQL endpoints may require explicit catalog and schema notation (my-catalog-name.my_schema.my_table).
If Langchain does not expose a hook for custom schema fetching, consider subclassing or contributing a patch.

Additional Notes

The error is not with your credentials or setup, but with auto-reflection logic.
The recommended path is to query information_schema.columns directly for schema extraction with Databricks SQL.
Replacing or patching the schema introspection in Langchain or SQLAlchemy may be necessary for full automation.

References

Re: Langchain SQLDatabase not fetching column names from table in Unity Catalog

Thompson2345 — Sun, 09 Nov 2025 07:56:54 GMT

The issue happens because SQLDatabase.from_databricks doesn’t automatically fetch column metadata from Unity Catalog tables in some LangChain versions, leading to malformed SELECT statements. A workaround is to manually query the catalog like you did (information_schema.columns) and pass the column info to LangChain, or use include_tables/table_metadata parameters if supported in your LangChain version. Also, ensure you’re using the latest langchain and databricks-sql-connector releases, as newer versions handle Unity Catalog metadata better.