Langchain SQLDatabase not fetching column names from table in Unity Catalog

[ericy]

I am building a text-to-sql agent using Langchain API.

I created a SQLDatabase using:

db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")

When I call db.get_table_info() I get the following error:

sqlalchemy.exc.DatabaseError: (databricks.sql.exc.ServerOperationError) [UNRESOLVED_COLUMN.WITHOUT_SUGGESTION] A column, variable, or function parameter with name `FROM` cannot be resolved. SQLSTATE: 42703; line 2 pos 0
[SQL: SELECT
FROM my_table
LIMIT %(param_1)s]
[parameters: {'param_1': 3}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)

Stepping through the code I found that SQLDatabase is not fetching the table columns. However, if I execute the following command using a connection from db._engine.connect(), I can retrieve the table column definitions:

schema_query = """
SELECT 
    table_name,
    column_name,
    data_type
FROM `my-catalog-name`.information_schema.columns 
WHERE table_schema = 'my_schema'
ORDER BY table_name, ordinal_position
"""

Not sure what I am missing.

Any help would be greatly appreciated.

[sakshi_in_loops]

Hi there,

I am facing the similar issue. Were you able to fix this?

[mark_ott]

The error occurs because the underlying query used by Langchain's SQLDatabase.get_table_info() method does not properly specify columns in the SELECT statement, resulting in a malformed query:
SELECT FROM my_table LIMIT %(param_1)s
which is missing column names and thus generates the "UNRESOLVED_COLUMN.WITHOUT_SUGGESTION" exception from Databricks SQL.

Root Cause

• The Langchain SQLDatabase relies on SQLAlchemy's metadata inspection to fetch table and column information.

• Databricks' SQL dialect or driver may not fully support SQLAlchemy’s reflection methods, or they require explicit column listing in the query.

• Your manual query works because it directly queries the information_schema.columns view, which is the correct way to retrieve schema details for Databricks.

Solutions

1. Use Custom Schema Queries

Since your manual query works, override or replace the schema-fetching logic in your agent with your custom query to the information_schema.columns view:

python

schema_query = """ SELECT table_name, column_name, data_type FROM `my-catalog-name`.information_schema.columns WHERE table_schema = 'my_schema' ORDER BY table_name, ordinal_position """ with db._engine.connect() as conn: results = conn.execute(schema_query).fetchall() # Transform results to desired format

Use these results to inform your agent’s schema construction.

2. Manually Construct Table and Column Metadata

You can manually instantiate table metadata for Langchain by building a dictionary from your results:

python

from langchain.sql_database import SQLDatabase table_info = { table: [col for col in cols] # Build table: columns mapping from the query results } # Inject this info into your agent where schema details are needed, or use for prompt construction.

This bypasses reflection issues.

3. Check SQLAlchemy + Databricks Compatibility

• Ensure you have the latest versions of sqlalchemy and Databricks connectors.

• Reflection with Databricks SQL endpoints may require explicit catalog and schema notation (my-catalog-name.my_schema.my_table).

• If Langchain does not expose a hook for custom schema fetching, consider subclassing or contributing a patch.

Additional Notes

• The error is not with your credentials or setup, but with auto-reflection logic.

• The recommended path is to query information_schema.columns directly for schema extraction with Databricks SQL.

• Replacing or patching the schema introspection in Langchain or SQLAlchemy may be necessary for full automation.

References

• Langchain SQLDatabase source

• Databricks SQL information_schema

• SQLAlchemy Databricks dialect discussion

[Thompson2345]

The issue happens because SQLDatabase.from_databricks doesn’t automatically fetch column metadata from Unity Catalog tables in some LangChain versions, leading to malformed SELECT statements. A workaround is to manually query the catalog like you did (information_schema.columns) and pass the column info to LangChain, or use include_tables/table_metadata parameters if supported in your LangChain version. Also, ensure you’re using the latest langchain and databricks-sql-connector releases, as newer versions handle Unity Catalog metadata better.