Databricks Community

Hatter1337

Hi Databricks Community,I'm trying to read an index from OpenSearch or write a DataFrame into an OpenSearch index using the native Spark OpenSearch connector:host = dbutils.secrets.get(scope="opensearch", key="host") port = dbutils.secrets.get(scope=...

Hatter1337

I was close For opensearch.nodes, you should include https:# OpenSearch connection details host = dbutils.secrets.get(scope="opensearch", key="host") port = dbutils.secrets.get(scope="opensearch", key="port") os_url = f"https://{host}" index = "inde...

Hatter1337

Probably, this syntax is more correct:def save_to_open_search(df): writer = ( df.write.format("opensearch") .option("opensearch.nodes", host) .option("opensearch.port", port) .option("pushdown", "true") .op...

Databricks Community

User Stats

User Activity

Write Spark DataFrame into OpenSearch

Re: Write Spark DataFrame into OpenSearch

Re: Write Spark DataFrame into OpenSearch