Databricks Community

Andrei_Radulesc · 02-22-2023

I'm trying to get rid of the warning below:/databricks/spark/python/pyspark/sql/context.py:117: FutureWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.In my setup, I have a front-end notebook that gets parameters from the ...

Andrei_Radulesc · 02-07-2023

We have a team of multiple people, and they need to do Databricks Terraform config from the CLI. We'd like to do this without having all people be account admins in Databricks.What is the best way to do that?Right now, we have Terraform set up so tha...

Andrei_Radulesc · 02-06-2023

With Databricks Terraform connector version 1.2.0, I use the following to make the AWS instance profile available to all users in the workspace:// Create AWS instance profileresource "aws_iam_instance_profile" "this" { name = "${var.prefix}_instance_...

Andrei_Radulesc · 11-28-2022

Only the GUI seems to allow SELECT and USE_SCHEMA 'account users' permissions on catalogs. Terraform gives me an error. Here is my Terraform config:resource "databricks_grants" "staging" { provider = databricks.workspace catalog = databricks_catalog....

Andrei_Radulesc · 11-17-2022

I can create account groups in Terraform as follows:resource "databricks_group" "dev_perception" { provider = databricks.mws display_name = "Perception"}Or I can create a workspace group, using the workspace provider instead of the account provide...

Andrei_Radulesc · 04-14-2023

I have the same problem, and would be interested in a solution

Andrei_Radulesc · 04-10-2023

I think the Service Principal definitely works for workspaces created from scratch with the Service Principal. I've tested that in my test environment, and have not encountered these kinds of authentication problem.The issue I am seeing is with works...

Andrei_Radulesc · 04-10-2023

Thank you Suteja. It sort-of works, but I am still encountering authentication problems. I have updated to Databricks terraform provider 1.13.0 for this. Here is my account provider:provider "databricks" { alias = "mws" host = "https://accou...

Andrei_Radulesc · 03-17-2023

That fixes it. Thanks. I need to do spark = SparkSession.builder.getOrCreate() df = spark.table("prod.some_schema.some_table")instead of sc = SparkSession.builder.getOrCreate() sqlc = SQLContext(sc) df = sqlc.table(f"prod.some_schema.some...

Andrei_Radulesc · 02-22-2023

It just seems to me I already call SparkSession.builder.getOrCreate() , and still get the warning.

Databricks Community

User Stats

User Activity

FutureWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.

Run Terraform from CLI by non-admin users

" Please migrate to `databricks_group_role` "

Terraform can set ALL_PRIVILEGES and USE_CATALOG on catalogs for 'account users', but not # SELECT or USE_SCHEMA

How do I associate an account group with a workspace in Terraform?

Re: How to instantiate Databricks spark context in a python script?

Re: Run Terraform from CLI by non-admin users

Re: Run Terraform from CLI by non-admin users

Re: FutureWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.

Re: FutureWarning: Deprecated in 3.0.0. Use SparkSession.builder.getOrCreate() instead.