- 950 Views
- 3 replies
- 0 kudos
Databricks Clean Rooms with 3 or more collaborators
Let's say I create a clean room with 2 other collaborators, call them collaborator A and collaborator B (so 3 in total, including me) and then shared some tables to the clean room. If collaborator A writes code that does a "SELECT * FROM creator.<tab...
- 950 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @pardeep7 , As per my understanding, all participants of clean room can only see metadata. The raw data in your tables is not directly accessed by other collaborators.Any output tables created by Collaborators based on the queries/notebooks will b...
- 0 kudos
- 559 Views
- 1 replies
- 0 kudos
Databricks System Table access to group
Can we give system Table USE and SELECT permission to Group (AD group) instead or individual user ?
- 559 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, You can grant SELECT to s system table for a group.
- 0 kudos
- 1099 Views
- 2 replies
- 1 kudos
Resolved! Connect databricks community edition to datalake s3/adls2
Can anybody know how can i connect with aws s3 object storage with databricks community edition or can i connect with community databricks account or not ?
- 1099 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @harsh_Dev ,You can read from/write to AWS S3 with Databricks Community edition. As you will not be able to use instance profiles, you will need to configure the AWS credentials manually and access S3 using S3 URI. Try below code spark._jsc.hadoop...
- 1 kudos
- 618 Views
- 1 replies
- 1 kudos
Required versus current compute setup
To run demo and lab notebooks, I am required to have the following Databricks runtime(s): 15.4.x-cpu-ml-scala2.12 but the compute in my setup is of the following runtime version, will that be an issue? 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.1...
- 618 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @AGnewbie, Firstly, regarding the Databricks runtime: your compute setup is currently running version 11.3 LTS, which will indeed be an issue as the specified version is not present in your current runtime. Hence, you need to update your runtim...
- 1 kudos
- 3331 Views
- 1 replies
- 0 kudos
Decimal(32,6) datatype in Databricks - precision roundoff
Hello All,I need your assistance. I recently started a migration project from Synapse Analytics to Databricks. While dealing with the datatypes, I came across a situation where in Dedicated Sql Pool the value is 0.033882, but in DataBricks the value ...
- 3331 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Boyeenas ,I believe your assumption is correct. Databricks is built on Apache Spark and the system applies rounding automatically based on the value of the subsequent digit. In your case, if the original value had a 7th decimal digit of 5 or high...
- 0 kudos
- 4302 Views
- 0 replies
- 0 kudos
How to resolve "cannot import name 'Iterable' from 'collections'" error?
I'm running a DBR/Spark job using a container. I've set docker_image.url to `docker.io/databricksruntime/standard:13.3-LTS`, as well as the Spark env var `DATABRICKS_RUNTIME_VERSION=13.3`. At runtime, however, I'm encountering this error: ImportError...
- 4302 Views
- 0 replies
- 0 kudos
- 609 Views
- 1 replies
- 0 kudos
job "run name" in "system" "lake flow" "job run timeline" table
For few jobs in unity catalog the "run name" is coming out to be "null" whereas for few we the complete name with system generated batch id. I am not sure how this field is populated and why for some job's the "run name" is present whereas for some i...
- 609 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @mishrarit! Run name in Unity Catalog job runs is determined by how the job is triggered. For manual runs, Databricks automatically generates a name, and for scheduled or API-triggered runs, the run name remains null unless explicitly defined.
- 0 kudos
- 1621 Views
- 2 replies
- 0 kudos
Set up compute policy to allow installing python libraries from a private package index
In our organization, we maintain a bunch of libraries we share code with. They're hosted on a private python package index, which requires a token to allow downloads. My idea was to store the token as a secret which would then be loaded into a cluste...
- 1621 Views
- 2 replies
- 0 kudos
- 0 kudos
I figured it out, seems like secrets can only be loaded into environment variables if the content is the secret and nothing else:"value": "{{secrets/global/arneCorpPyPI_token}}" # this will work"value": "foo {{secrets/global/arneCorpPyPI_toke...
- 0 kudos
- 759 Views
- 1 replies
- 0 kudos
Creating Unity Catalog in Personal AZURE Portal Account
Seeking advice on the following:1. Given that I have a Personal - and not an Organization-based - AZURE Portal Account, 2. that I can see I am Global Admin and have Admin Role in Databricks, 3. then why can I not get "Manage Account" for a...
- 759 Views
- 1 replies
- 0 kudos
- 0 kudos
@GerardAlexander Try signing in to the Account Console (https://accounts.azuredatabricks.net/login) using a user account with the appropriate permissions, rather than accessing it from the workspace.If you are unable to sign in, the following resourc...
- 0 kudos
- 3015 Views
- 0 replies
- 0 kudos
Is there a way to prevent databricks-connect from installing a global IPython Spark startup script?
I'm currently using databricks-connect through VS Code on MacOS. However, this seems to install (and re-install upon deletion) an IPython startup script which initializes a SparkSession. This is fine as far as it goes, except that this script is *glo...
- 3015 Views
- 0 replies
- 0 kudos
- 3317 Views
- 3 replies
- 1 kudos
Power BI - Azure Databricks Connector shows Error AAD is not setup for domain
Hi Team,What I would like to do is understand what is required for PowerBI gateway to use single sign-on (AAD) to Databricks. Is that something you could have encountered before and know the fix? I currently get message from Power BI that AAD is not ...
- 3317 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello, did you have any solution for this? I am facing the same issue.
- 1 kudos
- 3141 Views
- 0 replies
- 0 kudos
Can AWS workspaces share subnets?
The docs state:"You can choose to share one subnet across multiple workspaces or both subnets across workspaces."as well as:"You can reuse existing security groups rather than create new ones."and on this page:"If you plan to share a VPC and subnets ...
- 3141 Views
- 0 replies
- 0 kudos
- 3102 Views
- 0 replies
- 0 kudos
.py script execution failed but succeeded when run in Python notebook
Background:My code executing without problem if run in a python notebook. However, the same code fails when execute from a .py script in the workspace. Seems like the 2 execution methods don't have identical version of the packagesError message: Attr...
- 3102 Views
- 0 replies
- 0 kudos
- 3302 Views
- 0 replies
- 0 kudos
Updating a Delta Table in Delta Live Tables (DLT) from Two Event Hubs
I am working with Databricks Delta Live Tables (DLT) and need to ingest data from two different Event Hubs. My goal is to:Ingest initial data from the first Event Hub (Predictor) and store it in a Delta Table (data_predictions).Later, update this tab...
- 3302 Views
- 0 replies
- 0 kudos
- 1124 Views
- 5 replies
- 0 kudos
Serveless compute does need has cloud accout(AWS、Google 、Azure)
I am a Databricks beginner, and I would like to ask if the Compute created in the Databricks account , it means also exists in the cloud account (e.g., AWS)? If the AWS account is deactivated, the existing compute will not be usable. This is what I h...
- 1124 Views
- 5 replies
- 0 kudos
- 0 kudos
@FanMichelleTW No, Databricks recommends using serverless compute, and you can use serverless compute as well.To do so, open a notebook and check the top-right corner to see if a serverless compute option is in a Ready state. If it is, simply select ...
- 0 kudos
