- 1018 Views
- 1 replies
- 0 kudos
Databricks System Table access to group
Can we give system Table USE and SELECT permission to Group (AD group) instead or individual user ?
- 1018 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, You can grant SELECT to s system table for a group.
- 0 kudos
- 1921 Views
- 2 replies
- 1 kudos
Resolved! Connect databricks community edition to datalake s3/adls2
Can anybody know how can i connect with aws s3 object storage with databricks community edition or can i connect with community databricks account or not ?
- 1921 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @harsh_Dev ,You can read from/write to AWS S3 with Databricks Community edition. As you will not be able to use instance profiles, you will need to configure the AWS credentials manually and access S3 using S3 URI. Try below code spark._jsc.hadoop...
- 1 kudos
- 1225 Views
- 1 replies
- 1 kudos
Required versus current compute setup
To run demo and lab notebooks, I am required to have the following Databricks runtime(s): 15.4.x-cpu-ml-scala2.12 but the compute in my setup is of the following runtime version, will that be an issue? 11.3 LTS (includes Apache Spark 3.3.0, Scala 2.1...
- 1225 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @AGnewbie, Firstly, regarding the Databricks runtime: your compute setup is currently running version 11.3 LTS, which will indeed be an issue as the specified version is not present in your current runtime. Hence, you need to update your runtim...
- 1 kudos
- 5034 Views
- 1 replies
- 0 kudos
Decimal(32,6) datatype in Databricks - precision roundoff
Hello All,I need your assistance. I recently started a migration project from Synapse Analytics to Databricks. While dealing with the datatypes, I came across a situation where in Dedicated Sql Pool the value is 0.033882, but in DataBricks the value ...
- 5034 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Boyeenas ,I believe your assumption is correct. Databricks is built on Apache Spark and the system applies rounding automatically based on the value of the subsequent digit. In your case, if the original value had a 7th decimal digit of 5 or high...
- 0 kudos
- 1337 Views
- 1 replies
- 0 kudos
job "run name" in "system" "lake flow" "job run timeline" table
For few jobs in unity catalog the "run name" is coming out to be "null" whereas for few we the complete name with system generated batch id. I am not sure how this field is populated and why for some job's the "run name" is present whereas for some i...
- 1337 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @mishrarit! Run name in Unity Catalog job runs is determined by how the job is triggered. For manual runs, Databricks automatically generates a name, and for scheduled or API-triggered runs, the run name remains null unless explicitly defined.
- 0 kudos
- 2932 Views
- 2 replies
- 0 kudos
Set up compute policy to allow installing python libraries from a private package index
In our organization, we maintain a bunch of libraries we share code with. They're hosted on a private python package index, which requires a token to allow downloads. My idea was to store the token as a secret which would then be loaded into a cluste...
- 2932 Views
- 2 replies
- 0 kudos
- 0 kudos
I figured it out, seems like secrets can only be loaded into environment variables if the content is the secret and nothing else:"value": "{{secrets/global/arneCorpPyPI_token}}" # this will work"value": "foo {{secrets/global/arneCorpPyPI_toke...
- 0 kudos
- 1224 Views
- 1 replies
- 0 kudos
Creating Unity Catalog in Personal AZURE Portal Account
Seeking advice on the following:1. Given that I have a Personal - and not an Organization-based - AZURE Portal Account, 2. that I can see I am Global Admin and have Admin Role in Databricks, 3. then why can I not get "Manage Account" for a...
- 1224 Views
- 1 replies
- 0 kudos
- 0 kudos
@GerardAlexander Try signing in to the Account Console (https://accounts.azuredatabricks.net/login) using a user account with the appropriate permissions, rather than accessing it from the workspace.If you are unable to sign in, the following resourc...
- 0 kudos
- 4335 Views
- 3 replies
- 1 kudos
Power BI - Azure Databricks Connector shows Error AAD is not setup for domain
Hi Team,What I would like to do is understand what is required for PowerBI gateway to use single sign-on (AAD) to Databricks. Is that something you could have encountered before and know the fix? I currently get message from Power BI that AAD is not ...
- 4335 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello, did you have any solution for this? I am facing the same issue.
- 1 kudos
- 2274 Views
- 5 replies
- 0 kudos
Serveless compute does need has cloud accout(AWS、Google 、Azure)
I am a Databricks beginner, and I would like to ask if the Compute created in the Databricks account , it means also exists in the cloud account (e.g., AWS)? If the AWS account is deactivated, the existing compute will not be usable. This is what I h...
- 2274 Views
- 5 replies
- 0 kudos
- 0 kudos
@FanMichelleTW No, Databricks recommends using serverless compute, and you can use serverless compute as well.To do so, open a notebook and check the top-right corner to see if a serverless compute option is in a Ready state. If it is, simply select ...
- 0 kudos
- 6923 Views
- 4 replies
- 4 kudos
Creating a hierarchy without recursive statements
I am looking to build a hierarchy from a parent child relationship table, which I would typically use a recursive statement for in SQL Server / Azure SQL. This would mean setting an anchor, most commonly the top record of the tree, and then join back...
- 6923 Views
- 4 replies
- 4 kudos
- 2943 Views
- 4 replies
- 0 kudos
Error with Read XML data using the spark-xml library
hi, would appritiate any help with an error with loading an XML file with spark-xml library.my enviorment :14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12)library : com.databricks:spark-xml_2.12:0.15.0on databricks notebook.when running this scrip...
- 2943 Views
- 4 replies
- 0 kudos
- 0 kudos
UPDATE:It is now possible to read xml files directly: https://docs.databricks.com/en/query/formats/xml.html Make sure to update your Databricks Runtime to 14.3 and above, and remove the spark-xml maven library from your cluster.
- 0 kudos
- 1015 Views
- 1 replies
- 0 kudos
Handling Over-Usage of Capacity in Databricks Jobs/Processes
Hi all,Is there a tool or method in Databricks to ensure data integrity and stability when a job or process exceeds the allocated capacity? Specifically, I’m looking for ways to:Prevent failures or data loss due to resource overuse.Automatically scal...
- 1015 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @smanda88 - For point 1, please see; https://docs.databricks.com/en/lakehouse-architecture/reliability/best-practices.html For 2, you can use auto-scaling, please refer to: https://docs.databricks.com/en/lakehouse-architecture/cost-optimization...
- 0 kudos
- 1022 Views
- 1 replies
- 0 kudos
Where to find Jupyter Notebook course materials for Get Started with Databricks for Generative AI
Hello, I can't seem to find any way to gain access to the Jupyter Notebook demo source of "Get Started with Databricks for Generative AI" course. Please help. Thank you kindly in advance.
- 1022 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @nathanmle! We are sorry to inform you that we are no longer offering notebooks or the DBC files for the self-paced courses due to recent changes.If you’re interested in working on labs in a provided Databricks environment, you can purchase the...
- 0 kudos
- 3367 Views
- 0 replies
- 0 kudos
Databricks CE - Where is the quickstart tutorial?
Hello! I was looking through Databricks tutorials online, but my interface looks different from many of the videos I'm seeing. What happened to the Quickstart tutorials on the home page? Are they no longer available on the dashboard?
- 3367 Views
- 0 replies
- 0 kudos
- 2629 Views
- 1 replies
- 2 kudos
Databricks asset bundles dependencies
Is anyone aware of a way to include a requirements.txt within the job definition of a databricks asset bundle? Documentation mentions how to have dependencies in workspace files, or Unity Catalog volumes, but I wanted to ask if it is possible to decl...
- 2629 Views
- 1 replies
- 2 kudos
- 2 kudos
I have the same question.
- 2 kudos
