- 2514 Views
- 2 replies
- 0 kudos
REST API call throwing 403
Hi, Calling curl works just fine from my workspace ```` %sh curl --request GET "https://${DATABRICKS_HOST}/api/2.0/clusters/list" --header "Authorization: Bearer ${DATABRICKS_API_TOKEN}"```But when I transpose the same logic to a script in javascrip...
- 2514 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks for sharing your findings, you can archive the ticket from Options on the right hand side.
- 0 kudos
- 1618 Views
- 0 replies
- 0 kudos
How to optimize a View ?
I have create a view from my underlying delta live table. Generally delta live table get auto optimize with vaccum in place. If thats so why do my view take 1 hour to be queried ? Any other way to optimize it ?
- 1618 Views
- 0 replies
- 0 kudos
- 3011 Views
- 2 replies
- 1 kudos
How to Determine Which CLI Methods to Use?
The Databricks CLI is considered Legacy for versions below 0.17 and is in Public Preview for versions above 0.20. I have access to documentation for both these versions separately. As I am developing a new project, I prefer not to use legacy options ...
- 3011 Views
- 2 replies
- 1 kudos
- 1 kudos
While the legacy CLI (versions 0.18 and below) is still available, it is not receiving any non-critical updates and it is recommend migrating to the new CLI as soon as possible. "Databricks recommends that you use Databricks CLI version 0.205 or abov...
- 1 kudos
- 3020 Views
- 1 replies
- 0 kudos
Community version now shows only the MLpersona
hiI have been using the community version for sometime and had saved some notebooks. While the notebooks are there in the workspace the different personas are not available as seen today. Only machine learning persona is visible. Would like to check...
- 3020 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi MSD, We removed the different personas that was observed earlier and now it is a flat system (It is not a recent change, been there for sometime). The notebooks would be saved in your workspace hence they are accessible. Irrespective of the perso...
- 0 kudos
- 2358 Views
- 0 replies
- 0 kudos
dbdemo LLAM CHATBOT RAG
i have an issue when running the below code using the default dbdemos in the advanced preparation , i have reduced the chunk_size and max_batch_size and running the code in a proper compute resources , could anyone help on that please :(spark.readStr...
- 2358 Views
- 0 replies
- 0 kudos
- 8297 Views
- 4 replies
- 0 kudos
pytest error
Hello,I have a quick question. If my source code call pysark collect() or any method related to rdd methods, then pytest on my local PC will report the following error. My local machine doesn't have any specific setting for pyspark and I used findspa...
- 8297 Views
- 4 replies
- 0 kudos
- 0 kudos
Thank you very much, brockb. Probably I will try it in databricks. Thanks.
- 0 kudos
- 7147 Views
- 0 replies
- 0 kudos
SQL Statement Execution API w/ Javascript (REST)
I need to use Databricks SQL Statement Execution API w/ Javascript (see example post )For some reason, Curl Works, Python works, but Javascript fails.This works : (curl)______________________________curl --request POST \https://adb-5750xxxxxxx.azured...
- 7147 Views
- 0 replies
- 0 kudos
- 1815 Views
- 2 replies
- 0 kudos
DataBricks Certification Exam Got Suspended. Require Immediate support
Hi Team,Today (29th May 2024), I began my Databricks assessment exam, but it was abruptly suspended by the proctor without any explanation. This was my first exam and it has been a disappointing experience.I started the exam calmly, but the proctor w...
- 1815 Views
- 2 replies
- 0 kudos
- 0 kudos
Thank you, @Cert-Team , for your quick response.I am looking forward to the resolution of my issue.
- 0 kudos
- 1322 Views
- 0 replies
- 0 kudos
Create a community user group Romania
Hello,I would like to receive some support in creating a Community User Group in Romania, Cluj-Napoca. Our intention as a company is to become a partner and us to create a community: events, meetups so on.Looking for your help.Thank youNicu
- 1322 Views
- 0 replies
- 0 kudos
- 3141 Views
- 3 replies
- 1 kudos
"AWS S3 resource has been disabled" error on job, not appearing on notebook
I am getting an "INTERNAL_ERROR" on a databricks job submitted through the API. Which says:"Run result unavailable: run failed with error message All access to AWS S3 resource has been disabled"However, when I click on the notebook created by the job...
- 3141 Views
- 3 replies
- 1 kudos
- 1 kudos
@Retired_mod In the s3 logs of the run, I am seeing this:24/05/29 06:39:30 WARN FileSystem: Failed to initialize fileystem dbfs:///: java.io.FileNotFoundException: Bucket user-workspace-s3-bucket does not exist24/05/29 06:39:30 ERROR DbfsHadoop3: Fai...
- 1 kudos
- 3598 Views
- 2 replies
- 0 kudos
Insufficient privileges error when running query from Notebook
Hello all, I am running into a permission issue when running a simple MERGE INTO query from a Notebook: " AnalysisException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User does not have USE SCHEMA on Schema 'system.query'."I can run the que...
- 3598 Views
- 2 replies
- 0 kudos
- 0 kudos
Are you using an all purpose cluster or a SQL warehouse to run this query in the notebook?
- 0 kudos
- 5631 Views
- 3 replies
- 0 kudos
issue related to Cluster Policy
Hello Databricks Community,I am currently working on creating a Terraform script to provision clusters in Databricks. However, I've noticed that by default, the clusters created using Terraform have the policy set to "Unrestricted."I would like to co...
- 5631 Views
- 3 replies
- 0 kudos
- 0 kudos
The policy id will persist, it is tied to the configuration you have set, even if changing the configuration of a custom policy it will persist with same policy id
- 0 kudos
- 3336 Views
- 1 replies
- 2 kudos
MLOps on Azure: API vs SDK vs Databricks CLI?
Hello fellow community members,In our organization, we have developed, deployed and utilized an API-based MLOps pipeline using Azure DevOps.The CI/CD pipeline has been developed and refined for about 18 months or so, and I have to say that it is pret...
- 3336 Views
- 1 replies
- 2 kudos
- 2 kudos
Hello @ManiMar, In my opinion it's up to you to choose, and you're in the right path by comparing the pros/cons of each approach. I'd like to highlight that one of the advantages of the Databricks CLI is being able to use Databricks Asset Bundles. I...
- 2 kudos
- 1666 Views
- 1 replies
- 0 kudos
Unity Catalog - Quality - Monitor error
Monitor errorAn error occurred while configuring your monitor for this table:Error while creating dashboard for unity-catalog-xxx: com.databricks.api.base.DatabricksServiceException: INTERNAL_ERROR: An internal error occurredPlease delete and recreat...
- 1666 Views
- 1 replies
- 0 kudos
- 0 kudos
If you have too many dashboards, there's a chance that the workspace reached the quota. I recommend you contacting Databricks Support for a more in-depth analysis.
- 0 kudos
- 4274 Views
- 2 replies
- 0 kudos
count or toPandas taking too long
Hi,I am fetching data from unity catalog from notebooks using spark.sql(). The query takes just a few seconds - I am actually trying to retrieving 2 rows - but some operations like count() or toPandas() take forever. I wonder why does it take so long...
- 4274 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey @jimcast how are you? You can check the internals and have a good hint of what's happening using the SparkUI. Filter and select the jobs that are taking the longest and check what is being requested on the SQL/Data Frame tab, as well as their pla...
- 0 kudos
