- 31880 Views
- 12 replies
- 10 kudos
Resolved! Databricks rbase container: Rstudio doesn´t work
Hello, How are you? I hope you are doing well!I´m trying to use a databrick´s image (link: containers/ubuntu/R at master · databricks/containers (github.com)) to run a container when starting a cluster. I need that Rstudio is installed on the contain...
- 31880 Views
- 12 replies
- 10 kudos
- 10 kudos
If the issue is resolved would you be happy to mark the answer as best so that others can quickly find the solution in the future.
- 10 kudos
- 13455 Views
- 7 replies
- 2 kudos
Resolved! Can I disable the workspace directory for specific user groups?
We want to use the REPO directory in our production environment only and have a dev environment with less restrictions. If I use the checkbox on the group admin screen to disable workspace access, it locks out the entire Data Engineering section.
- 13455 Views
- 7 replies
- 2 kudos
- 2 kudos
So I found a way to get 85% of the way there:1) Disable workspace access for the users group.2) Create a new group or use another group that you created for the next step.3) Go to the workspace and right click on whitespace in the root directory.4) A...
- 2 kudos
- 13145 Views
- 4 replies
- 5 kudos
Resolved! Is it possible to show multiple cmd output in a dashboard?
I have a loop that outputs a dataframe for values in a list; basically a loop. I can create a dashboard if there is only one df but in the loop, I'm only able to see the charts in the notebook if I switch the view to charts not in the dashboard. In t...
- 13145 Views
- 4 replies
- 5 kudos
- 5 kudos
If you want to be able to easily run and kill multiple process with ctrl-c, this is my favorite method: spawn multiple background processes in a (…) subshell, and trap SIGINT to execute kill 0, which will kill everything spawned in the subshell group...
- 5 kudos
- 9352 Views
- 2 replies
- 5 kudos
Resolved! %pip/%conda doesn't work with encrypted clusters starting DBR 9.x
While trying to use the magic command %pip/%conda with DBR 9.x or above it fails with the following error: %pip install numpy org.apache.spark.SparkException: %pip/%conda commands use unencrypted NFS and are disabled by default when SSL encryption ...
- 9352 Views
- 2 replies
- 5 kudos
- 5 kudos
If you are not aware of the traffic encryption between cluster worker nodes, you can refer to the below link.https://docs.microsoft.com/en-us/azure/databricks/security/encryption/encrypt-otw
- 5 kudos
- 14626 Views
- 5 replies
- 7 kudos
Resolved! Best mechanism to logging the notebook run/metadata and error details
Hi,How we can integrate log analytics with databricks to log notebook run details and code validations.Thank you
- 14626 Views
- 5 replies
- 7 kudos
- 7 kudos
I think you are looking for sending application logs.I'd use Log4j as this is already used by Databricks.The link does not use notebooks but it should work in notebooks too.
- 7 kudos
- 1376 Views
- 1 replies
- 0 kudos
An set up corporation’s image is the entirety. The right campaign strategies can make or ruin a organization’s brand image.business consultant Through...
An set up corporation’s image is the entirety. The right campaign strategies can make or ruin a organization’s brand image.business consultant Through digital advertising and marketing, powerful campaigns may be designed and the scope fixing any glit...
- 1376 Views
- 1 replies
- 0 kudos
- 0 kudos
CP500 is known as “Advance payment of income tax in installments” or “NotisBayaranAnsuran”. CP500 is a tax payment scheme designed by the IRM / LHDN for taxpayers to report their other forms of income, such as rental income, royalties, or other busin...
- 0 kudos
- 987 Views
- 0 replies
- 0 kudos
Find a local spokesperson for advice. Ask about their career path, how did they "get here"?Read books about speaking and writing.Analyze fa...
Find a local spokesperson for advice. Ask about their career path, how did they "get here"?Read books about speaking and writing.Analyze famous speeches text to speech software for yourself and do not rely on books that tell you the "why" and "how" o...
- 987 Views
- 0 replies
- 0 kudos
- 14881 Views
- 2 replies
- 4 kudos
Resolved! Parameterize a notebook
I was wondering if there's a way to parameterize a notebook similar to how the Papermill library allows you to parameterize Jupyter notebooks?
- 14881 Views
- 2 replies
- 4 kudos
- 4 kudos
you can try with widgets:https://docs.databricks.com/notebooks/widgets.htmlNot exactly the same as Papermill but it works fine. You can pass the values from your job orchestration tool into a widget so the notebook gets executed with the correct val...
- 4 kudos
- 8225 Views
- 4 replies
- 1 kudos
- 8225 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Vamsee krishna kanth Arcot Yes, currently you will have to download the JDBC from https://databricks.com/spark/jdbc-drivers-download and connect from other applications with JDBC URL just like you mentioned in your example. There is an internal ...
- 1 kudos
- 10987 Views
- 5 replies
- 4 kudos
Resolved! Programatic access to Files in Repos
Hi, we are testing the new Files support in Databricks repos. Is there a way how to programmatically read notebooks?Thanks
- 10987 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Jiri Koutny these files anyway should be synced to your remote repository (git, bitbucket, GitLab etc). The APIs from version control tools Git API for example might help you achieve what you want. https://stackoverflow.com/questions/38491722/r...
- 4 kudos
- 1683 Views
- 1 replies
- 0 kudos
Is there an equivalent of the %debug from Jupyter notebooks in Databricks notebooks for debugging python notebooks?
Is there an equivalent of the %debug from Jupyter notebooks in Databricks notebooks for debugging python notebooks?
- 1683 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Nathan Tong You can go through the 2 articles below that I found online for Debugging in Databricks.1. 7 Tips to Debug Apache Spark Code Faster with Databricks 2. Easier Spark Code Debugging
- 0 kudos
- 4139 Views
- 4 replies
- 0 kudos
I am not able to create a cluster
Hi,I am new on the Databricks platform, few weeks before I created a community version and it was working perfectly till 2 days before, now I can not create a cluster anymore, after few minutes it time out whenever I am trying to create a new cluster...
- 4139 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Ashwinkumar Jayakumar and @Prabakar Ammeappin , I am facing the same issue for 3-4 days.Is there something wrong with Community Edition right now or does my account facing some issues?
- 0 kudos
- 3938 Views
- 2 replies
- 0 kudos
Resolved! External metastore version
I am setting up an external metastore to connect my Databricks cluster. Which is the preferred and recommended Hive metastore version? Also are there any preference or recommendations on the database instance size/type
- 3938 Views
- 2 replies
- 0 kudos
- 0 kudos
@Harikrishnan Kunhumveettil we use databricks runtime 7.3LTS and 9.1LTS. And external hive metastore hosted on azue sql db. Using global init script I have set spark.sql.hive.metastore.version 2.3.7 and downloaded spark.sql.hive.metastore.jars f...
- 0 kudos
- 1851 Views
- 0 replies
- 0 kudos
Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot modify the value of a Spark config: spark.executor.memory;
I am trying to read a 16mb excel file and I was getting a gc overhead limit exceeded error to resolve that i tried to increase my executor memory with,spark.conf.set("spark.executor.memory", "8g")but i got the following stack :Using Spark's default l...
- 1851 Views
- 0 replies
- 0 kudos
- 10006 Views
- 3 replies
- 2 kudos
Resolved! Is there a way to refresh tokens issued on behalf of service principal?
I want to be able to refresh tokens generated on behalf of a service principal via Token Management API, just like with any other service where OAuth is used and refresh token endpoint is available. Allowing indefinite or very long expiration for acc...
- 10006 Views
- 3 replies
- 2 kudos
- 2 kudos
Refresh option would be useful.In Azure you could use Azure automation to make "refresh" script: delete if still existscreate token via: "databricks tokens create" put it to Azure Key Vault with expiration data
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 567 | |
| 349 | |
| 287 |
