- 4233 Views
- 7 replies
- 0 kudos
preloaded_docker_images: how do they work?
At my org, when we start a databricks cluster, it oftens takes awhile to become available (due to (1) instance provisioning, (2) library loading, and (3) init script execution). I'm exploring whether an instance pool could be a viable strategy for im...
- 4233 Views
- 7 replies
- 0 kudos
- 0 kudos
Hello, when we specify docker image with credentials in instance pool configuration, should we also specify credentials in cluster configuration?. as we already have image pulled into the pool instance.
- 0 kudos
- 4675 Views
- 2 replies
- 0 kudos
Is there an automated way to strip notebook outputs prior to pushing to github?
We have a team that works in Azure Databricks on notebooks.We are not allowed to push any data to Github per corporate policy.Instead of everyone having to always remember to clear their notebook outputs prior to commit and push, is there a way this ...
- 4675 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi,pushing to GitHub isn’t allowed, but clearing notebook outputs before internal version control is still important, you can automate this process by using a pre-commit hook or a script within your internal CI/CD pipeline (if one exists). Tools like...
- 0 kudos
- 3320 Views
- 3 replies
- 1 kudos
Replacing Excel with Databricks
I have a client that currently uses a lot of Excel with VBA and advanced calculations. Their source data is often stored in SQL Server.I am trying to make the case to move to Databricks. What's a good way to make that case? What are some advantages t...
- 3320 Views
- 3 replies
- 1 kudos
- 1 kudos
To add on this, my team and I have been using Databricks in an enterprise environment to replace Excel based calculation relying on SQL stored data with Calculations served as model serving endpoints (API) - the initial 'translation' work can be tedi...
- 1 kudos
- 1499 Views
- 2 replies
- 1 kudos
Resolved! Search page to search code inside .py files
Hello, hope you are doing good.When on the search page, it seems it's not searching for code inside .py files but rather only the filename.Is there an option somewhere I'm missing to be able to search inside .py files ? Best,Alan
- 1499 Views
- 2 replies
- 1 kudos
- 1 kudos
Hello, so it seems Databricks does not allow it - an easy workaround for us is to search directly on our Azure DevOps Repos.
- 1 kudos
- 1534 Views
- 1 replies
- 0 kudos
COMMUNITY EDITION CLUSTER DETACH JAVA.UTIL.TIMEOUTEXCEPTION.
Hi folks i was exploring the databricks community edition and came across a cluster issue mostly because of jdbc driver java.util.timeoutexception . basically the cluster connects and executes for 15 sec or so which is a socket limit and disables any...
- 1534 Views
- 1 replies
- 0 kudos
- 0 kudos
@Kishore23 paturnpikewrote:Hi folks i was exploring the databricks community edition and came across a cluster issue mostly because of jdbc driver java.util.timeoutexception . basically the cluster connects and executes for 15 sec or so which is a so...
- 0 kudos
- 52510 Views
- 11 replies
- 1 kudos
Error: Folder xxxx@xxx.com is protected
Hello, On Azure Databricks i'm trying to remove a folder on the Repos folder using the following command : databricks workspace delete "/Repos/xxx@xx.com"I got the following error message:databricks workspace delete "/Repos/xxxx@xx.com"Error: Folder ...
- 52510 Views
- 11 replies
- 1 kudos
- 1 kudos
Hello Databricks Forums,When you see the Azure Databricks error message "Folder xxxx@xxx.com is protected," it means that you are attempting to remove a system-protected folder, which is usually connected to a user's workspace, particularly under the...
- 1 kudos
- 1048 Views
- 1 replies
- 0 kudos
dev and prod
"SELECT * FROM' data call on my table in PROD is giving all the rows of data, but a call on my table in DEV is giving me just one row of data. what could be the problem??
- 1048 Views
- 1 replies
- 0 kudos
- 0 kudos
Tell us more about your environment. Are you using Unity Catalog? What is the table format? What cloud platform are you on? More information is needed.
- 0 kudos
- 1434 Views
- 2 replies
- 0 kudos
Will auto loader read files if it doesn't need to?
I want to run auto loader on some very large json files. I don't actually care about the data inside the files, just the file paths of the blobs. If I do something like``` spark.readStream .format("cloudFiles") .option("cloudFiles.fo...
- 1434 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @charliemerrell Yes, Databricks will still open and parse the JSON files, even if you're only selecting _metadata.It must infer schema and perform basic parsing, unless you explicitly avoid it.So, even if you do:.select("_metadata")It doesn't skip...
- 0 kudos
- 1464 Views
- 2 replies
- 1 kudos
Enroll, Learn, Earn Databricks !!
Hello Team,I had attended the session in CTS Manyata on 22nd April. I am interested in pursuing for the certifications but while enrolling it shows you are not a member of any group.Link for the available certifications and courses: https://community...
- 1464 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @samgupta88 you can find it on the partner academy. Everything is listed in the partner portal.
- 1 kudos
- 1473 Views
- 4 replies
- 0 kudos
UCX Installation error
Error Message: databricks.sdk.errors.platform.ResourceDoesNotExist: Can't find a cluster policy with id: 00127F76E005AE12.
- 1473 Views
- 4 replies
- 0 kudos
- 0 kudos
Click into each policy in the Compute UI of the Workspace to see if the policy ID exists. If it does, then the account that invoked the SDK method didn't have workspace admin permissions.
- 0 kudos
- 2217 Views
- 3 replies
- 0 kudos
Resolved! .py file running stuck on waiting
Hello, hope you are doing well.We are facing an issue when running .py files. This is fairly recent and we were not experiencing this issue last week.As shown in the screenshots below, the .py file hangs on "waiting" after we press "run all". No matt...
- 2217 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello, thanks a lot for your answer.We were getting the required permissions to use Firefox in our org, but in the meantime it seemed it worked again in Edge when it updated to version 135.0.3179.85 (Official build) (64-bit).
- 0 kudos
- 9645 Views
- 2 replies
- 2 kudos
Resolved! requirements.txt with cluster libraries
Cluster libraries are supported from version 15.0 - Databricks Runtime 15.0 | Databricks on AWS.How can I specify requirements.txt file path in the libraries in a job cluster in my workflow? Can I use relative path? Is it relative from the root of th...
- 9645 Views
- 2 replies
- 2 kudos
- 2 kudos
how to install requirement.txt using github action.- name: Install workspace requirements.txt on clusterenv:CLUSTER_ID: ${{ secrets.DATABRICKS_CLUSTER_ID }}run: |databricks libraries install \--cluster-id "$CLUSTER_ID" \--whl "dbfs:/FileStore/enginee...
- 2 kudos
- 3198 Views
- 5 replies
- 0 kudos
Debugging notebook access to external REST API
I'm using a Python Notebook with a REST API to access a system outside Databricks, in this case it's to call a SAS program. Identical python code works fine if I call it from jupyter on my laptop, but fails with a timeout when I run it from my Databr...
- 3198 Views
- 5 replies
- 0 kudos
- 0 kudos
What happens if you run command in notebook:nc -vz hostname 443If it fails to connect this will mean that the firewall or security groups associated with the VPC or VNet are not allowing this connection, you will need to check with your networking te...
- 0 kudos
- 1884 Views
- 1 replies
- 0 kudos
Trusted assets vs query examples
¡Hi community! In recent days I explored trusted assets in my genie space and this working very well! but I feel a little confused :sIn my genie space I have many queries examples when I create a new function with the same query example for verify th...
- 1884 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Dulce42! It depends on your use case. If your function covers the scenario well, you don’t need a separate query example. Having both for the same purpose can create redundancy and make things more complex. Choose the option that best fits you...
- 0 kudos
- 2167 Views
- 2 replies
- 0 kudos
Resolved! Need help to add personal email to databricks partner account
I have been actively using the Databricks Partner Academy for the past three years through my current organization. As I am planning to transition to a new company, I would like to ensure continued access to my training records and certifications.Cur...
- 2167 Views
- 2 replies
- 0 kudos
