- 1781 Views
- 1 replies
- 1 kudos
Create job cluster with a docker image in azure data factory
Is there a way to create a job cluster in azure data factory with a docker image either through API or UI
- 1781 Views
- 1 replies
- 1 kudos
- 1 kudos
Does anyone handle how to run from ADF custom image as jobs compute?
- 1 kudos
- 1579 Views
- 1 replies
- 3 kudos
Resolved! stream from specific time
Hello,How should I start stream a delta table from specific point in time?
- 1579 Views
- 1 replies
- 3 kudos
- 3 kudos
If you are streaming from a Delta table, you can specify the starting version or timestamp. You can refer the document for complete detail: https://docs.databricks.com/structured-streaming/delta-lake.html#specify-initial-position
- 3 kudos
- 2337 Views
- 5 replies
- 1 kudos
When developing a Delta Live Table, can I test my code in the notebook?
Just wondering about the dev flow when building Delta Live Tables. I write my code in the notebook so it would be useful to be able to test it out from within that environment.
- 2337 Views
- 5 replies
- 1 kudos
- 1876 Views
- 2 replies
- 1 kudos
IllegalArgumentException: requirement failed: Invalid uri
Hi all, I'm trying to connect to MongoDB using the Databricks notebook. I keep getting the error that my MongoDB uri is invalid. The uri works when connecting from my local machine using the Rust driver. I pretty much followed the tutorial that was g...
- 1876 Views
- 2 replies
- 1 kudos
- 1 kudos
Also take in mind firewall issues in case the mongodb is on-prem or on some other location.
- 1 kudos
- 2898 Views
- 3 replies
- 2 kudos
Resolved! Cannot import-dir from AzureDevops, but works fine locally.
Hello,as i'm trying to create a CI/CD for the project, I'm finding myself stuck.Tried to upload the Notebooks from my Azure DevOps Release and I'm getting 403-forbidden access.I used 'cat ~/.databrickscfg file and matched with the local config that I...
- 2898 Views
- 3 replies
- 2 kudos
- 2 kudos
Hey everyone! I can totally relate to the frustration of encountering authentication issues when setting up a CI/CD pipeline. It's great that you're able to import the notebooks locally, but facing difficulties on Azure DevOps can be quite puzzling.F...
- 2 kudos
- 3173 Views
- 2 replies
- 1 kudos
Resolved! Cluster Health Dashboard
Is there a cluster health dashboard which has the details of the total number of running interactive cluster, the total number of job clusters? Also Flag clusters with issues.
- 3173 Views
- 2 replies
- 1 kudos
- 2605 Views
- 2 replies
- 0 kudos
Connect to Azure DevOps repository using service principle
My source code is in the VSTS repository and I am using PAT token to connect VSTS from Azure data bricks notebook and then building packages and installing my cluster. For the production environment, I can't use PAT token, so is there any way to conn...
- 2605 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey everyoneI've been working with Azure DevOps and VSTS repositories, and I can relate to the challenges of connecting them securely. thushar, I understand your concern about using a PAT token for production environments. Fortunately, there is indee...
- 0 kudos
- 2582 Views
- 3 replies
- 7 kudos
Resolved! How to start with Databricks in Google Cloud?
I am looking through Google Cloud Platform and I am looking to get started with Databricks on GCP. Happy if anyone can point me in the direction that can provide guidance on how to get started.Thansk
- 2582 Views
- 3 replies
- 7 kudos
- 7 kudos
Hey boyelana Databricks on Google Cloud Platform is definitely an interesting and powerful combination, and I'm thrilled to see that you're looking to get started with it, boyelana!To begin your journey with Databricks on GCP, there are a few steps y...
- 7 kudos
- 2683 Views
- 2 replies
- 2 kudos
Resolved! Access to personal access token via python
Is there a way to get an existing personal access token via python ? Either through and sdk or a rest endpoint ? Or is the only way to do that to store the PAT in a key vault and retrieve it via a secret scope ? Thank you !
- 2683 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Ajay-Pandey Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...
- 2 kudos
- 2022 Views
- 3 replies
- 2 kudos
KB Feedback Discussion In addition to the Databricks Community, we have a Support team that maintains a Knowledge Base (KB). The KB contains answers t...
KB Feedback DiscussionIn addition to the Databricks Community, we have a Support team that maintains a Knowledge Base (KB). The KB contains answers to common questions about Databricks, as well as information on optimisation and troubleshooting.These...
- 2022 Views
- 3 replies
- 2 kudos
- 1235 Views
- 2 replies
- 1 kudos
Python error on install epmwebapi library
The error below occurred when trying to install the mentioned library. how do i fix this
- 1235 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @VitorGhiotti, I am able to install this package, can you share your cluster configuration and are you using a private endpoint?
- 1 kudos
- 905 Views
- 1 replies
- 1 kudos
Yes, still illegal. And I also don’t understand why it is equated with drugs, but alcohol is not! Not a single murder has yet been committed under can...
Yes, still illegal. And I also don’t understand why it is equated with drugs, but alcohol is not! Not a single murder has yet been committed under cannabis, not a single war has been unleashed. It's just that people who don't use don't understand how...
- 905 Views
- 1 replies
- 1 kudos
- 1 kudos
You are absolutely right! I have found it to be a big relief medically. I have nerve conditions which is not operable. The legal medical pills almost literally killed me, and if it wasn't for my husband's quick thinking, I wouldn't be here to share t...
- 1 kudos
- 3352 Views
- 4 replies
- 4 kudos
Data preparation in Databricks
Data preparation in Databricks Good data is important to ensure accurate and useful results. To get good data following tasks must be done Cleaning and formatting data - Handling missing values or outliers, ensuring data is in the correct format, and...
- 3352 Views
- 4 replies
- 4 kudos
- 4 kudos
Data governance and data lineage are other things to call out.Here's a cheat sheet that is also useful -> Data Preparation Cheatsheet
- 4 kudos
- 745 Views
- 1 replies
- 0 kudos
Badge not received for Databricks Certified Data Engineer Associate
Hello,I passed the certification but haven't received a badge. In fact, I created my databricks academy account only after completing the test. Could you please ensure I do receive that certification? Thanks
- 745 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @gpierard Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 0 kudos
- 7128 Views
- 1 replies
- 0 kudos
Migration scripts for distribution, embedded in library
I'm working on a python package that can be installed via pip. The package will manage a delta table for the user, and new versions of the package may need to run migrations on this tableIs this an okay format to use?def migrate(table_path): mm_p...
- 7128 Views
- 1 replies
- 0 kudos
- 0 kudos
Not much community happening here
- 0 kudos