cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
akihiko
by • New Contributor III
  • 3205 Views
  • 4 replies
  • 1 kudos

Resolved! Attach notebook to cluster via REST API

Is it possible to attach a notebook to cluster and run it via the REST API?The closest approach I have found is to run a notebook, export the results (HTML!) and import it into the workspace again, but this does not allow us to retain the original ex...

Data Engineering
Reply
  • 3205 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
baert23
New Contributor II
  • 1 kudos

I'm looking for a way to programmatically copy a notebook in Databricks using the workspace/export and workspace/import APIs. Once the notebook is copied, I want to automatically attach it to a specific cluster using its cluster ID. The challenge is ...

  • 1 kudos
3 More Replies
weldermartins
by • Honored Contributor
  • 6711 Views
  • 6 replies
  • 10 kudos

Resolved! Spark - API Jira

Hello guys. I use pyspark in my daily life. A demand has arisen to collect information in Jira. I was able to do this via Talend ESB, but I wouldn't want to use different tools to get the job done. Do you have any example of how to extract data from ...

Data Engineering
Reply
  • 6711 Views
  • 6 replies
  • 10 kudos
View Replies
Latest Reply
Marty73
New Contributor II
  • 10 kudos

Hi,There is also a new Databricks for Jira add-on on the Atlassian Marketplace. It is easy to setup and exports are directly created within Jira. They can be one-time, scheduled, or real-time. It can also export additional Jira data such as Assets, C...

  • 10 kudos
5 More Replies
michael_mehrten
by • New Contributor III
  • 28103 Views
  • 25 replies
  • 14 kudos

Resolved! How to use Databricks Repos with a service principal for CI/CD in Azure DevOps?

Databricks Repos best-practices recommend using the Repos REST API to update a repo via your git provider. The REST API requires authentication, which can be done one of two ways:A user / personal access tokenA service principal access tokenUsing a u...

Data Engineering
Reply
  • 28103 Views
  • 25 replies
  • 14 kudos
View Replies
Latest Reply
martindlarsson
New Contributor III
  • 14 kudos

Having the exact same problem. Did you find a solution @michael_mehrten ?In my case Im using a managed identity so the solution some topics suggest on generating an access token from a Entra ID service principal is not applicable.

  • 14 kudos
24 More Replies
Baldrez
by • New Contributor II
  • 3700 Views
  • 4 replies
  • 5 kudos

Resolved! REST API for Stream Monitoring

Hi, everyone. I just recently started using Databricks on Azure so my question is probably very basic but I am really stuck right now.I need to capture some streaming metrics (number of input rows and their time) so I tried using the Spark Rest Api ...

Data Engineering
Reply
  • 3700 Views
  • 4 replies
  • 5 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 5 kudos

hi @Roberto Baldrez​ ,if you think that @Gaurav Rupnar​ solved your question, then please select it as best response to it can be moved to the top of the topic and it will help more users in the future.Thank you

  • 5 kudos
3 More Replies
missyT
by • New Contributor III
  • 2041 Views
  • 3 replies
  • 4 kudos

Resolved! AI assistant and machine Learning

I am looking to create a basic virtual assistant (AI) that implements machine learning mechanisms.I have some basic knowledge of Python and I have seen some courses on the internet (youtube in particular) that look very interesting.But for the moment...

Data Engineering
Reply
  • 2041 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
valeryuaba
New Contributor III
  • 4 kudos

Hey everyone!I'm clearly excited about this topic since I'm a huge fan of AI assistants and machine learning. MissyT, creating a basic virtual assistant with machine learning capabilities is an excellent idea! With your simple knowledge of Python and...

  • 4 kudos
2 More Replies
Sujitha
by • Databricks Employee
  • 1525 Views
  • 1 replies
  • 2 kudos

Take advantage of the Data + AI Summit Virtual Experience next week! Data + AI Summit is just a few days away! With data professionals from 155+ count...

Take advantage of the Data + AI Summit Virtual Experience next week!Data + AI Summit is just a few days away! With data professionals from 155+ countries already registered, this is truly the premier event for the global data, analytics and AI commun...

Data Engineering
Reply
  • 1525 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Priyag1
Honored Contributor II
  • 2 kudos

Thank you for informing highlights of the upcoming sessions.

  • 2 kudos
leelingmin
by • Databricks Employee
  • 1316 Views
  • 0 replies
  • 1 kudos

dbricks.co

Hi everyone!We’re excited to gather everyone for Data + AI Summit 2023 — the premier AI and LLM global event for the data, analytics and AI community. Join thousands of data engineers, data scientists and data analyst experts virtually from June 29–3...

Data Engineering
Reply
  • 1316 Views
  • 0 replies
  • 1 kudos
jwu1
by • Databricks Employee
  • 1109 Views
  • 1 replies
  • 3 kudos

www.databricks.com

Attention Community! For a limited period, we are offering a generous 50% discount on training at the Data + AI Summit. Simply apply the code FLS4vop5ep during the registration process. Hurry, though, as this offer will expire on June 12, 2023. Don'...

Data Engineering
Reply
  • 1109 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 3 kudos

Thank you for sharing this @Juliet Wu​!!!

  • 3 kudos
Aviral-Bhardwaj
by • Esteemed Contributor III
  • 6734 Views
  • 5 replies
  • 8 kudos

Resolved! MCQ of The Week (Data Engineer Associate Preparation)

A data engineer, User A, has promoted a new pipeline to production by using the REST API to programmatically create several jobs. A DataOps engineer, User B, has configured an external orchestration tool to trigger job runs through the REST API. Both...

Data Engineering
Reply
  • 6734 Views
  • 5 replies
  • 8 kudos
View Replies
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 8 kudos

@Ajay Pandey​  II really appreciate your efforts and you are right in terms of UI, but when we carefully see the question we foundWhich statement describes the contents of the workspace audit logs concerning these events?audit logs are generated and...

  • 8 kudos
4 More Replies
Ismail1
by • New Contributor III
  • 2401 Views
  • 2 replies
  • 3 kudos

Resolved! API Authentication

I am trying to run some API calls to the account console. I tried with every syntax possible encoded and decoded to get the call successfully but it returns a "user not authenticated" error. But when I tried with the Account admin it worked. I need t...

Data Engineering
Reply
  • 2401 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
Ismail1
New Contributor III
  • 3 kudos

Hi Venkat, that sounds like a good idea. Thanks

  • 3 kudos
1 More Replies
Anonymous
by • Not applicable
  • 637 Views
  • 0 replies
  • 2 kudos

community.databricks.com

We're excited to announce the first four winners of our Raffle, and we want to thank everyone who has participated so far. If you haven't yet entered, don't worry! We still have four more tickets to give away for the world's largest Data + AI summit...

Data Engineering
Reply
  • 637 Views
  • 0 replies
  • 2 kudos
Anonymous
by • Not applicable
  • 777 Views
  • 0 replies
  • 1 kudos

 Hello Everyone, I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU​ ​. Please join me in congratu...

 Hello Everyone,I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU​ ​. Please join me in congratulating him on this remarkable achievement!Your dedication and hard work have paid off, and we are del...

Winner4
Data Engineering
Reply
  • 777 Views
  • 0 replies
  • 1 kudos
Anonymous
by • Not applicable
  • 1581 Views
  • 3 replies
  • 2 kudos

 Hello Everyone, I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating ...

 Hello Everyone,I am thrilled to announce that we have our first winner for the raffle contest - @Vincent Slegers​. Please join me in congratulating him on this remarkable achievement!Vincent, your dedication and hard work have paid off, and we are d...

2
Data Engineering
Reply
  • 1581 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
yogu
Honored Contributor III
  • 2 kudos

congrats @Vsleg

  • 2 kudos
2 More Replies
Anonymous
by • Not applicable
  • 937 Views
  • 0 replies
  • 1 kudos

Hi Everyone, As we continue to see the rise of AI, particularly with technologies such as ChatGPT, it's important to consider how this will impact...

Hi Everyone,As we continue to see the rise of AI, particularly with technologies such as ChatGPT, it's important to consider how this will impact the future of our workplaces and long-term career goals. With that in mind, I would love to hear your th...

Data Engineering
Reply
  • 937 Views
  • 0 replies
  • 1 kudos
Labels