cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

grazie
by Contributor
  • 1192 Views
  • 3 replies
  • 2 kudos

Do you need to be workspace admin to create jobs?

We're using a setup where we use gitlab ci to deploy workflows using a service principal, using the Jobs API (2.1) https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsCreateWhen we wanted to reduce permissions of the ci to minimu...

  • 1192 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Geir Iversen​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 2 kudos
2 More Replies
doalmeida
by New Contributor
  • 2006 Views
  • 2 replies
  • 0 kudos

Endpoint not found when creating automl experiment through API and UI

Hi everyone, I'm getting this exact error as shown bellow, when trying to create an automl experiment. This happens both through the UI and the API with my code or with databrick's example code. I've tried looking into this but had no luck finding an...

image
  • 2006 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Diogo Almeida​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
1 More Replies
iwan_aucamp
by New Contributor III
  • 761 Views
  • 2 replies
  • 1 kudos

Account SCIM API OpenAPI specification issues

I'm trying to get a list of all users, groups and service principals on Azure from a python script. As I understand things I should be using the Account SCIM API for this. According to the azure documentation [ref], the OpenAPI specification for this...

  • 761 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Iwan Aucamp​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 1 kudos
1 More Replies
KayCon86
by New Contributor
  • 1438 Views
  • 3 replies
  • 0 kudos

Creating a Api links by url & list from a saved df

I have 106,000 + api's I need to call, so instead of calling them one by one I would like to create a loop as I have the list of location Id's which I've called from there api locations list and these will sit at the end of the url to get more info o...

  • 1438 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Kay Connolly​ :It looks like you are trying to concatenate a string with a column object, which is causing the error. You need to convert the column object to a string first before concatenating it to the URL. Here's a modified code snippet that sho...

  • 0 kudos
2 More Replies
iwan_aucamp
by New Contributor III
  • 910 Views
  • 2 replies
  • 2 kudos

Are there any python API bindings for the Databricks Account API?

All I could find in terms of API bindings for python is https://pypi.org/project/databricks-cli/, and this does not include the Account API and it is also not official.I will just use the OpenAPI spec, but just want to be sure I'm not doing unnecessa...

  • 910 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Iwan Aucamp​ : Yes, there are Python API bindings available for the Databricks Account API.For Databricks Account API with Python, please refer to the Databricks documentation: https://docs.databricks.com/dev-tools/api/latest/accounts.html#python-ap...

  • 2 kudos
1 More Replies
VictoriaM
by New Contributor II
  • 738 Views
  • 2 replies
  • 0 kudos

@Chris Grabiel​  Do you have any experience connecting REDCAP API to Databricks you would be able to share?

@Chris Grabiel​  Do you have any experience connecting REDCAP API to Databricks you would be able to share?

  • 738 Views
  • 2 replies
  • 0 kudos
Latest Reply
Chris_Grabiel
New Contributor III
  • 0 kudos

We absolutely do. We ingest to the lake via Redcap API AND folks use it in notebooks. How can we help?

  • 0 kudos
1 More Replies
lasmali
by New Contributor II
  • 1249 Views
  • 3 replies
  • 0 kudos

Instance Profile creation via the Databricks REST API returns "INVALID_PARAMETER_VALUE"

# Problem StatementWe have a need to create Instance Profiles via the Databricks REST API, but the endpoint returns an "INVALID_PARAMETER_VALUE" error with "Syntactically invalid AWS instance profile ARN" message even when provided an appropriate ARN...

  • 1249 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @lasse Lidegaard​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 0 kudos
2 More Replies
whh99
by New Contributor II
  • 1072 Views
  • 3 replies
  • 1 kudos

Given user id, what API can we use to find out which cluster the user is connected to?

I want to know the cluster that user is connected to in databricks. It would be great if we can also get the duration that the user is connected.

  • 1072 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Hui Hui Wong​  (Customer)​, We haven’t heard from you since the last response from @Daniel Sahal​ (Customer)​ , and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as...

  • 1 kudos
2 More Replies
hanish
by New Contributor II
  • 1542 Views
  • 3 replies
  • 2 kudos

Job cluster support in jobs/runs/submit API

We are using jobs/runs/submit API of databricks to create and trigger a one-time run with new_cluster and existing_cluster configuration. We would like to check if there is provision to pass "job_clusters" in this API to reuse the same cluster across...

  • 1542 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Hanish Bansal​ Shared job cluster for  jobs/runs/submit API is not supported at the moment.

  • 2 kudos
2 More Replies
RRO
by Contributor
  • 1026 Views
  • 1 replies
  • 3 kudos

AutoML forecasting with monthly data?

ARIMA and FBProphet have the capability to forecast monthly data. When using AutoML (via the API or the UI) it seems like it is not possible to have a monthly freq (e.g. 'MS').Is there a way / workaround to make it work with monthly data or is it pla...

  • 1026 Views
  • 1 replies
  • 3 kudos
Latest Reply
MateuszLomanski
New Contributor II
  • 3 kudos

It is possible to use AutoML to forecast monthly data, but it may require some additional steps or adjustments.One approach is to resample the monthly data to a lower frequency such as weekly or daily, and then use AutoML to forecast at that lower fr...

  • 3 kudos
sfalquier
by New Contributor II
  • 1352 Views
  • 3 replies
  • 0 kudos

HTTP 403 on git-credentials API

Hi,I am trying to set git credentials for my service principal. I follow the process described here but I get a 403 error when making the POST request to ${DATABRICKS_HOST}/api/2.0/git-credentials with service principal token.By the way, I also canno...

  • 1352 Views
  • 3 replies
  • 0 kudos
Latest Reply
Vivian_Wilfred
Honored Contributor
  • 0 kudos

Hi @Sébastien FALQUIER​ it works for me, there are no restrictions. Maybe the PAT token you generated for the service principle got expired. Can you generate a new token and try to run GET/git-credentials API?How are you creating PAT for service prin...

  • 0 kudos
2 More Replies
architect
by New Contributor
  • 996 Views
  • 1 replies
  • 0 kudos

Does Databricks provide a mechanism to have rate limiting for receivers?

from pyspark.sql import SparkSession   scala_version = '2.12' spark_version = '3.3.0'   packages = [ f'org.apache.spark:spark-sql-kafka-0-10_{scala_version}:{spark_version}', 'org.apache.kafka:kafka-clients:3.2.1' ]   spark = SparkSession.bui...

  • 996 Views
  • 1 replies
  • 0 kudos
Latest Reply
Rajani
Contributor
  • 0 kudos

hi @Software Architect​  i dont think so

  • 0 kudos
AnubhavG
by Contributor
  • 1448 Views
  • 1 replies
  • 2 kudos

External APIs

Does databricks provide a way to integrate to external sw/API's? Whether it is in the form of UDF/external function? Can somebody point me how this can be achieved? My use case is to talk to external API's from databricks to perform certain operation...

  • 1448 Views
  • 1 replies
  • 2 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 2 kudos

You can write your own code to fetch data from external API.Example: https://insightsndata.com/how-to-call-rest-api-store-data-in-databricks-8383f2458d7d

  • 2 kudos
JamesKuo
by New Contributor III
  • 3367 Views
  • 3 replies
  • 6 kudos

Resolved! Where can I find API documentation to dbutils.notebook.entry_point?

dbutils.notebook.help only lists "run" and "exit" methods. I could only find references to dbutils.notebook.entry_point spread across the web but there does not seem to be an official Databricks API documentation to its complete APIs anywhere. Can so...

  • 3367 Views
  • 3 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @James kuo​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 6 kudos
2 More Replies
tum
by New Contributor II
  • 2188 Views
  • 3 replies
  • 4 kudos

Create new job api error "MALFORMED_REQUEST"

hi,i'm trying to test create a new job api (v 2.1) with python, but i got error:{ 'error_code': 'MALFORMED_REQUEST', 'message': 'Invalid JSON given in the body of the request - expected a map'}How do i validate json body before posting ?this is my js...

  • 2188 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @tum m​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 4 kudos
2 More Replies
Labels