- 398 Views
- 3 replies
- 2 kudos
Resolved! Specify cluster when using dbutils.notebook API
Hello! Anyone know if there is a way of specifying cluster when using dbutils.notebook.run() ? As i understand it, this command will create a job compute on the run? but what if I want to use my general purpose cluster? I have been thinking of %run b...
- 398 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @ErikJ ,Check the docs:Also you may run the help method:.The parameters of the method are:1. path2. timeoutSeconds3. arguments
- 2 kudos
- 360 Views
- 2 replies
- 0 kudos
Lake house monitoring prodile_metrics and drift_metrics are empty
I am wondering why when i create a lakehouse monitor using either the API or UI, that my profile_metrics and data_drift tables are empty. This error came after i changed my date from type date to timestamp. Any information would be greatly appreciate...
- 360 Views
- 2 replies
- 0 kudos
- 0 kudos
The issue comes where you don't include data with a timestamp that's within the last 30 days, as the analysis done on the table is only done within the last 30 days worth of data.
- 0 kudos
- 3355 Views
- 4 replies
- 2 kudos
Cluster compute metrics
I want to fetch compute metrics(hardware, gpu and spark) and use them in certain dashboard on Databricks, however i'm not able to fetch them. i have tried GET API request and system tables. The system tables only have CPU utilization and memory utili...
- 3355 Views
- 4 replies
- 2 kudos
- 93 Views
- 0 replies
- 0 kudos
deleteing old databricks workspace but keeping the data
Hi! I am currently working on a old databricks workspace where we have stored a lot of data, we have migrated over to a new workspace and i want to delete the old one. Before deleting the workspace i want to store the data in a blob storage in azure....
- 93 Views
- 0 replies
- 0 kudos
- 3449 Views
- 9 replies
- 2 kudos
Create a Workflow Schedule with varying Parameters
We aim to reduce the amount of notebooks we create to a minimum and instead make these fairly flexible. Therefore we have a Factory setup that takes in a parameter to varies the logic.However when it comes to Workflows we are forced to create multipl...
- 3449 Views
- 9 replies
- 2 kudos
- 2 kudos
We're also running into this issue on my team where having multiple cron schedules would be handy. We have some pipelines that we want run on multiple schedules, say to refresh data "Run every Sunday at midnight" and "Run on the first day of the mont...
- 2 kudos
- 253 Views
- 0 replies
- 0 kudos
Unable to login to community addition account
Hello, I created a community edition account yesterday. Earlier today, I was still able to sign in and everything worked as expected. Now, whenever I try to sign in, I get the following error: "We were not able to find a Community Edition workspace w...
- 253 Views
- 0 replies
- 0 kudos
- 217 Views
- 0 replies
- 1 kudos
Accessing data in Databricks/Unity Catalog from a SharePoint list
Our organization is moving all of it's corporate master data to Databricks. Our business operations units need access to this data in various tools in the form of look-ups or drop-down lists in tools like Excel and SharePoint lists. It's simple eno...
- 217 Views
- 0 replies
- 1 kudos
- 3886 Views
- 7 replies
- 1 kudos
log signature and input data for Spark LinearRegression
I am looking for a way to log my `pyspark.ml.regression.LinearRegression` model with input and signature ata. The usual example that I found around are using sklearn and they can simply do # Log the model with signature and input example signature =...
- 3886 Views
- 7 replies
- 1 kudos
- 1 kudos
@MohsenJ @javierbg @Abi105 I have found a solution to this issue as I was trying to deploy Spark ML Models to Unity Catalog. Please view my blog and let me know if it helps solve your issues! https://medium.com/p/7d04e8539540
- 1 kudos
- 489 Views
- 1 replies
- 0 kudos
FAILED_READ_FILE.NO_HINT error
We read data from csv in the volume into the table using COPY INTO. The first 200 files were added without problems, but now we are no longer able to add any new data to the table and the error is FAILED_READ_FILE.NO_HINT. The CSV format is always th...
- 489 Views
- 1 replies
- 0 kudos
- 0 kudos
Py4JJavaError: An error occurred while calling o392.sql. : org.apache.spark.SparkException: [FAILED_READ_FILE.NO_HINT] Error while reading file dbfs:/Volumes/...txt. SQLSTATE: KD001 at org.apache.spark.sql.errors.QueryExecutionErrors$.cannotReadFiles...
- 0 kudos
- 129 Views
- 0 replies
- 0 kudos
Account Level API GET Call requiring User-Agent Header
We recently Enabled Unity Catalog so using Account Level API and little stuck on a GET call. For some reason, just the GET call requires a User-Agent header while POST, PUT, PATCH.. all work without it. For workspace API, there was no need for User-A...
- 129 Views
- 0 replies
- 0 kudos
- 633 Views
- 1 replies
- 1 kudos
Summit23
I have really enjoyed the summit so far! I am a biostatistics graduate student at Harvard and am learning so much here.
- 633 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi,If you had a great time at the Summit, be sure to visit our community events portal for upcoming events and join your regional user group for meetups.Thanks,Anushree
- 1 kudos
- 794 Views
- 1 replies
- 0 kudos
Summit 2024
It has been an exciting event that brought together industry leaders, researchers, and practitioners to explore the latest advancements in artificial intelligence and data science. With cutting-edge presentations, hands-on workshops, and networking o...
- 794 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,If you had a great time at the Summit, be sure to visit our community events portal for upcoming events and join your regional user group for meetups.Thanks,Anushree
- 0 kudos
- 1479 Views
- 1 replies
- 0 kudos
- 1479 Views
- 1 replies
- 0 kudos
- 0 kudos
You can leverage this code base. It works as expected using "next_page_token" parameter-Don't forget to mark this solution as correct if this helped you import requests token = 'your token' url = 'your URL' params = {'expand_tasks': 'true'} heade...
- 0 kudos
- 465 Views
- 1 replies
- 0 kudos
Unity Catalog
Can MLflow Experiments be incorporated into Unity Catalog (may be in volume or somewhere in unity catalog) similar to models and feature tables?
- 465 Views
- 1 replies
- 0 kudos
- 0 kudos
As of now Databricks supports DBFS, S3, and Azure Blob storage artifact locations. Our teams are working on intruducing Volumes as repository which might be coming soon
- 0 kudos
- 5232 Views
- 13 replies
- 3 kudos
Resolved! Unable to access Account console under Azure Databricks
I am the only user for the Azure enviornment and assigned global admin.For the resource group, I am the owner.For the Azure Databricks Service I am the service admin already.I would like to try to go into the Account console to make some changes for ...
- 5232 Views
- 13 replies
- 3 kudos
