cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Sujitha
by Databricks Employee
  • 786 Views
  • 0 replies
  • 4 kudos

Hello Databricks Community!  We are getting really excited about the upcoming event of the year Data & AI Summit 2023! The world’s largest data, a...

Hello Databricks Community! We are getting really excited about the upcoming event of the year Data & AI Summit 2023!The world’s largest data, analytics and AI conference returns live, to San Francisco and virtually. Four days (June 26–29, 2023) pack...

  • 786 Views
  • 0 replies
  • 4 kudos
Pritam
by New Contributor II
  • 980 Views
  • 0 replies
  • 1 kudos

Not able to create jobs via jobs API in databricks

I am not able to create jobs via jobs API in databricks.Error=INVALID_PARAMETER_VALUE: Job settings must be specified.I simply copied the JSON file and saved it. Loaded the same JSON file and tried to create the job via API but the got the above erro...

  • 980 Views
  • 0 replies
  • 1 kudos
User16752245312
by Databricks Employee
  • 3720 Views
  • 5 replies
  • 2 kudos

When running structured streaming jobs in production, what are the general best practices to reduce cost?

Consider a basic structured streaming use case of aggregating the data, perform some basic data cleaning transformation, and merge into a historical aggregate dataset.

  • 3720 Views
  • 5 replies
  • 2 kudos
Latest Reply
lawrence009
Contributor
  • 2 kudos

I second the recommendations: auto load with trigger, batch processing instead of continuous streaming where use case permits. In addition, test with a small batch firstfavor fewer larger workers over more smaller workersadjust your job cluster over...

  • 2 kudos
4 More Replies
lawrence009
by Contributor
  • 1419 Views
  • 2 replies
  • 3 kudos

Streaming Source for Feature Store (and outputMode)

To save computing resource and time, can I use streaming source in a batch mode (similar to Auto Loader) to update my feature store as my source table receives row update or is appended with new rows?

  • 1419 Views
  • 2 replies
  • 3 kudos
Latest Reply
Meghala
Valued Contributor II
  • 3 kudos

yes you can schedule the job to process the data with auto loader

  • 3 kudos
1 More Replies
Aviral-Bhardwaj
by Esteemed Contributor III
  • 5915 Views
  • 2 replies
  • 36 kudos

Delta lake Vs Data lake in Databricks Delta Lake is an open-source storage layer that sits on top of existing data lake storage, such as Azure Data La...

Delta lake Vs Data lake in DatabricksDelta Lake is an open-source storage layer that sits on top of existing data lake storage, such as Azure Data Lake Store or Amazon S3. It provides a more robust and scalable alternative to traditional data lake st...

  • 5915 Views
  • 2 replies
  • 36 kudos
Latest Reply
Meghala
Valued Contributor II
  • 36 kudos

this data is very much informative and i understood much in it so thank you @Aviral Bhardwaj​ sir

  • 36 kudos
1 More Replies
ptawil
by New Contributor III
  • 1877 Views
  • 2 replies
  • 2 kudos

Model Serving Status Failed

I'm trying to enable serving for my model but I keep getting Pending into Failed Status.Here are the model event logs.2022-11-15 15:43:13ENDPOINT_UPDATEDFailed to create model 3 times2022-11-15 15:43:03ENDPOINT_UPDATEDFailed to create cluster 3 times...

  • 1877 Views
  • 2 replies
  • 2 kudos
Latest Reply
171499
New Contributor III
  • 2 kudos

Any update on this? I'm running into the same issue

  • 2 kudos
1 More Replies
GabrieleMuciacc
by New Contributor III
  • 9533 Views
  • 10 replies
  • 12 kudos

Resolved! Delete feature tables through the Python API

The documentation explains how to delete feature tables through the UI. Is it possible to do the same using the Python FeatureStoreClient? I cannot find anything in the docs: https://docs.databricks.com/_static/documents/feature-store-python-api-refe...

  • 9533 Views
  • 10 replies
  • 12 kudos
Latest Reply
User16788316720
New Contributor III
  • 12 kudos

from databricks import feature_store fs = feature_store.FeatureStoreClient() fs.drop_table(FEATURE_TABLE_NAME)As of Databricks Runtime 10.5 for ML. Docs

  • 12 kudos
9 More Replies
parthsalvi
by Contributor
  • 2849 Views
  • 1 replies
  • 2 kudos

py4j.security.Py4JSecurityException: Constructor public com.databricks.backend.daemon.dbutils.FSUtilsParallel(org.apache.spark.SparkContext) is not whitelisted. - DBR 10.4 LTS Shared mode

We're trying to update permission in Unity Catalog using spark sql statetment spark.sql("ALTER TABLE %s.%s OWNER TO `%s`;" % (raw_schema, table_name, 'Metastore_Admins'))But we're running into following error We're facing issues like these in DBR 10....

image.png
  • 2849 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

I have seen this beforeso there is one file in the DBFS where we have to write the whitelisted file , so you have to find out where is your whitelisted file location after that you have to add that "com.databricks.....parallel" line in that file and ...

  • 2 kudos
isaac_gritz
by Databricks Employee
  • 2086 Views
  • 1 replies
  • 4 kudos

Databricks MLOps Best Practices

Where to find the best practices on MLOps on DatabricksWe recommend checking out the Big Book of MLOps for detailed guidance on MLOps best practices on Databricks including reference architectures.For a deep dive on the Databricks Feature store, we r...

  • 2086 Views
  • 1 replies
  • 4 kudos
Latest Reply
sher
Valued Contributor II
  • 4 kudos

you can check here https://docs.databricks.com/machine-learning/mlops/mlops-workflow.html

  • 4 kudos
Sujitha
by Databricks Employee
  • 1260 Views
  • 2 replies
  • 6 kudos

Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week. Databricks platform release notes December ...

Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week.Databricks platform release notes December 5 - 16, 2022Databricks JDBC driver 2.6.32Version 2.6.32 of the Databricks JDBC driver (download and M...

  • 1260 Views
  • 2 replies
  • 6 kudos
Latest Reply
Harun
Honored Contributor
  • 6 kudos

Thanks for sharing @Sujitha Ramamoorthy​ 

  • 6 kudos
1 More Replies
Geeya
by New Contributor II
  • 975 Views
  • 0 replies
  • 1 kudos

parallel run in job pipeline

I am trying to build a pipeline which deploys a ML model, and I want to build the pipeline in Workflow/jobs. In task of prediction of the model, I have hundreds of groups of input features, I use a for loop to get one group of input features and do ...

  • 975 Views
  • 0 replies
  • 1 kudos
PNegro
by New Contributor III
  • 3723 Views
  • 4 replies
  • 4 kudos

conda-env: error: unrecognized arguments: 'virtualenv': 'python_env.yaml'

I have registered an experiment as model in the model registry and when I start serving the model I get the following error:usage: conda-env [-h] {create,export,list,remove,update,config} ...conda-env: error: unrecognized arguments: 'virtualenv': 'py...

  • 3723 Views
  • 4 replies
  • 4 kudos
Latest Reply
PNegro
New Contributor III
  • 4 kudos

Hi Follks, Is there any new on this?.What should I do?ThanksBestPablo

  • 4 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels