cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

jonathan-dufaul
by Valued Contributor
  • 3545 Views
  • 5 replies
  • 5 kudos

Does FeatureStoreClient().score_batch support multidimentional predictions?

I have a pyfunc model that I can use to get predictions. It takes time series data with context information at each date, and produces a string of predictions. For example:The data is set up like below (temp/pressure/output are different than my inpu...

  • 3545 Views
  • 5 replies
  • 5 kudos
Latest Reply
EmilAndersson
New Contributor II
  • 5 kudos

I have the same question. I've decided to look for alternative Feature Stores as this makes it very difficult to use for time series forecasting.

  • 5 kudos
4 More Replies
vayunandan_tupa
by New Contributor
  • 2532 Views
  • 3 replies
  • 0 kudos
  • 2532 Views
  • 3 replies
  • 0 kudos
Latest Reply
dplante
Contributor II
  • 0 kudos

You can take all the Databricks exams as many times as you want, but you have to pay a fee each time you take the exam.

  • 0 kudos
2 More Replies
sridhar0109
by New Contributor
  • 1100 Views
  • 2 replies
  • 0 kudos

Tracking changes in data distribution by using pyspark

Hi All,I'm working on creating a data quality dashboard. I've created few rules like checking nulls in a column, checking for data type of the column , removing duplicates etc.We follow medallion architecture and are applying these data quality check...

  • 1100 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Sridhar Varanasi​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...

  • 0 kudos
1 More Replies
Santhanalakshmi
by New Contributor II
  • 2734 Views
  • 3 replies
  • 0 kudos

Throwing IndexoutofBound Exception in Pyspark

Hello All,I am trying to read the data and trying to group the data in order to pass it to predict function via @F.pandas_udf method.#Loading Model pkl_model = pickle.load(open(filepath,'rb'))   # build schema for output labels filter_schema=[] ...

error_db error_2_db error_3_db
  • 2734 Views
  • 3 replies
  • 0 kudos
Latest Reply
Vindhya
New Contributor II
  • 0 kudos

@Santhanalakshmi Manoharan​  Was this issue resolved, Am also getting same error, any guidance would be of great help.Appreciate your help.

  • 0 kudos
2 More Replies
Orianh
by Valued Contributor II
  • 1364 Views
  • 2 replies
  • 0 kudos

TF SummaryWriter flush() don't send any buffered data to storage.

Hey guys, I'm training a TF model in databricks, and logging to tensorboard using SummaryWriter. At the end of each epoch SummaryWriter.flush() is called which should send any buffered data into storage. But i can't see the tensorboard files while th...

  • 1364 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @orian hindi​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...

  • 0 kudos
1 More Replies
Kaan
by New Contributor
  • 2339 Views
  • 1 replies
  • 1 kudos

Resolved! Using databricks in multi-cloud, and querying data from the same instance.

I'm looking for a good product to use across two clouds at once for Data Engineering, Data modeling and governance. I currently have a GCP platform, but most of my data and future data goes through Azure, and currently is then transfered to GCS/BQ.Cu...

  • 2339 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Karl Andrén​ :Databricks is a great option for data engineering, data modeling, and governance across multiple clouds. It supports integrations with multiple cloud providers, including Azure, AWS, and GCP, and provides a unified interface to access ...

  • 1 kudos
Sujitha
by Databricks Employee
  • 722 Views
  • 0 replies
  • 4 kudos

Hello Databricks Community!  We are getting really excited about the upcoming event of the year Data & AI Summit 2023! The world’s largest data, a...

Hello Databricks Community! We are getting really excited about the upcoming event of the year Data & AI Summit 2023!The world’s largest data, analytics and AI conference returns live, to San Francisco and virtually. Four days (June 26–29, 2023) pack...

  • 722 Views
  • 0 replies
  • 4 kudos
NhatHoang
by Valued Contributor II
  • 5763 Views
  • 3 replies
  • 15 kudos

Resolved! Do One-Hot-Encoding (OHE) before or after split data to train and test dataframe

Hi,I wonder that I should do OHE before or after I split data to build up a ML model.Please give some advise.

  • 5763 Views
  • 3 replies
  • 15 kudos
Latest Reply
LandanG
Databricks Employee
  • 15 kudos

Hi @Nhat Hoang​ ,While not Databricks-specific, here's a good answer:"If you perform the encoding before the split, it will lead to data leakage (train-test contamination). In this sense, you will introduce new data (integers of Label Encoders) and u...

  • 15 kudos
2 More Replies
Labels