- 1028 Views
- 1 replies
- 0 kudos
How can I add custom models to Time Series AutoML?
Time Series AutoML currently has very few models for comparison. How can I add some custom models into the mix so that they are compared each time?
- 1028 Views
- 1 replies
- 0 kudos
- 0 kudos
could you please let us know what are the custom models you are looking to add to time series AutoML?
- 0 kudos
- 2972 Views
- 3 replies
- 5 kudos
Resolved! What is the use case of having Azure Synapse(DWH) and Delta Lake ( Gold) given we can connect BI to delta directly
The curated zone is pushed to cloud data warehouse such as Synapse Dedicated SQL Pools which then acts as a serving layer for BI tools and analyst.I believe we can have models in gold layer and have BI connect to this layer or we can have serverless ...
- 2972 Views
- 3 replies
- 5 kudos
- 5 kudos
Thank you, so for a large workload, where we need lot of optimization we might need Synapse, but for a small/medium workload, we might have to stick to Delta Table
- 5 kudos
- 5686 Views
- 3 replies
- 2 kudos
Resolved! How to extract name of Primary Key Columns of feature store table in Databricks environment ?
Hello,Please suggest how to obtain name of primary key columns in my feature store table in hive metastore.'describe' gives me the name of the columns but not the indexesThanks in advance for your help.Regards,
- 5686 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi Vidula,Yes, I solved the query !I used the function getTable of class FeatureStoreClient(). You have all datas you need : primary keys, timestamp_keys, features of feature store table.Regards
- 2 kudos
- 3857 Views
- 2 replies
- 0 kudos
Can't overwrite to S3 object
We are trying to write the data frame to s3 using: df.write.mode('overwrite').save("s3://BUCKET-NAME/temp"), but recently we are getting the following error: 'com.amazonaws.services.s3.model.MultiObjectDeleteException: One or more objects could not ...
- 3857 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Mayank Kasturia​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 0 kudos
- 1412 Views
- 1 replies
- 0 kudos
Sending R functions to worker nodes
Hi!If I need to use many workers to distributes regular pandas, I would use a pandas_UDF. (having regular python crunching a slice of my data, on each node, and combining all results back to the driver node)Is there something equivalent for R?Thanks,
- 1412 Views
- 1 replies
- 0 kudos
- 1799 Views
- 2 replies
- 1 kudos
MLFlow Python Error for Forecast problem
Hi Team,I am trying to implement automl in python for my timeseries forecast problem.But, I was facing below error during the model training:AttributeError: 'StanModel' object has no attribute 'fit_class'Due to the above error, the experiment failed ...
- 1799 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Prakash Thavamurugan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
- 1 kudos
- 2233 Views
- 3 replies
- 0 kudos
Error in importing feature_store
from databricks import feature_storeI am trying to import feature_store but it is showing this error.ImportError: cannot import name 'feature_store' from 'databricks' (/databricks/python/lib/python3.8/site-packages/databricks/__init__.py)
- 2233 Views
- 3 replies
- 0 kudos
- 0 kudos
Is this issue resolved completely? We are facing the same problem. this might help.
- 0 kudos
- 5945 Views
- 5 replies
- 0 kudos
Error when running job in databricks
Hello, I am very new with databricks and MLflow. I faced with the problem about running job. When the job is run, it usually failed and retried itself, so it incasesed running time, i.e., from normally 6 hrs to 12-18 hrs. From the error log, it shows...
- 5945 Views
- 5 replies
- 0 kudos
- 0 kudos
Hey there @Tanawat Benchasirirot​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hea...
- 0 kudos
- 1692 Views
- 3 replies
- 0 kudos
EOFError trying to assign a model using a custom module
I'm in a Data Science Bootcamp, and the final case study includes data preprocessing (done), using a linear regression model on the data, then porting to SQL for visualization. The model build uses custom python code provided as part of the exercise....
- 1692 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Joe DiGiovanni​ Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
- 0 kudos
- 2425 Views
- 2 replies
- 0 kudos
Error in importing mlflow.sklearn
ImportError: cannot import name '_MIN_SKLEARN_VERSION' from 'mlflow.sklearn.utils' (/databricks/python/lib/python3.8/site-packages/mlflow/sklearn/utils.py)
- 2425 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Sameer Gupta​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 0 kudos
- 1269 Views
- 2 replies
- 0 kudos
How can we automate MLFLOW model serving in databricks?
Can we enable model serving either using cli or any other tools without go to the databricks model UI?
- 1269 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Shawn Feng​ Does @Atanu Sarkar​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
- 7920 Views
- 10 replies
- 0 kudos
Resolved! How to set sparkTrials? I am receiving this TypeError: cannot pickle '_thread.lock' object
I am trying to distribute hyperparameter tuning using hyperopt on a tensorflow.keras model. I am using sparkTrials in my fmin:spark_trials = SparkTrials(parallelism=4)...best_hyperparam = fmin(fn=CNN_HOF, space=space, ...
- 7920 Views
- 10 replies
- 0 kudos
- 0 kudos
This can happen when you try to serialize a keras model with an unserializable layer. What does your model look like? Also what is in that search space variable? What are you trying to optimize on?
- 0 kudos
- 1728 Views
- 3 replies
- 0 kudos
No saved model after stopping the cluster.
I have saved a keras model in some directories in dbfs to load and retrain that with more data, etc. The problem is that when cluster stops and restarts, seems those directories and model are no longer available there and it starts training a new mod...
- 1728 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Vidula Khanna​ I figured it out by replacing OS library module with dbutils utilities. It looks like mre compatible with DBFS.
- 0 kudos
- 5621 Views
- 5 replies
- 5 kudos
Feature table: merge very slow
Hi All, We're just started to look at the feature store capabilities of Databricks. Our first attempt to create a feature table has resulted in very slow write. To avoid the time incurred by the feature functions I generated a dataframe with same...
- 5621 Views
- 5 replies
- 5 kudos
- 5 kudos
Hi @Ashley Betts​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 5 kudos
- 1601 Views
- 2 replies
- 4 kudos
Resolved! Azure Data Factory: allocate resources per Notebook
I'm using Azure Data Factory to create pipeline of Databricks notebooks, something like this:[Notebook 1 - data pre-processing ] -> [Notebook 2 - model training ] -> [Notebook 3 - performance evaluation].Can I write some config file, that would allow...
- 1601 Views
- 2 replies
- 4 kudos
- 4 kudos
I understand that, in your case, auto-scaling will take too much time.The simplest option is to use a different cluster for another notebook (and be sure that the previous cluster is terminated instantly).Another option is to use REST API 2.0/cluster...
- 4 kudos
