- 8405 Views
- 1 replies
- 0 kudos
Optimizing for Recall in Azure AutoML UI
Hi all, I've been using Azure AutoML and noticed that I can choose 'recall' as my optimization metric in the notebook but not in the Azure AutoML UI. The Databricks documentation also doesn't list 'recall' as an optimization metric.Is there a reason ...
- 8405 Views
- 1 replies
- 0 kudos
- 0 kudos
On the databricks notebook itself, I can see that databricks.automl supports using recall as a primary metric Help on function classify in module databricks.automl: :param primary_metric: primary metric to select the best model. Each trial will...
- 0 kudos
- 8584 Views
- 6 replies
- 7 kudos
How to save model produce by distributed training?
I am trying to save model after distributed training via the following codeimport sys from spark_tensorflow_distributor import MirroredStrategyRunner import mlflow.keras mlflow.keras.autolog() mlflow.log_param("learning_rate", 0.001) import...
- 8584 Views
- 6 replies
- 7 kudos
- 7 kudos
I think I finally worked this out.Here is the extra code to save out the model only once and from the 1st node:context = pyspark.BarrierTaskContext.get() if context.partitionId() == 0: mlflow.keras.log_model(model, "mymodel")
- 7 kudos
- 3228 Views
- 0 replies
- 0 kudos
'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'Too many sources. It cannot be more than 100'
I am getting the following error while saving a delta table in the feature storeWARNING databricks.feature_store._catalog_client_helper: Failed to record data sources in the catalog. Exception: {'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'To...
- 3228 Views
- 0 replies
- 0 kudos
- 4408 Views
- 2 replies
- 1 kudos
AutoMl Dataset too large
Hello community,i have the following problem: I am using automl to solve a regression model, but in the preprocessing my dataset is sampled to ~30% of the original amount.I am using runtime 14.2 ML Driver: Standard_DS4_v2 28GB Memory 8 coresWorker: S...
- 4408 Views
- 2 replies
- 1 kudos
- 1 kudos
I am pretty sure that i know what the problem was. I had a timestamp column (with second precision) as a feature. If they get one hot encoded, the dataset can get pretty large.
- 1 kudos
- 3488 Views
- 2 replies
- 0 kudos
Error: batch scoring with mlflow.keras flavor model
I am logging a trained keras model using the following: fe.log_model( model=model, artifact_path="wine_quality_prediction", flavor= mlflow.keras, training_set=training_set, registered_model_name=model_name )And when I call the following:predictions_...
- 3488 Views
- 2 replies
- 0 kudos
- 1898 Views
- 1 replies
- 0 kudos
BAD_REQUEST: ExperimentIds cannot be empty when checking ACLs in bulk
I encountered the error when using Databricks CE to log experiments from mlflow. It worked perfectly fine before, but now I cannot open any of my experiments. I tried clean the cookies, change the browser, and create a new account to manually create ...
- 1898 Views
- 1 replies
- 0 kudos
- 3761 Views
- 0 replies
- 0 kudos
MLFlow connection pool warning
Hi,I have a transformer model from Hugging Face I have logged to MLFlow.When I load in using mlflow.transformers.load_model I receive a bunch of warnings: WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: xxxx. Connection...
- 3761 Views
- 0 replies
- 0 kudos
- 1536 Views
- 0 replies
- 0 kudos
ApplyInPandas failing at a particular grouped item
Hello,I have a code that performs a forecast for 21k items in parallel. It looks like this: def forward_forecast(data): model = ETSModel(window_data, error='add', trend='add', seasonal=None) fitted_model = model.fit(disp=0) ...
- 1536 Views
- 0 replies
- 0 kudos
- 3612 Views
- 1 replies
- 0 kudos
Feature Store Log Model and Score Batch - env_manager
Hi Everyone. I have a couple of questions about the feature store log model and score batch. After you log a model with the feature store then use fs.score_batch is it possible to pass the env_manager to predict with the same env as training as descr...
- 3612 Views
- 1 replies
- 0 kudos
- 0 kudos
I also like to know if that works.
- 0 kudos
- 3172 Views
- 2 replies
- 1 kudos
Model Lineage with Feature Engineering is missing tables and notebooks
I am trying to track the lineage of model and tables using the FeatureEngineeringClient. The table lineage shows the relevant tables and notebooks but the model lineage shows only the model. No notebook and tables. here is my code fe = FeatureEngine...
- 3172 Views
- 2 replies
- 1 kudos
- 1 kudos
ok I realized something else. That although I used FeatureEngineeringCient, MLflow model artifact suggest I used FeatureStoreClient. Please see attachment.
- 1 kudos
- 14520 Views
- 6 replies
- 6 kudos
Displaying graphviz images in a notebook
Hi,I'm experimenting with process mining in a Databricks notebook using the OSS library PM4PY. I've been working through some tutorials and the notebook they provide on Github:https://github.com/pm4py/pm4py-core/blob/release/notebooks/3_process_disco...
- 14520 Views
- 6 replies
- 6 kudos
- 6 kudos
@Rob_S i am also in the same situation the code cell executes but no visualization how did you tackle this problem?
- 6 kudos
- 5985 Views
- 1 replies
- 0 kudos
parallel execution with applyinpandas on partitioned table
Hi,I have a job that uses df.groupby(“Country”).applyInPandas(..) to run pandas-based hyperparameter tuning in parallel for 6 countries.It runs on a cluster with 4 workers (Chosen like this because the countries’ datasets are of different sizes – so ...
- 5985 Views
- 1 replies
- 0 kudos
- 10480 Views
- 5 replies
- 2 kudos
Serving API endpoint failing
Hi Team,I registered my ML model in databricks but while trying to serve an API endpoint for the model it is failing with the following error logs.Service logs: There are currently no replicas in a running state.Build logs :Build never started - chec...
- 10480 Views
- 5 replies
- 2 kudos
- 2 kudos
@ombhuyan We currently only upload logs during the build phase to the user (i.e where we install the pip dependencies) but we don't upload logs during the pre-build phase (i.e where we download the model). That's why you may not see clear error messa...
- 2 kudos
- 14509 Views
- 6 replies
- 3 kudos
Databricks Notebook Rendering Issue: IPython.lib.display.IFrame
Similar issue here: https://stackoverflow.com/questions/71336374/randomforestclassifier-explainer-dashboard-output-in-databricks-notebook-is-notActual output – Databricks Notebook Expected Output – Jupyter Notebook Reproducible Code Example#pip insta...
- 14509 Views
- 6 replies
- 3 kudos
- 3 kudos
Hi Abhishek, I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster?
- 3 kudos
- 3171 Views
- 1 replies
- 0 kudos
Resolved! Github Datasets/Labs for Large Language Models: Application through Production is not working
I've signed up for the module for certification on Large Language Models: Application through Production.Follow the Github instructions and install the notebooks provided.Unfortunately none of the workbooks are working due to the- Badly setup file pa...
- 3171 Views
- 1 replies
- 0 kudos
- 0 kudos
No further instructions on the Read-me here: https://github.com/databricks-academy/large-language-models/tree/publishedFollowed all the setup steps, but the file paths in /include are not working fine.Why does not Databricks provide the direct links ...
- 0 kudos
