- 1027 Views
- 1 replies
- 0 kudos
- 1027 Views
- 1 replies
- 0 kudos
- 0 kudos
Have a look at https://databricks.com/product/managed-mlflow
- 0 kudos
- 1151 Views
- 1 replies
- 0 kudos
Runtime 8.4 ML still showing TF 2.3
I updated my cluster runtime to use 8.4 ML, I did a !pip list and saw that tensorflow was 2.3. What should I do to resolve this? Not sure if I should just do a pip install Tensorflow==2.5.0
- 1151 Views
- 1 replies
- 0 kudos
- 0 kudos
If the answer is the latter, do i need to install the supported versions of cudnn and cuda?
- 0 kudos
- 1207 Views
- 1 replies
- 0 kudos
- 1207 Views
- 1 replies
- 0 kudos
- 0 kudos
I think I need a little more context here on what you're trying to achieve. If you're generally interested in schema evolution, this post talks about feature store: https://databricks.com/blog/2021/05/27/databricks-announces-the-first-feature-store-i...
- 0 kudos
- 913 Views
- 0 replies
- 0 kudos
Deep Learning on Spark within AWS EMR
I'd like to use Deep Learning on Spark within AWS EMR.Is there a best practice or 'recommended' DL framework to run on Spark? It looks like Databricks' spark-deep-learning has been replaced by Horovod—should this the first option to consider? If th...
- 913 Views
- 0 replies
- 0 kudos
- 1453 Views
- 1 replies
- 0 kudos
- 1453 Views
- 1 replies
- 0 kudos
- 0 kudos
I am not aware of any special requirement for this migration, my suggestion to you is to try it on a small scale (one notebook) and observe the results showing in the tracker server, if everything looks OK, then migrate the rest.
- 0 kudos
- 3259 Views
- 1 replies
- 1 kudos
- 3259 Views
- 1 replies
- 1 kudos
- 1 kudos
If you have configured your Structured Streaming query to use RocksDB as the state store, you can now get better visibility into the performance of RocksDB, with detailed metrics on get/put latencies, compaction latencies, cache hits, and so on. Thes...
- 1 kudos
- 712 Views
- 0 replies
- 1 kudos
docs.databricks.com
Advantage of using Photon EngineThe following summarizes the advantages of Photon:Supports SQL and equivalent DataFrame operations against Delta and Parquet tables.Expected to accelerate queries that process a significant amount of data (100GB+) and ...
- 712 Views
- 0 replies
- 1 kudos
- 2410 Views
- 1 replies
- 2 kudos
- 2410 Views
- 1 replies
- 2 kudos
- 2 kudos
check if your workspace has the IP access list feature enabled, call the get feature status API (GET /workspace-conf). Pass keys=enableIpAccessLists as arguments to the request.In the response, the enableIpAccessListsthe field specifies either true o...
- 2 kudos
- 2831 Views
- 1 replies
- 0 kudos
Can multiple users collaborate together on MLflow experiments?
Wondering about best practices for how to handle collaboration between multiple ML practitioners working on a single experiment. Do we have to share the same notebook between people or is it possible to have individual notebooks going but still work ...
- 2831 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, multiple users could work on individual notebooks and still use the same experiment via mlflow.set_experiment(). You could also assign different permission levels to experiments from a governance point of view
- 0 kudos
- 2622 Views
- 1 replies
- 0 kudos
Resolved! Can I save MLflow artifacts to locations other than the dbfs?
The default location or MLflow artifacts is on dbfs, but I would like to save my models to an alternative location. Is this supported, and if it is how can I accomplish it?
- 2622 Views
- 1 replies
- 0 kudos
- 0 kudos
You could mount an s3 bucket in the workspace and save your model using the mounts DBFS path For e.gmodelpath = "/dbfs/my-s3-bucket/model-%f-%f" % (alpha, l1_ratio) mlflow.sklearn.save_model(lr, modelpath)
- 0 kudos
- 1597 Views
- 1 replies
- 2 kudos
- 1597 Views
- 1 replies
- 2 kudos
- 2 kudos
Not yet, but stay-tuned it's being cooked in the kitchen
- 2 kudos
- 1130 Views
- 1 replies
- 0 kudos
- 1130 Views
- 1 replies
- 0 kudos
- 0 kudos
Data is stored in the control plane. Metadata (eg feature table descriptions, column types, etc) is stored in the control plane. The location where the Delta table is stored is determined by the database location. The customer could call CREATE DATA...
- 0 kudos
- 1469 Views
- 0 replies
- 1 kudos
Dev and Prod environments
Do we have general guidance around how other customers manage Dev and Prod environments in Databricks? Is it recommended to have separate workspaces for them? What are the pros and cons of using the same workspace with folder or repo level isolation?
- 1469 Views
- 0 replies
- 1 kudos
- 2183 Views
- 1 replies
- 0 kudos
Delta Lake MERGE INTO statement error
I'm trying to run Delta Lake MergeMERGE INTO source USING updates ON source.d = updates.sessionId WHEN MATCHED THEN UPDATE * WHEN NOT MATCHED THEN INSERT *I'm getting an SQL errorParseException: mismatched input 'MERGE' expecting {'(', 'SELECT', 'FR...
- 2183 Views
- 1 replies
- 0 kudos
- 0 kudos
The merge SQL support is added in Delta Lake 0.7.0. You also need to upgrade your Apache Spark to 3.0.0 and enable the integration with Apache Spark DataSourceV2 and C
- 0 kudos
- 802 Views
- 0 replies
- 0 kudos
Should I be saving my SparkML models in MLflow using MLeap?
There's a lot of different ML formats out there and I am confused about how they should be fitting together. How should I be thinking about MLflow and MLeap working together?
- 802 Views
- 0 replies
- 0 kudos
