- 4798 Views
- 2 replies
- 4 kudos
Feature Store - Feature Lookup Engine with join on partial key and Filter
Hello ,I am working with lookupEngine functions. However, we have some feature tables with granularity level most detailled of dataframe input.Please find an example :table A with unique keys on two features : numero_p, numero_s So while performing F...
- 4798 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi @SERET Nathalie​ , I can check internally on the ask here. In the meantime please let us know if this helps: https://docs.databricks.com/machine-learning/feature-store/feature-tables.htmlhttps://docs.databricks.com/machine-learning/feature-store/i...
- 4 kudos
- 19953 Views
- 2 replies
- 2 kudos
Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
I have been getting this error sporadically. I'm loading a dataset and training a model using the dataset in notebook. Sometimes it works and sometimes it doesn't. I have seen similar posts and tried all solutions mentioned, log output size limit, sp...
- 19953 Views
- 2 replies
- 2 kudos
- 2 kudos
@Leo Bao​ Are you seeing this issue whenever you are getting different sizes of data sets, or your data set size is same. if issue you are seeing is due to larger dataset, please check below link and try to increase partition size Databricks Spark Py...
- 2 kudos
- 4635 Views
- 1 replies
- 1 kudos
Resolved! Problem with Autoloader, S3, and wildcard
Hello, I have an autoloader code and it is pretty standard, we have this variable file path that points to an S3 bucket. example #2 executed successfully and example 1 throws an exception.it seems like source 1 always throws an exception whereas sour...
- 4635 Views
- 1 replies
- 1 kudos
- 1 kudos
The error was more related to a lot of stuff on the AWS side, so we deleted and cleared the SQS and SNS. we also used the CloudFilesAWSResourceManagerval manager = CloudFilesAWSResourceManager .newManager .option("path", filePath) .create...
- 1 kudos
- 2127 Views
- 1 replies
- 3 kudos
Can the HTML behind a SQL visualisations be accessed?
We are using MLFlow to manage the usage of some self service notebooks. This involves logging parameters, tables and figures. Figures are logged using:mlflow.log_figure( figure=fig, artifact_file="visual/fig.html" )Usually the fig object is gener...
- 2127 Views
- 1 replies
- 3 kudos
- 3 kudos
There is no way to access the html used. You can download the images. The editor uses redash, so you can try looking at that library for more information.
- 3 kudos
- 4295 Views
- 2 replies
- 2 kudos
Is it wise to use a more recent MLFlow Python package version or is the DB Runtime compatibility matrix strict about MLFlow versions?
More concretely, should we fix the dependency version at MAJOR, MINOR or PATCH?For example, MLFlow 1.30.0 is available and latest DBR 11.3 LTS is compatible with 1.29.0 My question comes from the fact that installing our own libraries that use MLFlow...
- 4295 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi! thanks for the reply, although maybe you didn't notice that I linked to the same url, so we're aware of the matrix. The question is, is it compatible solely with 1.29.0? We want to know which dependency should we use in all our projects that migh...
- 2 kudos
- 3821 Views
- 3 replies
- 0 kudos
Two or more different ml model on one cluster.
Hi, have you already dealt with the situation that you would like to have two different ml models in one cluster? i.e: I have a project which contains two or more different models with more different pursposes. The goals is to have three differ...
- 3821 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Tomas Peterek​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 0 kudos
- 4881 Views
- 4 replies
- 3 kudos
Resolved! Certification badge not received
I have completed the Databricks certified Data Engineer Associate exam on 29th October, received a mail with score and it is mentioned in mail that I would receive badge within 24 hours. It has been 4 days since I completed the exam still certificate...
- 4881 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Manasa Tanguturu​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue.If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please visit th...
- 3 kudos
- 7454 Views
- 2 replies
- 4 kudos
AttributeError: 'NoneType' object has no attribute 'enum_types_by_name'
I run into this error while using MLFlow: AttributeError: 'NoneType' object has no attribute 'enum_types_by_name'Here is the relevant stack trace:/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/mlflow/tracking/fluent....
- 7454 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi, Could you please refer to this to check if this is an issue: https://github.com/protocolbuffers/protobuf/issues/10151, Also, could you please let us know the DBR version you are using?
- 4 kudos
- 1490 Views
- 0 replies
- 1 kudos
docs.azure.cn
https://docs.azure.cn/en-us/databricks/_static/notebooks/mlflow/mlflow-end-to-end-example-azure.htmlI imported the above notebook and try in Databricks community, but those subplots for Box plots are giving me errors as below:AttributeError ...
- 1490 Views
- 0 replies
- 1 kudos
- 2609 Views
- 2 replies
- 1 kudos
medium.datadriveninvestor.com
say, I want to download 2 files from this directory (dbfs:/databricks-datasets/abc-quality/") to my local filesystem, how do I do it?I understand that if those files are inside FileStore directory, it is much straightforward, which someone posts some...
- 2609 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @THIAM HUAT TAN​ ,isn't this dbfs://databricks-datasets Databricks owned s3:// mounted to the workspace? You got an error - 403 access denied to PUT files into the s3 bucket: https://databricks-datasets-oregon.s3.us-west-2.amazonaws.com you shoul...
- 1 kudos
- 10906 Views
- 1 replies
- 3 kudos
String data type - Max number of chars
Hello. Could anyone please confirm the maximum number of characters for sql string data type? Thank you in advance
- 10906 Views
- 1 replies
- 3 kudos
- 3 kudos
I think it is 256 characters it support​
- 3 kudos
- 5491 Views
- 3 replies
- 4 kudos
Resolved! How to avoid an error when using the automl python api on a classification problem
I am working through a basic example to get familiar with databricks automl. When I run classify, I hit an mlflow error. How can I avoid this error? My code:summary = databricks.automl.classify(train_df, target_col='new_cases', data_dir='dbfs:/automl...
- 5491 Views
- 3 replies
- 4 kudos
- 4 kudos
I had (accidentally) manually installed a very early version of databricks automl. Once I upgraded the error resolved.
- 4 kudos
- 5323 Views
- 3 replies
- 2 kudos
Resolved! Problem creating FeatureStore
Hi,When trying to create the first table in the Feature Store i get a message: ''DataFrame' object has no attribute 'isEmpty'... but it is not. So I cannot use the function: feature_store.create_table()With this code you should be able to reproduce t...
- 5323 Views
- 3 replies
- 2 kudos
- 2 kudos
@Hubert Dudek​Sry about the 'df_train', I forgot to change it (the error I commented is real with the proper DF). Changing the DBR to 11.3 LTS solved the problem. Thanks!
- 2 kudos
- 1804 Views
- 0 replies
- 0 kudos
Prophet/PyStan compiling error in Runtime 10.4 LTS ML
We're upgrading our ML jobs from using Runtime 9.1 LTS ML to Runtime 10.4 LTS ML in Databricks. One of the libraries our jobs relying on is Prophet. From 9.1 to 10.4, both the versions of Prophet (1.0.1) and PyStan (2.19.1.1) haven't changed, however...
- 1804 Views
- 0 replies
- 0 kudos
- 1245 Views
- 0 replies
- 0 kudos
How to check unlinked databricks configs which are not used in any shards
We have a limit of deploying databricks shards and there are few shards that are unused. How can we check and remove these unlinked databricks shards using api calls
- 1245 Views
- 0 replies
- 0 kudos
