- 6235 Views
- 2 replies
- 4 kudos
- 6235 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi @maddy v​ ,I recommend that you use the Databricks SQL module for this type of reports and email alerts. It is a very interesting module with multiple options for your use case.https://learn.microsoft.com/en-us/azure/databricks/sql/user/dashboards...
- 4 kudos
- 1571 Views
- 1 replies
- 3 kudos
Spark SQL & Spark ML
I am using Spark SQL to import their data into a machine learning pipeline. Once data is imported I want performs machine learning tasks using Spark ML. So I should use what compute tools is best suited for this use case? Please help me!!! Thank you ...
- 1571 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi, please refer https://docs.databricks.com/machine-learning/index.html, please let us know if this helps.
- 3 kudos
- 1610 Views
- 1 replies
- 2 kudos
I am using Spark SQL to import their data into a machine learning pipeline. Once data is imported I want performs machine learning tasks using Spark...
I am using Spark SQL to import their data into a machine learning pipeline. Once data is imported I want performs machine learning tasks using Spark ML. So I should use what compute tools is best suited for this use case? Please help me!!! Thank y...
- 1610 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi, please refer https://docs.databricks.com/machine-learning/index.html, please let us know if this helps.
- 2 kudos
- 1814 Views
- 1 replies
- 2 kudos
How to Enable Files in Repos in DBR 7.3 LTS ML ?
we need a custom version of a GPU cluster with following requirements for a certain project:Ubuntu 18.04Cuda 10.1.Tesla T4 GPU.Availability of /Workspace/Repos folder.All of these requirements are available with DBR ML 7.3 LTS. But one critical compo...
- 1814 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi, To work with non-notebook files in Databricks Repos, you must be running Databricks Runtime 8.4 or above.https://docs.databricks.com/files/workspace.html#configure-support-for-workspace-files
- 2 kudos
- 1553 Views
- 1 replies
- 2 kudos
How to access image data for yolov5 object detection module
How to access image data for the yolov5 object detection module, I am unable to do so.
- 1553 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi, Could you please provide your used case briefly on the same?
- 2 kudos
- 3505 Views
- 1 replies
- 2 kudos
How to find the road distance between two cities? We can use Azure databricks and azure map.
We will be given an excel file, in which we can see column sender_city and destination_city. We have to find the distance between these two cities and the distance calculated should be written in a column total_distance. All these processes should be...
- 3505 Views
- 1 replies
- 2 kudos
- 2 kudos
heywithout using latitude and longitude it is hard to find out but you can try some distance-based algorithm
- 2 kudos
- 4377 Views
- 2 replies
- 3 kudos
Auto Loader: Empty fields (discovery_time, commit_time, archive_time) in cloud_files_state
Hi! Why are the fields discovery_time, commit_time, and archive_time NULL in cloud_files_state? Do I need to configure anything when creating my Auto Loader? df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "json") \ ...
- 4377 Views
- 2 replies
- 3 kudos
- 3 kudos
Please be sure that the DBR version is 10.5 or highercommit_time and archive_time can be null but discovery_time is set even as NOT NULL in the table definition so it is a bit strange. Please change the DBR version first.
- 3 kudos
- 4221 Views
- 3 replies
- 4 kudos
Resolved! Bug in Delta Live Tables when missing files option?
When using Delta Live Tables with SQL-syntax ignoreMissingFiles-option is not working and you are getting error anyway.(See picture below)Link to feature: https://learn.microsoft.com/en-us/azure/databricks/ingestion/auto-loader/options#generic-option...
- 4221 Views
- 3 replies
- 4 kudos
- 4 kudos
You could also use inferSchema. ignoreMissingFiles option is to handle files that were accidentally deleted before being fully processed, so it has nothing related to the schema.
- 4 kudos
- 5534 Views
- 3 replies
- 2 kudos
Azure DataBricks New Job Cluster Libraries Installation Issues
Dear Team,We are trying to install runtime Libraries from Azure Data factory to Azure DataBricks and linkedservice, we are using New Job Cluster to spin the notebooks to execute the python code. we are using the third party Libraries which is prophet...
- 5534 Views
- 3 replies
- 2 kudos
- 2 kudos
init scripts are great way to handle the dependent libraries installation in cluster.https://stackoverflow.com/questions/62516102/install-python-packages-using-init-scripts-in-a-databricks-cluster
- 2 kudos
- 4145 Views
- 3 replies
- 4 kudos
How to execute Windows commands (.cmd file) from Databricks?
How to execute Windows commands (.cmd file) from Databricks?
- 4145 Views
- 3 replies
- 4 kudos
- 4 kudos
Databricks runs on linux servers. you can launch windows cmd or windows shell commands in databricks. you can run only linux commands.
- 4 kudos
- 3117 Views
- 3 replies
- 4 kudos
- 3117 Views
- 3 replies
- 4 kudos
- 9123 Views
- 4 replies
- 3 kudos
Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character
Though there are parquet files in the folder which I'm trying to query, I'm getting FileNotFoundException. Please Let me know if there is any issue in the code.Thanks
- 9123 Views
- 4 replies
- 3 kudos
- 3 kudos
alternatively you can use this as shell command. This will work.%sh ls /dbfs/mnt/mountpoint/path/to/raw/*.parquet
- 3 kudos
- 1889 Views
- 3 replies
- 2 kudos
Reward point for lakehouse fundamentals accreditation V2 and databricks community group join
Team, I am unable to find the points at my account yet for the subjected mentioned.Please let me know when it would be reflected on my side.Thanks in advance!​
- 1889 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @buvaneswari V​ Just a friendly follow-up, has your issue been resolved? Please let me know if you need any further assistance.Thanks and Regards
- 2 kudos
- 2017 Views
- 1 replies
- 0 kudos
User logging in Databricks
Hello experts. Is there a way to see user logs as who is running which notebook or a specific query? I couldn't find anything in "Driver logs" (stdout, log4j). Just to specify that I have admin role in the workspace. Thank you in advance
- 2017 Views
- 1 replies
- 0 kudos
- 0 kudos
Any update on this? Thank you
- 0 kudos
- 1884 Views
- 2 replies
- 3 kudos
AWS EC2 launched from Databricks tenancy
Hi,I was checking the EC2 details in our AWS account, and found all the EC2's launched from Databricks are with "dedicated" tenancy. I double checked the cluster launch configuration and didn't find anywhere to change the tenancy setting. How can we ...
- 1884 Views
- 2 replies
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
