- 1309 Views
- 1 replies
- 0 kudos
Workflows pricing
Hi there,I checked the Databricks page on the pricing of Databricks Workflows ( https://www.databricks.com/product/pricing/jobs ) and have a question regarding the cost components: the pricing page only mentions compute costs (depending whether it's ...
- 1309 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @BobBubble2000, Could you please file a support ticket to Databricks Help Centre?
- 0 kudos
- 745 Views
- 1 replies
- 0 kudos
Snowflake Data Formatting Issue
I'm loading snowflake data to delta tables in databricks, few columns in snowflake data have datatype as Number (20,7) after loading to delta table it is taking as decimal (20,7), for example, if the value is 0.0000000 in snowflake then it is showing...
- 745 Views
- 1 replies
- 0 kudos
- 0 kudos
explicit casting seems like the way to go.First try with one column, to see if that solves your issue.If so, you can write a function that casts all decimal columns to a certain precision, something like this:def convert_decimal_precision_scale(df, p...
- 0 kudos
- 1660 Views
- 4 replies
- 1 kudos
Why is importing python code supported in Repos but not in Workspaces ?
Hi, we currently use a one repo approach which does not require a local development environment (we utilize azure dev ops and nutter for automated tests). We also have shared code accross pipelines and started with %run-sytle modularization and have ...
- 1660 Views
- 4 replies
- 1 kudos
- 1 kudos
the why is most probably because of different development tracks/teams between workspace and repos.If they will consilidate in functionality? Can't tell, only Databricks knows that; but it seems reasonable to assume the files will also be added to w...
- 1 kudos
- 1070 Views
- 3 replies
- 0 kudos
hive_metastore Access Control by different cluster type
Hello Databricks Community,I'm reaching out with a query regarding access control in the hive_metastore. I've encountered behavior that I'd like to understand better and potentially address.To illustrate the situation:I've set up three users for test...
- 1070 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Debayan, thank you for your reply. with hive_metastore, still I cannot get the level of isolation, which means that if anyone activates the Single node cluster, she/he can see all the catalog, schema, and table. However, with Unity catalog appli...
- 0 kudos
- 2709 Views
- 3 replies
- 0 kudos
GDAL on Databricks Cluster Runtime 12.2 LTS
I need gdal in my course work.After reading this post, I used init script as follows to install gdal into runtime 12.2 LTS dbutils.fs.put("/databricks/scripts/gdal_install.sh",""" #!/bin/bash sudo add-apt-repository ppa:ubuntugis/ppa sudo apt-get up...
- 2709 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi, I think I understood the issue, for the no module found error sometimes you need to clean the apt list , something like below: %shrm -r /var/cache/apt/archives/* /var/lib/apt/lists/*sudo apt-get cleansudo apt-get updatesudo add-apt-repository -y ...
- 0 kudos
- 603 Views
- 0 replies
- 0 kudos
Thinking of Building a new generative AI speech model for personal use
I have been using a voice cloning AI for quite some time now, it works pretty fine and that made me think if I could deploy Databricks to to build and train a machine learning model for the speech technology industry. At first I'm trying to apply it ...
- 603 Views
- 0 replies
- 0 kudos
- 999 Views
- 2 replies
- 0 kudos
What's LLM?
What's LLM for generative AI?
- 999 Views
- 2 replies
- 0 kudos
- 0 kudos
LLMs are large language models like Google Bard or OpenAI or ChatGPT, etc. "It is a type of artificial intelligence (AI) algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate and predict new con...
- 0 kudos
- 1201 Views
- 1 replies
- 1 kudos
Compute cluster, SQL warehouse not starting - AWS
Hey,I am using "AWSQuickstartCloudformationLambda" to create a workspace in my trial environment and on the AWS side the stack create succeeded but on the Databricks side compute cluster is not starting. The error says:Aws Authorization Failure:Failu...
- 1201 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, the error suggests there is some error with the credentials. You can check the credentials chain to see where aws is looking for the credentials. https://blog.leapp.cloud/aws-sdk-unable-to-locate-credentials.You can refer to another article: http...
- 1 kudos
- 819 Views
- 1 replies
- 0 kudos
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
Hiiiii,I want to scratch a text from the web page with simple code:from selenium import webdriver # Set options to make browsing easier #Function is responsible to create a driver and return that driver. def get_driver(): chrome_options = webdr...
- 819 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, Looks like the issue is with your browser, could you please try to change the browser and confirm?Please tag @Debayan with your next comment, which will get me notified. Thanks!
- 0 kudos
- 2004 Views
- 4 replies
- 0 kudos
webscraping with Firefox driver failing - Python code
Getting below exception while running web scraping script, stuck at installing firefox from snap. Getting apparmor permission issue and tried to reinstall it and still getting the same. please check the error message section. Dependencies:Python li...
- 2004 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi, This is an issue with the dependencies of different packages. Please try as below: apt get update && apt get upgrade; apt install firefox;Please tag @Debayan with your next comment which will notify me.
- 0 kudos
- 8872 Views
- 1 replies
- 0 kudos
Run a job with an Azure service principal
We had a problem in pointing out the Service Principal in the “Run as” option of workflow. Below are the details about the issue we are facing.Issue Details:I had created a Workflow in Azure Databricks workspace to automate the Data movement process....
- 8872 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, Could you please confirm if the service principle is already added to the workspace via admin console? https://docs.databricks.com/en/administration-guide/users-groups/service-principals.html#assign-a-service-principal-to-a-workspace-using-the-ac...
- 0 kudos
- 1038 Views
- 2 replies
- 0 kudos
Update metastore bucket
Hello,Our Databricks workspace is currently hosted on AWS account1, while the meta store bucket is configured within a separate AWS account2. We want to transfer this bucket to account1 to eliminate our reliance on account2. How to configure this buc...
- 1038 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, You can use: https://github.com/databrickslabs/migrate/blob/master/METASTORE.md# for the metastore data migration. Please tag @Debayan with your next comment which will get me notified.
- 0 kudos
- 2032 Views
- 4 replies
- 3 kudos
Resolved! Apache spark proficient
What is the best way to be proficient in Apache spark?
- 2032 Views
- 4 replies
- 3 kudos
- 617 Views
- 0 replies
- 0 kudos
Using SQL to create delta live tables but column headers from csv have spaces
I know this has probably been asked many times, although I have struggled to find the answer using sql CREATE STREAMING LIVE TABLE bronze_site_usage COMMENT "ingest latest snapshot from api"TBLPROPERTIES ("layer" = "bronze") AS SELECT * FROM cloud_fi...
- 617 Views
- 0 replies
- 0 kudos
- 10697 Views
- 7 replies
- 0 kudos
Response for list job runs API request doesn't have next/prev_page_token field
HI,When I do GET request to obtain list of job runs using `/api/2.1/jobs/runs/list` there is no `prev_page_token`, `next_page_token` fields in the response despite of having `has_more: True`.
- 10697 Views
- 7 replies
- 0 kudos
- 0 kudos
@Hubert-Dudek @Kaniz_Fatma could you confirm that behavior is an issue? if yes, can I report it anywhere?Otherwise can I report it as a feature request (to get all runs)? If yes, do you have any service to report it?
- 0 kudos
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group-
12.2 LST
1 -
Access Data
2 -
Access Delta Tables
1 -
Account reset
1 -
ADF Pipeline
1 -
ADLS Gen2 With ABFSS
1 -
Analytics
1 -
Apache spark
1 -
API
1 -
API Documentation
1 -
Architecture
1 -
Auto-loader
1 -
Autoloader
2 -
AWS
2 -
AWS security token
1 -
AWSDatabricksCluster
1 -
Azure
2 -
Azure data disk
1 -
Azure databricks
10 -
Azure Databricks SQL
4 -
Azure databricks workspace
1 -
Azure Unity Catalog
4 -
Azure-databricks
1 -
AzureDatabricks
1 -
AzureDevopsRepo
1 -
Best Practices
1 -
Billing
1 -
Billing and Cost Management
1 -
Bronze Layer
1 -
Bug
1 -
Catalog
1 -
Certification
1 -
Certification Exam
1 -
Certification Voucher
1 -
CICD
2 -
Cli
1 -
Cloud_files_state
1 -
cloudera sql
1 -
CloudFiles
1 -
Cluster
3 -
clusterpolicy
1 -
Code
1 -
Community Group
1 -
Community Social
1 -
Compute
2 -
conditional tasks
1 -
Cost
1 -
Credentials
1 -
CustomLibrary
1 -
CustomPythonPackage
1 -
Data Engineering
2 -
Data Explorer
1 -
Data Ingestion & connectivity
1 -
DataAISummit2023
1 -
DatabrickHive
1 -
databricks
2 -
Databricks Academy
1 -
Databricks Alerts
1 -
Databricks Audit Logs
1 -
Databricks Cluster
1 -
Databricks Clusters
1 -
Databricks Community
1 -
Databricks connect
1 -
Databricks Dashboard
1 -
Databricks delta
1 -
Databricks Delta Table
2 -
Databricks Documentation
1 -
Databricks JDBC
1 -
Databricks Job
1 -
Databricks jobs
2 -
Databricks Lakehouse Platform
1 -
Databricks notebook
1 -
Databricks Notebooks
2 -
Databricks Platform
1 -
Databricks Pyspark
1 -
Databricks Python Notebook
1 -
Databricks Repo
1 -
Databricks SQL
1 -
Databricks SQL Alerts
1 -
Databricks SQL Warehouse
1 -
Databricks UI
1 -
Databricks Unity Catalog
3 -
Databricks Workflow
2 -
Databricks Workflows
2 -
Databricks workspace
1 -
DatabricksJobCluster
1 -
DataDays
1 -
DataMasking
2 -
dbdemos
1 -
DBRuntime
1 -
DDL
1 -
deduplication
1 -
Delt Lake
1 -
Delta
6 -
Delta Live Pipeline
3 -
Delta Live Table
5 -
Delta Live Table Pipeline
5 -
Delta Live Table Pipelines
4 -
Delta Live Tables
6 -
Delta Sharing
2 -
deltaSharing
1 -
denodo
1 -
Deny assignment
1 -
Devops
1 -
DLT
8 -
DLT Pipeline
6 -
DLT Pipelines
5 -
DLTCluster
1 -
Documentation
2 -
Dolly
1 -
Download files
1 -
dropduplicatewithwatermark
1 -
Dynamic Variables
1 -
Engineering With Databricks
1 -
env
1 -
External Sources
1 -
External Storage
2 -
FAQ for Databricks Learning Festival
1 -
Feature Store
1 -
Filenotfoundexception
1 -
Free trial
1 -
GCP Databricks
1 -
Getting started
1 -
glob
1 -
Good Documentation
1 -
Google Bigquery
1 -
hdfs
1 -
Help
1 -
How to study Databricks
1 -
informatica
1 -
Jar
1 -
Java
1 -
JDBC Connector
1 -
Job Cluster
1 -
Job Task
1 -
Kubernetes
1 -
Lineage
1 -
LLMs
1 -
Login
1 -
Login Account
1 -
Machine Learning
1 -
MachineLearning
1 -
masking
1 -
Materialized Tables
2 -
Medallion Architecture
1 -
Metastore
1 -
MlFlow
2 -
Mlops
1 -
Model Serving
1 -
Model Training
1 -
Mount
1 -
Networking
1 -
nic
1 -
Okta
1 -
ooze
1 -
os
1 -
Password
1 -
Permission
1 -
Permissions
1 -
personalcompute
1 -
Pipeline
1 -
policies
1 -
PostgresSQL
1 -
Pricing
1 -
pubsub
1 -
Pyspark
1 -
Python
1 -
Quickstart
1 -
RBAC
1 -
Repos Support
1 -
Reserved VM's
1 -
Reset
1 -
run a job
1 -
runif
1 -
S3
1 -
SAP SUCCESS FACTOR
1 -
Schedule
1 -
SCIM
1 -
Serverless
1 -
Service principal
1 -
Session
1 -
Sign Up Issues
2 -
Significant Performance Difference
1 -
Spark
2 -
sparkui
2 -
Splunk
1 -
sqoop
1 -
Start
1 -
Stateful Stream Processing
1 -
Storage Optimization
1 -
Structured Streaming ForeachBatch
1 -
Summit23
2 -
Support Tickets
1 -
Sydney
2 -
Table Download
1 -
tabrikck
1 -
Tags
1 -
Troubleshooting
1 -
ucx
2 -
Unity Catalog
1 -
Unity Catalog Error
2 -
UntiyCatalog
1 -
Update
1 -
user groups
1 -
Venicold
3 -
volumes
2 -
Voucher Not Recieved
1 -
Watermark
1 -
Weekly Documentation Update
1 -
with open
1 -
Women
1 -
Workflow
2 -
Workspace
2
- « Previous
- Next »
User | Count |
---|---|
14 | |
12 | |
11 | |
11 | |
8 |