- 1100 Views
- 1 replies
- 2 kudos
Issue: Your account {email} does not have the owner or contributor role on the Databricks workspace resource in the Azure portalÂ
Issue: Your account {email} does not have the owner or contributor role on the Databricks workspace resource in the Azure portal
- 1100 Views
- 1 replies
- 2 kudos
- 2 kudos
https://docs.microsoft.com/en-us/azure/databricks/scenarios/frequently-asked-questions-databricks#solution-1
- 2 kudos
- 2459 Views
- 1 replies
- 0 kudos
Streaming with Kafka with the same groupid
A kafka topic is having 300 partitions and I see two clusters are running and have the same group id, will the data be duplicate in my delta bonze layer
- 2459 Views
- 1 replies
- 0 kudos
- 0 kudos
By default, each streaming query generates a unique group ID for reading data ( ensuring it's own its own consumer group ) . In scenarios where you'd want to specify it (authz etc ) , it is not recommended to have two streaming applications specify ...
- 0 kudos
- 6299 Views
- 3 replies
- 0 kudos
Resolved! Delta lake Check points storage concept
In which format the Checkpoints are stored in storage and , how does it help in delta to increase performance.
- 6299 Views
- 3 replies
- 0 kudos
- 0 kudos
Great points above on how checkpointing helps with performance. In additional Delta Lake also provides other data organization strategies such as compaction, Z-ordering to help with both read and write performance of Delta Tables. Additional details ...
- 0 kudos
- 3756 Views
- 2 replies
- 0 kudos
- 3756 Views
- 2 replies
- 0 kudos
- 0 kudos
Temp Views and Global Temp Views are the most common way of sharing data across languages within a Notebook/Cluster
- 0 kudos
- 5077 Views
- 1 replies
- 0 kudos
How many records does Spark use to infer the schema? entire file or just the first "X" number of records?
It depends. If you specify the schema it will be zero, otherwise it will do a full file scan which doesn’t work well processing Big Data at a large scale.CSV files Dataframe Reader https://spark.apache.org/docs/latest/api/python/reference/api/pyspark...
- 5077 Views
- 1 replies
- 0 kudos
- 0 kudos
As indicated there are ways to manage the amount of data being sampled for inferring schema. However as a best practice for production workloads its always best to define the schema explicitly for consistency, repeatability and robustness of the pipe...
- 0 kudos
- 1536 Views
- 1 replies
- 0 kudos
- 1536 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes Convert to Delta allows for converting a parquet table into Delta format in place by adding a transaction log, infering the schema and also collecting stats to improve query performance - https://docs.databricks.com/spark/latest/spark-sql/languag...
- 0 kudos
- 2237 Views
- 3 replies
- 0 kudos
- 2237 Views
- 3 replies
- 0 kudos
- 0 kudos
And to the earlier comment of Delta being an extension of Parquet. You can start with a dataset in Parquet format in S3 and do an in-place conversion to Delta without having to duplicate the data. See - https://docs.databricks.com/spark/latest/spark-...
- 0 kudos
- 2854 Views
- 2 replies
- 1 kudos
- 2854 Views
- 2 replies
- 1 kudos
- 1 kudos
You can also use tags to setup a chargeback mechanism within your organization for distributed billing - https://docs.databricks.com/administration-guide/account-settings/usage-detail-tags-aws.html
- 1 kudos
- 3499 Views
- 2 replies
- 0 kudos
Resolved! Can a cluster be automatically deleted if it's inactive? How can I prevent it from deleting?
- 3499 Views
- 2 replies
- 0 kudos
- 0 kudos
Per the comment above the cluster deletion mechanism is designed to keep your cluster config experience organized and not have proliferation of cluster config. Its also a good idea to setup cluster policies and leverage those as a guide for what kind...
- 0 kudos
- 5461 Views
- 2 replies
- 0 kudos
Databricks SQL Visualizations - export/embed
Is it possible to embed Databricks SQL Dashboards or specific widgets/visualization into a webpage?
- 5461 Views
- 2 replies
- 0 kudos
- 0 kudos
Databricks SQL also integrates with several popular BI tools over JDBC/ODBC which you can use as a mechanism to embed visualizations into a webpage
- 0 kudos
- 4292 Views
- 1 replies
- 0 kudos
- 4292 Views
- 1 replies
- 0 kudos
- 0 kudos
See this docs article for details on setting up a Delta Presto integration - https://docs.databricks.com/delta/presto-integration.html
- 0 kudos
- 2314 Views
- 1 replies
- 0 kudos
- 2314 Views
- 1 replies
- 0 kudos
- 0 kudos
You can use libraries such as Seaborn, Bokeh, Matplotlib, Plotly for visualization inside of Python notebooks. See https://docs.databricks.com/notebooks/visualizations/index.html#visualizations-in-pythonAlso, Databricks has its own built-in visualiza...
- 0 kudos
- 11247 Views
- 2 replies
- 1 kudos
- 11247 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks @Digan Parikh​ . Credit to Tahir Fayyaz, Found a couple of different paths depending on whether you're looking to bring in raw GA data vs aggregated GA data. 1) For Raw You can bring in data from GA Universal Analytics 360 Paid version or GA ...
- 1 kudos
- 1296 Views
- 0 replies
- 0 kudos
- 1296 Views
- 0 replies
- 0 kudos
- 1851 Views
- 1 replies
- 0 kudos
What is databricks Sync
I am trying to migrate my workload to another workspace ( from ST to E2), I am planning to use data bricks sync, but still I am not sure, will it migrate everything like , currents, user , groups, job, notebook etc or has some limitations which I s...
- 1851 Views
- 1 replies
- 0 kudos
- 0 kudos
Here is the support matrix for import/export operations for databricks-syncAlso checkout https://github.com/databrickslabs/migrate
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 483 | |
| 349 | |
| 287 |
