- 3718 Views
- 2 replies
- 0 kudos
Databricks SQL Visualizations - export
How can I export Databricks SQL Dashboards as a .pdf?
- 3718 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello, I'm trying to export my dashboard to PDF but when I run the export, none of the data is exported, only blank visuals. I would appreciate any help in getting this resolved.Dashboard in DBX: Exported PDF:
- 0 kudos
- 3633 Views
- 3 replies
- 0 kudos
Reading from /tmp no longer working
Since yesterday, reading a file copied into the cluster is no longer working.What used to work:blob = gcs_bucket.get_blob("dev/data.ndjson") -> worksblob.download_to_filename("/tmp/data-copy.ndjson") -> worksdf = spark.read.json("/tmp/data-copy.ndjso...
- 3633 Views
- 3 replies
- 0 kudos
- 0 kudos
I encountered this same issue, and figured out a fix!For some reason, it seems like only %sh cells can access the /tmp directory. So I just did...%sh ch /tmp/<file> /dbfs/<desired-location> and then accessed it form there using Spark.
- 0 kudos
- 3549 Views
- 4 replies
- 1 kudos
Resolved! I haven't received badge for completion
Hi,Today I completed the test for Lakehouse fundamentals by scored 85%, still I haven't received the badge through my email francis@intellectyx.comKindly let me know please !-Francis
- 3549 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi, I have same problem but you must wait up to 24 hours and you will receive the badge
- 1 kudos
- 1524 Views
- 3 replies
- 2 kudos
I haven't received certificate or badge after clearing data engineer associate exam
Hello Databricks Team,I completed my data engineer associate exam & passed the exam. But still now I haven't received certificate or badge. Also created help ticket, Ticket Number: #[00232273]​ Mail id : satishshravan888@gmail.comThanks,Satish
- 1524 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Satish Shravan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 2 kudos
- 13379 Views
- 3 replies
- 6 kudos
Resolved! "User not authorized" error when trying to deploy access connector on Azure
Hi,I have been trying to deploy Access Connector resource on Azure using Azure Pipelines (YAML) and a Bicep template but I cannot find a solution to this error:ERROR: {"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resou...
- 13379 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi,I fixed this issue by adding the service principal to the list of service principals in the Account Console. My guess is that after the access connector is created an API call is made to the Databricks account and the service principal making that...
- 6 kudos
- 3237 Views
- 2 replies
- 2 kudos
implement autoloader to ingest data into delta lake, i have 100 different tables with full load , append merge senarios
i want to implement autoloader to ingest data into delta lake from 5 different source systems and i have 100 different tables in each database how do we dynamically address this by using autoloader , trigger once option - full load , append merge sen...
- 3237 Views
- 2 replies
- 2 kudos
- 2 kudos
You can create a generic notebook that will be parametrized with the table name/source system and then just simply trigger notebook with different parameters (for each table/source system).For parametrization you can use dbutils.widgets (https://docs...
- 2 kudos
- 2381 Views
- 1 replies
- 0 kudos
Resolved! Error Using spark.catalog.dropTempView()
I have a set of Spark Dataframes that I convert into Temp Views to run Spark SQL with. Then, I delete them after my logic/use is complete. The delete step throws an odd error that I am not sure how to fix. Looking for some tips on fixing it. As a not...
- 2381 Views
- 1 replies
- 0 kudos
- 0 kudos
spark.sql("DROP TABLE "+prefix_updates) spark.sql("DROP TABLE "+prefix_main)Fixed it for me.
- 0 kudos
- 15908 Views
- 6 replies
- 6 kudos
Local Development on Databricks
How to Develop Locally on Databricks with your Favorite IDEdbx is a Databricks Labs project that allows you to develop code locally and then submit against Databricks interactive and job compute clusters from your favorite local IDE (AWS | Azure | GC...
- 15908 Views
- 6 replies
- 6 kudos
- 6 kudos
I'm actually not a fan of dbx. I prefer the AWS Glue interactive sessions way of using the IDE. It's exactly like the web notebook experience. I can see the reason why dbx exists, but I'd still like to use a regular notebook experience in my IDE.
- 6 kudos
- 2715 Views
- 4 replies
- 4 kudos
Passed Data Engineer Associate Exam but received twice the Lakehouse Accreditation
Hi all,today (27/12/22 14:00 Rome Time Zone) I passed the Data Engineer Associate exam, but I received the badge of Lakehouse Fundamentals (second time). My email address is: sisbarra@gmail.comMy company address is: antonio.sisbarra@nttdata.comCan ...
- 2715 Views
- 4 replies
- 4 kudos
- 4 kudos
Hello Antonio,I deleted the badge under: antonio.sisbarra@nttdata.com . You are good to go.Thanks!
- 4 kudos
- 5853 Views
- 9 replies
- 14 kudos
Resolved! Databricks Lakehouse Fundamentals Accreditation V2 badge/certificate not received.
Hello, this is regarding Databricks Lakehouse Fundamentals Accreditation V2. I haven't received my badge/certificate. I also raised a ticket but haven't received any response. My request id is #00248504.​. Kindly help me out with this.
- 5853 Views
- 9 replies
- 14 kudos
- 14 kudos
@Jose Gonzalez​ ,@Vidula Khanna​ ,I also did not received my badge in portalThis is my portal link- https://credentials.databricks.com/profile/aviralbhardwaj143185/walletMy case number is - 00250939Also, my points are not updating in the reward port...
- 14 kudos
- 2693 Views
- 3 replies
- 2 kudos
How do we concatenate some fixed string with a secret value in Spark Config in Databricks Job Cluster?
Hi Team,I am trying to configure access to adls through Service Principal through Spark Config in Databricks job cluster. like, fs.azure.account.oauth2.client.id.<adls_account_name>.dfs.core.windows.net {{secrets/scopeName/clientID}}The above stateme...
- 2693 Views
- 3 replies
- 2 kudos
- 2 kudos
@Kaniz Fatma​ We are blocked on this issue. Can you please look into the thread and give your suggestion to workaround it.
- 2 kudos
- 19959 Views
- 1 replies
- 0 kudos
Resolved! How to show all rows by "DataFrame.show()"?
Hi,DataFrame.show() has a parameter n to set "Number of rows to show".Is there any way to show all rows?
- 19959 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Medothis method will work fine df.show(df.count())
- 0 kudos
- 1859 Views
- 2 replies
- 3 kudos
Resolved! Can't do "Full screen" while taking Databricks Apache Spark developer course.
Hi, I see the option for "Full screen" on bottom right but its disabled/inactive. Attached is the screenshot for same.Please advise as its hard to read or see contents on half screen.Thanks
- 1859 Views
- 2 replies
- 3 kudos
- 1872 Views
- 3 replies
- 3 kudos
Access file location problem
import pandas as pd from apiclient.discovery import build from oauth2client.service_account import ServiceAccountCredentials df = spark.read.json("/FileStore/tables/cert.json") SCOPES = ['https://www.googleapis.com/auth/analytics.readonly'] KEY_FIL...
- 1872 Views
- 3 replies
- 3 kudos
- 3 kudos
Looks like it is because the oauth2client.service_account does not know about DBFS (whereas spark does).Is it an option to manage your secrets in databricks? https://docs.databricks.com/security/secrets/secrets.html
- 3 kudos
- 16857 Views
- 5 replies
- 7 kudos
Resolved! Best Practices for naming Tables and Databases in Databricks
We moved in Databricks since few months from now, and before that we were in SQL Server.So, all our tables and databases follow the "camel case" rule.Apparently, in Databricks the rule is "lower case with underscore".Where can we find an official doc...
- 16857 Views
- 5 replies
- 7 kudos
- 7 kudos
Hi @Salah KHALFALLAH​ , looking at the documentation it appears that Databricks' preferred naming convention is lowercase and underscores as you mentioned.The reason for this is most likely because Databricks uses Hive Metastore, which is case insens...
- 7 kudos