- 668 Views
- 1 replies
- 0 kudos
I am not able to login with my credentials. This is happening with me again and again.i have created different account then also,i am facing the same ...
I am not able to login with my credentials. This is happening with me again and again.i have created different account then also,i am facing the same proble..please help me to resolve this issue...i am a new learner here
- 668 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @PREM RANJAN​ It might be a temporary issue, for any issue with Academy learning/certifications, you can raise a ticket in the below link, sharing it with you for your future reference as well.https://help.databricks.com/s/contact-us?ReqType=train...
- 0 kudos
- 1255 Views
- 2 replies
- 0 kudos
Resolved! When Databricks on AWS will support c6i/m6i/r6i EC2 instance types?
The instances are almost 1.5 years old now and provide better efficiency that the 5 gen.
- 1255 Views
- 2 replies
- 0 kudos
- 0 kudos
@Sergey Ivanychev​ those instance types are under development and should be GA very soon. No official date AFAIK
- 0 kudos
- 3906 Views
- 7 replies
- 7 kudos
Databricks Jobs and CICD
Hi, We currently leverage Azure DevOps to source control our notebooks and use CICD to publish the notebooks to different environments and this works very well. We do not have the same functionality available for Databricks jobs (the ability to sourc...
- 3906 Views
- 7 replies
- 7 kudos
- 7 kudos
My team is currently looking at establishing REPO(s) for source control to start. I know I've seen some documentation for when a MERGE is completed to auto update the main branch in DBX remote repo. Does annyone have a template and/or best practices ...
- 7 kudos
- 2201 Views
- 4 replies
- 0 kudos
Running notebook from another notebook does not work when running notebook from github actions
I have a setup-notebook that users %run to run a series of notebooks. The notebook is in the root folder of my repo. In a subfolder I have several notebooks I want to run. If I run the notebook in databricks, the relative paths work (%run "./subfolde...
- 2201 Views
- 4 replies
- 0 kudos
- 0 kudos
It seems that the directory is different under GitHub actions. Not sure what exactly mentioned GitHub actions do and which one it is. Maybe you can share GitHub action used.
- 0 kudos
- 1644 Views
- 4 replies
- 1 kudos
Passing the secret scope to the url
We are trying to call an URL by using the credentials, we are able to get the data when we hard code the credentials.Not returning any data when we pass the secret scope credentials.below is the code.import requestssource_db_scope = "dev-hnd-secret-s...
- 1644 Views
- 4 replies
- 1 kudos
- 1 kudos
please try to debug what secret scope is returning. The ugly way to do it is:for letter in username: print(letter, ' ')
- 1 kudos
- 730 Views
- 1 replies
- 8 kudos
Now you can read the Delta Sharing table as a stream.Â
Now you can read the Delta Sharing table as a stream.
- 730 Views
- 1 replies
- 8 kudos
- 8 kudos
Very informative, Thanks for sharing
- 8 kudos
- 1713 Views
- 2 replies
- 2 kudos
Resolved! Does anyone have a single example of a graphframe with two+ types of vertices? (e.g. user and post, not user to user)
I have gone through about 75 pages and every single example has only relationships from one type of object to the same type of object. about 90% have the exact same example of "Alice Bob" "friends."Has anyone ever made a graphframe with two types of ...
- 1713 Views
- 2 replies
- 2 kudos
- 2 kudos
I feel your pain,I once tried to use graphframes to flatten a complex tree, ended up using graphX (which is even worse to use but at least it is more flexible).So maybe take a look at graphX? Beware, it is terrible to use.I wonder what happened to m...
- 2 kudos
- 3927 Views
- 3 replies
- 8 kudos
Data type not shown correctly in SQL editor
When selecting from a table in the SQL editor it doesn't always preview the actual data type that the column is. e.g. I have a decimal() data type in one of my tables and when I select it it previews to a float with 2 decimals (which is different fro...
- 3927 Views
- 3 replies
- 8 kudos
- 8 kudos
Hey thanks for your response!That definitely seems like what's happening! I'm new to Databricks -- where can I find that editor?So my problem was actually two-pronged but I only outlined part of the problem above, which you seem to have solved!The ot...
- 8 kudos
- 18826 Views
- 2 replies
- 0 kudos
Resolved! how to read delta table from the path?
an unmanaged delta table is dropped and the real data still there. Now I'm trying to rebuild it, but don't know the schema. So, I tried: val myTable = DeltaTable.forPath("myPath"). But how can I get the data or schema out from myTable?Thanks!
- 18826 Views
- 2 replies
- 0 kudos
- 0 kudos
@andrew li​ df = spark.read.format("delta").load("/file_path") df.printSchema()
- 0 kudos
- 14000 Views
- 6 replies
- 8 kudos
Resolved! Databricks to Oracle
How to write data from Databricks SQL to Oracle DB
- 14000 Views
- 6 replies
- 8 kudos
- 8 kudos
we can use JDBC driver to write dataframe to Oracle tables. Every database will use jdbc connect to connect & access database. You can follow same process for connecting to any database.Download Oracle ojdbc6.jar JDBC DriverYou need an Oracle jdbc dr...
- 8 kudos
- 1051 Views
- 1 replies
- 1 kudos
Resolved! Database CICD Pipelines
My team has a shared codebase and we are running into issues as we migrate to Databricks when two people are doing development on connected sections of our codebase.For example if I add a column to a table for changes on my branch, other members on m...
- 1051 Views
- 1 replies
- 1 kudos
- 1 kudos
@Coleman Milligan​ It's really hard to create something like this without basic knowledge about how CICD should work or even Terraform.You can start here, to understand some basics.https://servian.dev/how-to-hardening-azure-databricks-using-terraform...
- 1 kudos
- 2918 Views
- 3 replies
- 0 kudos
Query fails with 'Error occurred while deserializing arrow data' on Databricks SQL with Channel set to Preview
Noticed with a query based on inline select and joins fails to the client with 'Error occurred while deserializing arrow data' I.e the query succeeds on Databricks but client (DBeaver, AtScale) receives an errorThe error is only noticed with Databri...
- 2918 Views
- 3 replies
- 0 kudos
- 0 kudos
Opened an ES on this, looks like an issue with the Preview channel. Thanks for your help!
- 0 kudos
- 2578 Views
- 3 replies
- 4 kudos
databricks autoloader not updating table immediately
I have a simple autoloader job which looks like thisdf_dwu_limit = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "JSON") \ .schema(schemaFromJson) \ .load("abfss://synapse-usage@xxxxx.dfs.core.windows.net/synapse-us...
- 2578 Views
- 3 replies
- 4 kudos
- 2053 Views
- 2 replies
- 1 kudos
Resolved! Deploying Databricks Workflows and Delta Live Table pipelines across workspaces
Hello,I was wondering if there is a way to deploy Databricks Workflows and Delta Live Table pipelines across Workspaces (DEV/UAT/PROD) using Azure DevOps.
- 2053 Views
- 2 replies
- 1 kudos
- 1 kudos
Yes, for sure, using Rest API Calls to https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-api-guide.htmlYou can create DLT manually from GUI and take JSON representative of it, tweak it (so it uses your env variables, for examp...
- 1 kudos
- 2102 Views
- 2 replies
- 3 kudos
How can we save a data frame in Docx format using pyspark?
I am trying to save a data frame into a document but it returns saying that the below errorjava.lang.ClassNotFoundException: Failed to find data source: docx. Please find packages at http://spark.apache.org/third-party-projects.htm #f_d...
- 2102 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi,You cannot do it from Pyspark, but you can try to use Pandas to save to Excell. There is no Docx
- 3 kudos