I am having basic question, As managed tables doesnt store their data into ADLS Gen2. But in our architcture we created 3 containers in ADLS Gen2 (Bronze, Silver and Gold) . If I chose managed tables then neither metadata nor data doesnt store into ...
Hi Everyone, I have been running below code. However Im getting CUDA out of memory error even though I have 4 GPUs in cluster which should ideally have 64 GB GPU , but the code is failing with 16 GB. I assume that the code is not utilizing all 4 GPU ...
Your code is loading the full model into a single GPU so having multiple GPUs does not prevent out of memory errors. By default, transformer models only have DDP (distributed data parallel) so each GPU has a copy of your model for speeding up trainin...
How to mock a code that uses dbutils from"from databricks.sdk.runtime import dbutils"it shows databricks-sdk has no attribute runtime
Hi @samarth_solanki , I Hope you are doing well! Based on the information you have shared, it seems like you're trying to import dbutils from databricks.sdk.runtime, but you're encountering an error that says "databricks-sdk has no attribute runtime...
When I select text in a notebook cell the whole notebook scrolls up as a select. This happens when I use the mouse wheel and with shift+arrow key. It varies by cell - happens in some cells, but not other cells within the same notebook. When I refresh...
I'm relieved to know that I'm not the only one experiencing this issue.Please, address this as soon as possible. It's significantly impacting my productivity.
When I try to access my company files, I keep getting the error 6000 301 in QB. Please assist me in fixing this mistake.
Are you facing QB error 6000 301 while trying to log in to your company file in QB Desktop and don’t know what should be done next? If yes, then you should not panic at all because through this post I am going to tell you everything you need to know ...
I'm trying to create storage credentials for an Azure Databricks Connector at the workspace level with a service principal that has the CREATE_STORAGE_CREDENTIAL but is NOT an account admin. For this test, the SP has the owner role on the connector.I...
Hi @ledbutter , hope you are doing well today! I have gone through the details and this issue might be related to https://github.com/databricks/cli/issues/1080 Please refer to this for more details: https://github.com/databricks/cli/issues/1108 Plea...
Hello Community,I have the following issue. When I am running cells from a notebook, I have the print outputs from the previous cells that are appended to the current print output (meaning running cell 1 gives output 1, running cell 2 gives output 1 ...
Hi,are there any rss feeds for the databricks platform, sql & runtime releases? We have a big tech stack so it is sometimes hard to keep up with the ever changing technologies. We are using rss feeds to keep up with all of that.Cant find anything for...
Hi @florianb , You can keep these documents handy for the Platform releases and SQL Runtime releases: Platform Releases: https://docs.databricks.com/en/release-notes/product/index.html SQL Releases: https://docs.databricks.com/en/sql/release-notes/in...
Hi, I am trying to create a query definition using API '/api/2.0/preview/sql/queries' in postman but getting internal server error. Below is the snap for the snap. Let me know If I am doing anything wrong here.
I'm interested to know how this error was resolved. I'm getting an "Internal Server Error" returned when trying to create queries with version 1.36.1 of the Databricks Terraform Provider. The error provides no other information.
I was wondering if someone could help us with implementation here. Our current program will spin up 5 jobs through the Databricks API using the same Databricks cluster but each one needs their own spark context (specifically each one will connect to ...
you can set up buckets with different credentials, endpoints, and so on.https://docs.databricks.com/en/connect/storage/amazon-s3.html#per-bucket-configuration
Data Engineer – Databricks - RemoteApply Here: Job Application for Data Engineer – Databricks at Jenzabar (greenhouse.io)Jenzabar Website: Higher Education Software Solutions - JenzabarFor over four decades, the higher education experts at Jenzabar h...
Each year, we celebrate the amazing customers that rely on Databricks to innovate and transform their organizations — and the world — with the power of data and AI. The nomination form is now open to submit nominations. Nominations will close on Marc...
I see the current conversion of dataframe to xml need to be improved.My dataframe schema is a perfect nested schema based on structs but when I create a xml I have the follow issues:1) I can't add elements to root2) rootTag and rowTag are requiredIn ...
Here is one of the ways to use the struct field name as rowTag: import org.apache.spark.sql.types._ val schema = new StructType().add("Record", new StructType().add("age", IntegerType).add("name", StringType)) val data = Seq(Row(Row(18, "John ...
Taking from my experience, the default action of widgets triggering code execution upon value change poses risks that outweigh the convenience in certain scenarios. While this feature may seem advantageous in some cases, it can lead to unintended con...
I definitely have to agree with the original point- if you have a notebook that you import, and you touch any widget value you're running code, most likely accidentally. I'd love to see a workspace or user type option where you can change the default...
Hi.I'm trying to generate XML as output base on my nested dataframe. Everything is ok except by I don't know how to add elements to rootTag.I can add elements from rowtag but not in rootTag. Same problems to add attributes to root <books version = "...
Hi @RobsonNLPT ,Thanks for bringing up your concerns, always happy to help Can you please refer to the below document to read and write the XML files? https://docs.databricks.com/en/query/formats/xml.html Please let me know if this helps and leave a...
