- 15023 Views
- 13 replies
- 1 kudos
Resolved! Want to split JSON data into multiple rows
Hi,This is my sample JSON data which is generated from api response and it is all coming in a single row. I want to split this in multiple rows and store it in a dataframe.[{"transaction_id":"F6001EC5-528196D1","corrects_transaction_id":null,"transac...
- 15023 Views
- 13 replies
- 1 kudos
- 1 kudos
Yes indeed, it was datatype issue. After changing it to Longtype in the schema definition, it is working now. Thanks once again for all your inputs and time. Much appreciated !!!
- 1 kudos
- 2385 Views
- 2 replies
- 0 kudos
Bug with show tblproperties directly returning redacted in the result set where "userid" in value
When I use show tblproperties on a view/table to see the metadata, it will redact any value which has "userid" anywhere put in to it.And it is not just through the visual interface, when I query it through python directly, it contains the redacted va...
- 2385 Views
- 2 replies
- 0 kudos
- 0 kudos
I understand that yours is a View. For my case, it's a Table so I could use `desc detail <schema_name>.<table_name>` to get the table properties info that are not redacted in the `properties` column from the `desc detail` output.
- 0 kudos
- 6327 Views
- 4 replies
- 0 kudos
My exam got suspended ; Need help immediately (10/09/2023)
Hello Team,I encountered Pathetic experience while attempting my DataBricks Data engineer certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam.I want to file ...
- 6327 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello, @sirishavemula20 It's a general practice for a proctor to ask the test taker to pan the room(as part of security measures) and its the responsibility of the test taker to make sure the surroundings are clear of any other objects whilst attempt...
- 0 kudos
- 7587 Views
- 8 replies
- 6 kudos
Error at model serving for quantised models using bitsandbytes library
Hello,I've been trying to serve registered MLflow models at GPU Model Serving Endpoint, which works except for the models using bitsandbytes library. The library is used to quantise the LLM models into 4-bit/ 8-bit (e.g. Mistral-7B), however, it runs...
- 7587 Views
- 8 replies
- 6 kudos
- 6 kudos
@phi_alpacaWe have solved it by providing a conda_env.yaml when we log the model, all we needed was to add cudatoolkit=11.8 to the dependencies.
- 6 kudos
- 3217 Views
- 2 replies
- 0 kudos
Databricks Job cost(AWS)
Hi Databricks Community,I am looking for a formula/way to calculate the estimated cost for a job run, for which I have a few questions:1. Is there any formula to calculate the cost of any job like -> [(EC2 per hr cost) * (total time job ran)]and when...
- 3217 Views
- 2 replies
- 0 kudos
- 0 kudos
This looks a little bit confusing to me, I'm looking for a more straight forward answer, more like a simple formulaThanks though for your reply
- 0 kudos
- 1736 Views
- 0 replies
- 0 kudos
sparklyr::spark_read_csv forbidden 403 error
Hi,I am trying to read a csv file into a Spark DataFrame using sparklyr::spark_read_csv. I am receiving a 403 access denied error.I have stored my AWS credentials as environment variables, and can successfully read the file as an R dataframe using ar...
- 1736 Views
- 0 replies
- 0 kudos
- 6507 Views
- 0 replies
- 0 kudos
[Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server.
I am using databricks jdbc driver to run a certain app. It runs fine for a few mins to hours and then I get the error [Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: HTTP Response code: 502,...
- 6507 Views
- 0 replies
- 0 kudos
- 1686 Views
- 0 replies
- 0 kudos
Integrate databrick cluster with aws using cloudformation
Hello community, I am trying to launch databrick cluster using AWS cloudformation template. I have checked public extension option in cloudformation and databrick cluster extension is also available in third party extension however not getting how t...
- 1686 Views
- 0 replies
- 0 kudos
- 3386 Views
- 1 replies
- 2 kudos
Hide widgets logic
Hello,We have recently created a notebook in order to allow users inserting/updating values in specific tables. The logic behind the update statements is included in a separate notebook where users don't have access. However we would like to know if ...
- 3386 Views
- 1 replies
- 2 kudos
- 2 kudos
When you want users to perform some write action (for example, change parameters, etc.), it is usually easiest to build a small app in Azure PowerApps, save those values, and extract them to the table in Delta Lake (so your notebooks will take values...
- 2 kudos
- 12419 Views
- 1 replies
- 1 kudos
Resolved! error loading databricks.proto
Hi,I'm using a DBR 13.3 LTS ML, and I want to set up a webhook trigger. I'm following the example notebook at https://learn.microsoft.com/en-us/azure/databricks/_extras/notebooks/source/mlflow/mlflow-model-registry-webhooks-python-client-example.html...
- 12419 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @FranPérez , The issue is that databricks-registry-webhooks has a databricks.proto file that collides with mlflow. Here is the fix: %pip install databricks-registry-webhooks mlflow==2.2.2 I also posted the fix on StackOverflow: https://stackoverfl...
- 1 kudos
- 4054 Views
- 1 replies
- 0 kudos
My Gamification
Hi all,I've been training at https://partner-academy.databricks.com/ and I see this tab for My Gamification, however, whenever I open it it always says 0 badges, 0 points. I have completed a number of courses, but there's no change. Is this feature...
- 4054 Views
- 1 replies
- 0 kudos
- 0 kudos
Haven't received anything back on this and don't see any others with this issue, maybe its something on my side.
- 0 kudos
- 1164 Views
- 0 replies
- 0 kudos
Referencing a Technical Blog on LinkedIn - How to show the Blog Image instead of the Author Image?
When referencing a Technical Blog in a LinkedIn Post, the image of the author is displayed and not the image/picture of the blog itself - annoying.Example: Linkedin Post: https://www.linkedin.com/posts/axelschwanke_star-struct-the-secret-life-of-t...
- 1164 Views
- 0 replies
- 0 kudos
- 11049 Views
- 2 replies
- 1 kudos
Resolved! Configuring NCC does not show option to add private endpoints
Hi!I am following this guide: https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-private-linkHowever in Step 3: Create private endpoint rules, number 6 there is no option for me to Add a private...
- 11049 Views
- 2 replies
- 1 kudos
- 1 kudos
@saikumar246 you were correct. It was really super easy to set up and works flawlessly! Good job dev-team!
- 1 kudos
- 5601 Views
- 0 replies
- 0 kudos
FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'
import whisperimport ffmpegmodel = whisper.load_model("base")transcription = model.transcribe("dbfs:/FileStore/Call_Center_Conversation__03.mp3")print(transcription["text"])FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'I have import...
- 5601 Views
- 0 replies
- 0 kudos
- 2205 Views
- 0 replies
- 0 kudos
Transform a file into a bytes format without using BlobServiceClient.
Hello everyone,I would like to know if it was possible to transform, with PySpark, a flat file stored in a directory in Azure Blob storage into bytes format to be able to parse it, while using the connection already integrated into the cluster betwee...
- 2205 Views
- 0 replies
- 0 kudos
