- 8248 Views
- 3 replies
- 0 kudos
Resolved! Deploying Databricks via ARM, then configuring via databricks-cli: creating a repo results in INVALID_STATE error.
Hi,I'm using databricks-cli to configure a newly-deployed Azure Databricks instance. The ARM deployment works fine, and then the databricks cli commands to create a secrets scope and add users also works just fine.Then I add a GitCredential to Datab...
- 8248 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Chris Platts​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Thanks and Regards
- 0 kudos
- 8969 Views
- 6 replies
- 9 kudos
Resolved! Databricks cluster Init scripts on ABFSS location
HI,I have an init script which works on DBFS location during the cluster start up, but when the same shell script file is placed on ABFSS location (ADLS Gen 2 storage) I get the following init script failure error and the cluster is unable to start.E...
- 8969 Views
- 6 replies
- 9 kudos
- 9 kudos
Hi @Saravana KJ​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest pr...
- 9 kudos
- 3545 Views
- 1 replies
- 5 kudos
Resolved! sudden decrease in rewards point in Rewards portal
@Vidula Khanna​ May I know why the reward points decreased in Databricks community rewards portal. Could anyone help on this. I haven't ordered for anything. But today when I checked there was sudden decrease in points and the no of points came from ...
- 3545 Views
- 1 replies
- 5 kudos
- 2770 Views
- 1 replies
- 0 kudos
The hire react js developer from the best react js development company. We provide the most experienced hire reactjs developer as per the requirements...
The hire react js developer from the best react js development company. We provide the most experienced hire reactjs developer as per the requirements cost.
- 2770 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for sharing! By the way, if you are interested in the costs of hire react js developers, I advise you to read this: https://www.cleveroad.com/blog/how-much-does-it-cost-to-hire-a-react-developer/
- 0 kudos
- 11605 Views
- 2 replies
- 0 kudos
Issue with Pyspark GroupBy GroupedData
Hi Guys,I am working on streaming data movement from bronze to silver. My bronze table is having a entity_name column, based on the entity_name column i need to create multiple silver tables.I tried the below approach, But it is failing with error "'...
- 11605 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Harun Raseed Basheer​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best ...
- 0 kudos
- 6631 Views
- 2 replies
- 2 kudos
Resolved! Error while using Array_contains function in left join condition
'Item_id' is column in array format like ["ba1b-5fbe1547ddd5", "88f9-ac3b93334f69", "8bba-4075a47eb814"] in table1 and table2 has column Id with single value like ba1b-5fbe1547ddd5.While join two table select table1.*,table2.*from table1left join tab...
- 6631 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Rishabh Shanker​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 2 kudos
- 3526 Views
- 2 replies
- 1 kudos
Resolved! How does pyspark work in these two scenarios?
I have two scenarios with different outcomes:Scenario 1:from pyspark.sql.functions import *# create sample dataframesdf1 = spark.createDataFrame([(1, 2, 3), (2, 3, 4)], ["a", "b", "c"])df2 = spark.createDataFrame([(1, 5, 6, 7), (2, 8, 9, 10)], ["a", ...
- 3526 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Direo Direo​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 1 kudos
- 4635 Views
- 3 replies
- 1 kudos
read base64 json column with Autoloader and inferschema.
I have json files falling in our blob with two fields, 1. offset(integer), 2. value(base64).This value column is json with unicode. so they sent it as base64. Challenge is this json is very large with 100+ fields. so we cannot define the schema. We c...
- 4635 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @MerelyPerfect Per​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....
- 1 kudos
- 12202 Views
- 1 replies
- 1 kudos
Resolved! How to get today's date in the local time zone?
I am trying to get today's date in the local time zone:from pyspark.sql.functions import * date = to_date(from_utc_timestamp(current_timestamp(), 'Australia/Melbourne'))What I get using the above code is a column object. How can I get its value in a...
- 12202 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Mohammad Saber​ , you can use pytz and datetime python package for your usecase,, attaching code snippet in below screen shot.
- 1 kudos
- 2109 Views
- 2 replies
- 2 kudos
- 2109 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @may may​ Your input matters! Help our community thrive by coming back and marking the most helpful and accurate answers. Together, we can make a difference!Regards
- 2 kudos
- 3496 Views
- 2 replies
- 1 kudos
How install a library that is under the /Workspace/Shared/ directory using the init.sh script in a cluster?
I would like to install a library that is under the /Workspace/Shared/ directory using the init.sh script in a cluster. How to access the /Workspace/Shared/ folder in shell? This page only shows how to access manually but doesn't show how to access i...
- 3496 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Juned Mala​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 1 kudos
- 8182 Views
- 2 replies
- 3 kudos
Do we really need Autoloader for batch processing.?
Hi All,It seem AutoLoader is good option for even driven data ingestion but if my job runs only once , do I still need autoloader ? I dont want to spend money to spin a cluster whole day.I know we have RunOnce option available while running a job but...
- 8182 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Deepak Bhatt​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Thanks and regards
- 3 kudos
- 3141 Views
- 2 replies
- 3 kudos
DLT pipeline slow streaming (root cause needs to be identified)
Dear support,we have the following situation where a set of DLT pipelines are streaming with very low rate incoming data and we need to find the root cause of this delay.In order to provide more insight about the setup of the DLT pipelines and some m...
- 3141 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @EDDatabricks EDDatabricks​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that ...
- 3 kudos
- 6079 Views
- 2 replies
- 2 kudos
Issue with reading exported tables stored in parquet
Hi All, I am exported all tables from postgres snapshot into S3 in parquet format. I am trying to read the table using databricks and i am unable to do so. I get the following error: "Unable to infer schema for Parquet. It must be specified manually....
- 6079 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @shiva charan velichala​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that bes...
- 2 kudos
- 10653 Views
- 8 replies
- 10 kudos
Resolved! New strange error on Runtime 12 and above: java.lang.AssertionError: assertion failed
Hi allI struggle to find out why this error message suddenly pops up after running a cell in a notebook. The notebook is trying to run a simple "INSERT INTO" command in SQL. When I only do a SELECT clause, the cell runs without error. Also, I only ge...
- 10653 Views
- 8 replies
- 10 kudos
- 10 kudos
Thanks for reporting! We have identified a defect with an early version of DBR 12 related to INSERT INTO .. SELECT when certain query patterns are involved. The defect has since been fixed. Please let us know if you have any additional questions.
- 10 kudos
| User | Count |
|---|---|
| 1616 | |
| 787 | |
| 379 | |
| 349 | |
| 287 |
