- 2729 Views
- 3 replies
- 4 kudos
How to execute Windows commands (.cmd file) from Databricks?
How to execute Windows commands (.cmd file) from Databricks?
- 2729 Views
- 3 replies
- 4 kudos
- 4 kudos
Databricks runs on linux servers. you can launch windows cmd or windows shell commands in databricks. you can run only linux commands.
- 4 kudos
- 2088 Views
- 3 replies
- 4 kudos
- 2088 Views
- 3 replies
- 4 kudos
- 5376 Views
- 4 replies
- 3 kudos
Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character
Though there are parquet files in the folder which I'm trying to query, I'm getting FileNotFoundException. Please Let me know if there is any issue in the code.Thanks
- 5376 Views
- 4 replies
- 3 kudos
- 3 kudos
alternatively you can use this as shell command. This will work.%sh ls /dbfs/mnt/mountpoint/path/to/raw/*.parquet
- 3 kudos
- 1107 Views
- 3 replies
- 2 kudos
Reward point for lakehouse fundamentals accreditation V2 and databricks community group join
Team, I am unable to find the points at my account yet for the subjected mentioned.Please let me know when it would be reflected on my side.Thanks in advance!​
- 1107 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @buvaneswari V​ Just a friendly follow-up, has your issue been resolved? Please let me know if you need any further assistance.Thanks and Regards
- 2 kudos
- 1144 Views
- 1 replies
- 0 kudos
User logging in Databricks
Hello experts. Is there a way to see user logs as who is running which notebook or a specific query? I couldn't find anything in "Driver logs" (stdout, log4j). Just to specify that I have admin role in the workspace. Thank you in advance
- 1144 Views
- 1 replies
- 0 kudos
- 0 kudos
Any update on this? Thank you
- 0 kudos
- 1165 Views
- 2 replies
- 3 kudos
AWS EC2 launched from Databricks tenancy
Hi,I was checking the EC2 details in our AWS account, and found all the EC2's launched from Databricks are with "dedicated" tenancy. I double checked the cluster launch configuration and didn't find anywhere to change the tenancy setting. How can we ...
- 1165 Views
- 2 replies
- 3 kudos
- 1841 Views
- 3 replies
- 2 kudos
sharikrishna26.medium.com
Spark Dataframes SchemaSchema inference is not reliable.We have the following problems in schema inference:Automatic inferring of schema is often incorrectInferring schema is additional work for Spark, and it takes some extra timeSchema inference is ...
- 1841 Views
- 3 replies
- 2 kudos
- 2 kudos
one other difference between those 2 approaches is that In Schema DDL String approach we use STRING, INT etc.. But In Struct Type Object approach we can only use Spark datatypes such as StringType(), IntegerType(), etc..
- 2 kudos
- 2575 Views
- 2 replies
- 4 kudos
Resolved! Why CTE is having issues with Databricks here?
df = spark.createDataFrame([(2018,'Apple1',45000),(2019,'Apple1',35000),(2020,'Apple1',75000), (2018,'Samsung',15000),(2019,'Samsung',20000),(2020,'Samsung',25000), (2018,'Nokia',21000),(2019,'Nokia',17000),(2020,'Nokia',140...
- 2575 Views
- 2 replies
- 4 kudos
- 4 kudos
Issue is coming because of the semicolon beside PhoneBrandSales. Try removing that ; issue will be resolved. Please refer to the screenshot below.Please select this answer as best answer if it resolved your issueThanks,Varshith
- 4 kudos
- 2513 Views
- 5 replies
- 2 kudos
Cannot sign in at databricks partner-academy portal
Hi, I've registered some days ago to partner-academy.databricks using my company ID but that day I didn't login. Now, I'm trying to login but it's impossible, I'm also trying to recover my password through the forgot password option but never get the...
- 2513 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Cristhian Plazas​ Thanks for reaching us!Kindly, mail to Nadiya to resolve the issue.
- 2 kudos
- 2124 Views
- 4 replies
- 6 kudos
Resolved! Databricks Lakehouse Fundamentals Badge not received
Hi, I have successfully passed the test after completion of the course but I have'nt recieved any badge from your side. I simply got a certificate from your side post the test completion. I need to post my credentials on Linkedin with valid verificat...
- 2124 Views
- 4 replies
- 6 kudos
- 6 kudos
Hi @Anirudh Negi​ Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 6 kudos
- 1464 Views
- 2 replies
- 2 kudos
Badge not received for Databricks Lakehouse Fundamentals Accreditation
I have successfully passed the test. But I have'nt recieved any badge or points, could you help me with this?The e-mail registered in the community is the same as that registered in the databricks academy and in credentials.databricks.com as well.Pdf...
- 1464 Views
- 2 replies
- 2 kudos
- 2 kudos
Please submit a ticket to Databricks Training Team here: https://help.databricks.com/s/contact-us?ReqType=training ,they will reach out you soon, maybe due to holiday they revert late,please be patient
- 2 kudos
- 16469 Views
- 10 replies
- 6 kudos
Resolved! Permanently add python file path to sys.path in Databricks
If your notebook is in different directory or subdirectory than python module, you cannot import it until you add it to the Python path.That means that even though all users are using the same module, but since they are all working from different rep...
- 16469 Views
- 10 replies
- 6 kudos
- 6 kudos
setting the `spark.executorEnv.PYTHONPATH` did not work for me. it looked like Spark/Databricks overwrite this somewhere. I used a simple python UDF to print some properties like `sys.path` and `os.environ` and didn't see the path I added.Finally, I ...
- 6 kudos
- 1855 Views
- 2 replies
- 4 kudos
Databricks Terraform - how to manage databricks entirely through Terraform?
I'm stuck at a point where I can't automatically set up everything about a databricks environment due to the fact that service principals can't be made an admin at the account level (accounts.azuredatabricks.net, similar for aws). Going into a bare t...
- 1855 Views
- 2 replies
- 4 kudos
- 4 kudos
Unfortunately there are still some limitations with doing IaC on Databricks with Terraform (ex. another one is that you can't setup KeyVault as a secret store with Service Principal).I think that instead of doing stuff manually, you can authenticate ...
- 4 kudos
- 1433 Views
- 2 replies
- 2 kudos
Map("skipRows", "1") ignored during autoloader process. Something wrong with the format?
I've tried multiple variations of the following code. It seems like the map parameters are being completely ignored. CREATE LIVE TABLE a_raw2 TBLPROPERTIES ("quality" = "bronze") AS SELECT * FROM cloud_files("dbfs:/mnt/c-raw/a/c_medcheck_export*.csv"...
- 1433 Views
- 2 replies
- 2 kudos
- 2054 Views
- 2 replies
- 2 kudos
"org.apache.hadoop.hive.ql.metadata.HiveException: at least one column must be specified for the table" non deterministic error in a `insert ... select ... ` clause
Hi, first of all thahks for your work in databricks sql.Unfortunately i am having a problem running insert-selects statements programatically using the jdbc driver.They all have the form:`insert into `mytable` select 1, 'foo', moreLiterals`The statem...
- 2054 Views
- 2 replies
- 2 kudos
- 2 kudos
thanks for the suggestion, could tell me more about how to check logs in the cluster?
- 2 kudos