- 2024 Views
- 1 replies
- 0 kudos
- 2024 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Databricks_Work , Vacuum and Analzye are two separate commands that are used for optimizing the queries but they perform two different operations. Vacuum is used to clear the stale data files in your delta table. Vacuum should be run after a opti...
- 0 kudos
- 1839 Views
- 1 replies
- 1 kudos
Ifichangethetimestampformat from yyyy-MM-dd hh-mm-ss to MM-dd-yyyy hh-mm-ssinPostgrestableisthatfine
- 1839 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, You can check the function https://docs.databricks.com/en/sql/language-manual/functions/date_format.html, let us know if this helps.
- 1 kudos
- 3361 Views
- 1 replies
- 0 kudos
How to query sql warehouse tables with spark?
Hey there... I managed to query my data following this guide https://learn.microsoft.com/en-us/azure/databricks/dev-tools/python-sql-connectorusing databricks sql#!/usr/bin/env python3from databricks import sqlwith sql.connect(server_hostname = "adb-...
- 3361 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @mobe - Please refer to the github link for more examples - https://github.com/databricks/databricks-sql-python/blob/main/examples. Thanks,Shan
- 0 kudos
- 2681 Views
- 2 replies
- 1 kudos
Getting 'No GCP Marketplace token provided' error while signing up from GCP marketplace.
Hey guys,I was trying to sign up to the 14 day free trial from GCP marketplace. When I click 'SIGN UP WITH DATABRICKS', I get the error below.HTTP ERROR 401Problem accessing /sign-up. Reason: No GCP Marketplace token provided. Please start over fr...
- 2681 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks Walter,I have the IAM permissions in place and also have a valid billing account.However, I keep getting the same error regarding the missing Marketplace token. I am clicking the 'SIGN UP WITH DATABRICKS' button from the GCP UI, so am not sure...
- 1 kudos
- 1016 Views
- 2 replies
- 0 kudos
HELP opening notebook displays blank, creating new one gives and error and other issues
Hi,SituationI just literally started using data bricks. I created a workspace, a cluster and uploaded a notebook. But my workspace doesn’t seem to function correctly at the moment.I will attach what it looks like when I try to open a notebookopening ...
- 1016 Views
- 2 replies
- 0 kudos
- 0 kudos
UPDATEI have downloaded chrome and this does not happen for it as well
- 0 kudos
- 3376 Views
- 1 replies
- 0 kudos
PPT material or document from Databricks Learning
Hello Databricks Community,I am a beginner with Databricks. I am wondering if we can download power point slides or learning documents from the Databricks Learning Platform. I like to read after taking the online course. Could you let me know? Curren...
- 3376 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello Databricks Community,I am a beginner with Databricks. I am wondering if we can download power point slides or learning documents from the Databricks Learning Platform. I like to read after taking the online course. Could you let me know? Curren...
- 0 kudos
- 6851 Views
- 4 replies
- 0 kudos
Resolved! DatabaseError: (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER]
Hi,I am trying to connect my database through LLM and expecting to receive a description of the table and 1st 3 rows from the table. from langchain.agents import create_sql_agent from langchain.agents.agent_toolkits import SQLDatabaseToolkit from la...
- 6851 Views
- 4 replies
- 0 kudos
- 0 kudos
This is not databricks issue but from langchain. A PR has been raised to solve this: One workaround that worked is: https://github.com/langchain-ai/langchain/issues/11068 setting sample_rows_in_table_info to 0 when calling SQLDatabase.from_databricks...
- 0 kudos
- 7721 Views
- 11 replies
- 1 kudos
DLT Pipeline issue - Failed to read dataset .Dataset is not defined in the pipeline.
Background. I have created a DLT pipeline in which i am creating a Temorary table. There are 5 temporary tables as such. When i executed these in an independent notebook they all worked fine with DLT. Now i have merged this notebook ( keeping same ...
- 7721 Views
- 11 replies
- 1 kudos
- 1 kudos
I am sorry but information you are providing is not helping at all. Plase dump your code there.
- 1 kudos
- 5237 Views
- 0 replies
- 0 kudos
Data bricks for practice at no cost which cloud service or combination i need to use
Hi All Senior ,Context :I want to use databricks for practice to create projects and keep polishing my knowledge. My free credits are already used up . Now can you pls give me tips on how to run databricks in which cloud provider (storage account com...
- 5237 Views
- 0 replies
- 0 kudos
- 9068 Views
- 2 replies
- 0 kudos
Unlock Data Engineering Essentials in Just 90 Minutes - Get Certified for FREE!
There’s an increasing demand for data, analytics and AI talent in every industry. Start building your data engineering expertise with this self-paced course — and earn an industry-recognized Databricks certificate. This course provides four short tu...
- 9068 Views
- 2 replies
- 0 kudos
- 0 kudos
Same here. I am not able to download any Certificate even after passing the Quiz. But the Course link - https://www.databricks.com/learn/training/getting-started-with-data-engineering#data-videoclearly says: take a short knowledge test and earn a com...
- 0 kudos
- 687 Views
- 1 replies
- 0 kudos
Efficient Detection of Schema Mismatch in CSV Files During Single Pass Reading
Hello, when I read a CSV file with a schema object, if a column in the original CSV contains a value of a different datatype than specified in the schema, the result is a null cell. Is there an efficient way to identify these cases without having to ...
- 687 Views
- 1 replies
- 0 kudos
- 0 kudos
Maybe you can try to read the data and let AutoLoader move missmatch data e.g. to rescueColumnhttps://learn.microsoft.com/en-us/azure/databricks/ingestion/auto-loader/schema#--what-is-the-rescued-data-columnThen you can decide what you do with rescue...
- 0 kudos
- 6686 Views
- 5 replies
- 2 kudos
[Unity Catalog]-CosmosDB: Data source v2 are not supported
I've worked on azure databricks connected to azure cosmos. It works when my cluster is not enabling Unity Catalog (UC).But when I enable UC, it return error like below:AnalysisException: [UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION] The command(s...
- 6686 Views
- 5 replies
- 2 kudos
- 1870 Views
- 2 replies
- 1 kudos
dbutils.fs.ls versus pathlib.Path
Hello community members,The dbutils.fs.ls('/') exposes the distributed file system(DBFS) on the databricks cluster. Similary, the python library pathlib can also expose 4 files in the cluster like below:from pathlib import Pathmypath = Path('/')for i...
- 1870 Views
- 2 replies
- 1 kudos
- 1 kudos
I think it will be usefull if you look at this documentation to understand difrent files and how you can interact with them:https://learn.microsoft.com/en-us/azure/databricks/files/there is not much to say then that dbutils is "databricks code" that ...
- 1 kudos
- 4029 Views
- 5 replies
- 2 kudos
DLT Compute Resources - What Compute Is It???
Hi there, I'm wondering if someone can help me understand what compute resources DLT uses? It's not clear to me at all if it uses the last compute cluster I had been working on, or something else entirely.Can someone please help clarify this?
- 4029 Views
- 5 replies
- 2 kudos
- 2 kudos
Well, one thing they emphasize in the 'Adavanced Data Engineer' Training is that job-clusters will terminate within 5 minutes after a job is completed. So this could be in support of your theory to lower costs. I think job-cluster are actually design...
- 2 kudos
- 1577 Views
- 1 replies
- 0 kudos
python library in databricks
Hello community members,I am seeking to understand where databricks keeps all the python libraries ? For a start, I tried two lines below:import sys sys.path()This list all the paths but I cant look inside them. How is DBFS different from these paths...
- 1577 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello,all your libraries are installed on Databricks Cluster Driver node on OS Disk.DBFS is like mounted Cloude Storage account.You have veriouse ways of working with libraries but databricks only load some of libraries that comes with Cluster image....
- 0 kudos
