- 3844 Views
- 8 replies
- 5 kudos
Hi, I am facing a problem that I hope to get some help to understand. I have created a function that is supposed to check if the input data already exist in a saved delta table and if not, it should create some calculations and append the new data to...
- 3844 Views
- 8 replies
- 5 kudos
Latest Reply
Hi,im also having similar issue ..does creating temp view and reading it again after saving to a table works?? /
7 More Replies
- 467 Views
- 1 replies
- 2 kudos
Hi, I am facing an issue where one of my jobs taking so long since certain time, previously its only needs less than 1 hour to run a batch job that load json data and do a truncate and load to a delta table, but since june 2nd, it become so long that...
- 467 Views
- 1 replies
- 2 kudos
Latest Reply
Hi @krisna math​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
by
az38
• New Contributor II
- 3199 Views
- 2 replies
- 3 kudos
Hi, community!How do you think what is the best way to load from Azure ADLS (actually, filesystem doesn't matter) into df onli files modified after some point in time?Is there any function like input_file_name() but for last_modified to use it in a w...
- 3199 Views
- 2 replies
- 3 kudos
Latest Reply
_metadata will provide file modification timestamp. I tried on dbfs but not sure for ADLS.https://docs.databricks.com/ingestion/file-metadata-column.html
1 More Replies
- 897 Views
- 3 replies
- 1 kudos
I am using DLT to load csv in ADLS, below is my sql query in notebook:CREATE OR REFRESH STREAMING LIVE TABLE test_account_raw
AS SELECT * FROM cloud_files(
"abfss://my_container@my_storageaccount.dfs.core.windows.net/test_csv/",
"csv",
map("h...
- 897 Views
- 3 replies
- 1 kudos
Latest Reply
thank you every one, the problem is resolved, problem is gone when I have workspace admin access.
2 More Replies
- 2012 Views
- 5 replies
- 4 kudos
let us say I already have the data 'TotalData'write.csv(TotalData,file='/tmp/TotalData.csv',row.names = FALSE)I do not see any error from abovewhen I list files below:%fs ls /tmpI do not see any files written there. Why?
- 2012 Views
- 5 replies
- 4 kudos
Latest Reply
Hi @THIAM HUAT TAN​ We haven’t heard from you since the last response from @Cedric Law Hing Ping​ , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to oth...
4 More Replies
- 943 Views
- 3 replies
- 3 kudos
Hello,I have created my table in Databricks, at this point everything is perfect i got the same value than in my CSV. for my column "Exposure" I have :0 0,00
1 0,00
2 0,00
3 0,00
4 0,00
...But when I load my fi...
- 943 Views
- 3 replies
- 3 kudos
Latest Reply
Hi @Anis Ben Salem​ ,How do you read your CSV file? do you use Pandas or Pyspark APIs? also, how do you created your table?could you share more details on the code you are trying to run?
2 More Replies
