Hi,I'm trying to modify a delta table using following approach:Shallow clone of the table (source_table)Modification of the the clone (clonned_table)Deep clone of the modified table to the source table.Source delta table has 26 752 rows. Current Delt...
I have this error when I execute the Table ACL & Row and Column Level Security With Unity Catalog demo :Error in SQL statement: UnityCatalogServiceException: [RequestId=9644499b-3555-43d0-a2e7-0fdc29ea85ce ErrorClass=NOT_IMPLEMENTED.NOT_IMPLEMENTED] ...
There is a "Danger zone" appearing in the Github indicating that the repositories I share on Databricks should be Suspended and possibly that Databricks should be uninstalled. This may be something standard in Github. Has anyone run across it?
It's not telling you that you should do these things. It's telling you that you may break stuff by doing these things. Yes the "Danger Zone" is a thing on Github, it tries to warn you before you do things like click to delete a repo.
Specific line of code:from langchain.retrievers.merger_retriever import MergerRetrieverAll other langchain import works when this is commented out. Same line works on my local VS Code.Appreciate any help with this issue. Thanks!
I'd like to run python notebooks (.ipynb) from VSCode connecting to the ipython kernel from databricks. I have already connected to an execution cluster from VSCode and am able to run python scripts (.py files) and see the output on my local console....
Hi, I have to data sources: "admin table" which contains active and inactive employees and I have "hr table" which contains only active employees.I need to update admin table with data from hr table:You can see that I need to update employee with num...
I have created a delta table using Delta IO library, with following detailsTable Name: EmployeeColumns {Id Integer, name String, gender String, Salary decimal(5,2)}Now I want to upcast the salary from decimal(5,2) to decimal(10,4). If I use delta IO ...
hello,I am trying to switch into Databricks Community Edition after a 14 day trial. I was able to register, but when I try to start my new cluster, I get an error message, "Cluster start feature is currently disabled, and the cluster does not run".In...
Is Databricks platform still available in its Community edition (outside Azure, AWS or GCP)? Can someone share the updated link?Thanks,Elisa
Look : it is still available but afaik always linked to azure/gcp/aws
Hi,As mentioned in the title, receiving this error despite%pip install --upgrade langchainSpecific line of code:from langchain.retrievers.merger_retriever import MergerRetriever All other langchain import works when this is commented out. Same line w...
More specifically: langchain releases a new update every few days, and it is likely that you are using code or a library that needs a later version of langchain than you have (or, perhaps, a later version that removed whatever part of langchain you r...
I am running a databricks notebook. While running, i only see one task on one worker getting started. My cluster has min 6 workers but seems like they are not getting used.I am performing a read operation from Cosmos DB.Can someone please help me her...
If your code does not use Spark, it will not use any machines except the driver. If you're using Spark but your source data that you operate on has 1 partition, there will be only 1 task. Hard to say more without knowing what you are doing in more de...
Some questions related to fine-tuning and the MIT License, I read the MIT license but still confusing about some points.If I fine-tune the Dolly-v2 model, say using LoRA and my own dataset,Do I "own" the fine-tuned model?Am I allow to change the name...
I am not sure I agree with the discussion so far. While none of here are lawyers, I think it's fairly straightforward to reason about the licensing.You have created a combined, derivative work from the Dolly weights in this case. You have copyright i...
OK so I'm trying to use Open Source Rstudio on Azure Databricks.I'm following the instructions here: https://learn.microsoft.com/en-us/azure/databricks/sparkr/rstudio#install-rstudio-server-open-source-editionI've installed the necessary init script ...
Hi,Following is the code i am using the ingest the data incrementally (weekly).val ssdf = spark.readStream.schema(schema) .format("cloudFiles").option("cloudFiles.format", "parquet").load(sourceUrl).filter(criteriaFilter)val transformedDf = ssdf.tran...
Currently, in our company we are using ADF+DATABRICKS for all batch integration. Using ADF first data is copied to ADLS gen 2 (from different sources like on prem servers, ftp solution file sharing, etc), then it is reformatted to csv and it is copie...
@-werners- Is there any benefit of doing the extract part in databricks itself? Unlike our current architecture, where we first load to adls using adf. I guess it is worth doing all end to end using databricks if there is better processing, lower lat...
