- 3071 Views
- 1 replies
- 0 kudos
Github "Danger Zone"
There is a "Danger zone" appearing in the Github indicating that the repositories I share on Databricks should be Suspended and possibly that Databricks should be uninstalled. This may be something standard in Github. Has anyone run across it?
- 3071 Views
- 1 replies
- 0 kudos
- 0 kudos
It's not telling you that you should do these things. It's telling you that you may break stuff by doing these things. Yes the "Danger Zone" is a thing on Github, it tries to warn you before you do things like click to delete a repo.
- 0 kudos
- 1159 Views
- 0 replies
- 0 kudos
Notebook Langchain ModuleNotFoundError Problem
Specific line of code:from langchain.retrievers.merger_retriever import MergerRetrieverAll other langchain import works when this is commented out. Same line works on my local VS Code.Appreciate any help with this issue. Thanks!
- 1159 Views
- 0 replies
- 0 kudos
- 1646 Views
- 0 replies
- 0 kudos
Connecting to databricks ipython kernel from VSCode
I'd like to run python notebooks (.ipynb) from VSCode connecting to the ipython kernel from databricks. I have already connected to an execution cluster from VSCode and am able to run python scripts (.py files) and see the output on my local console....
- 1646 Views
- 0 replies
- 0 kudos
- 3720 Views
- 0 replies
- 0 kudos
Update values in DataFrame with values from another DataFrame
Hi, I have to data sources: "admin table" which contains active and inactive employees and I have "hr table" which contains only active employees.I need to update admin table with data from hr table:You can see that I need to update employee with num...
- 3720 Views
- 0 replies
- 0 kudos
- 1887 Views
- 0 replies
- 1 kudos
Unable to change/cast column datatype using Delta IO
I have created a delta table using Delta IO library, with following detailsTable Name: EmployeeColumns {Id Integer, name String, gender String, Salary decimal(5,2)}Now I want to upcast the salary from decimal(5,2) to decimal(10,4). If I use delta IO ...
- 1887 Views
- 0 replies
- 1 kudos
- 1860 Views
- 0 replies
- 1 kudos
How to switch from free trial to Community Edition of Databricks in my Azure workspace?
hello,I am trying to switch into Databricks Community Edition after a 14 day trial. I was able to register, but when I try to start my new cluster, I get an error message, "Cluster start feature is currently disabled, and the cluster does not run".In...
- 1860 Views
- 0 replies
- 1 kudos
- 7857 Views
- 3 replies
- 3 kudos
Resolved! Databricks community edition still available?
Is Databricks platform still available in its Community edition (outside Azure, AWS or GCP)? Can someone share the updated link?Thanks,Elisa
- 7857 Views
- 3 replies
- 3 kudos
- 3 kudos
Look : it is still available but afaik always linked to azure/gcp/aws
- 3 kudos
- 21799 Views
- 1 replies
- 1 kudos
Resolved! Notebook Langchain ModuleNotFoundError: No module named 'langchain.retrievers.merger_retriever'
Hi,As mentioned in the title, receiving this error despite%pip install --upgrade langchainSpecific line of code:from langchain.retrievers.merger_retriever import MergerRetriever All other langchain import works when this is commented out. Same line w...
- 21799 Views
- 1 replies
- 1 kudos
- 1 kudos
More specifically: langchain releases a new update every few days, and it is likely that you are using code or a library that needs a later version of langchain than you have (or, perhaps, a later version that removed whatever part of langchain you r...
- 1 kudos
- 2657 Views
- 2 replies
- 1 kudos
Databricks notebook execution using only one task
I am running a databricks notebook. While running, i only see one task on one worker getting started. My cluster has min 6 workers but seems like they are not getting used.I am performing a read operation from Cosmos DB.Can someone please help me her...
- 2657 Views
- 2 replies
- 1 kudos
- 1 kudos
If your code does not use Spark, it will not use any machines except the driver. If you're using Spark but your source data that you operate on has 1 partition, there will be only 1 task. Hard to say more without knowing what you are doing in more de...
- 1 kudos
- 6436 Views
- 2 replies
- 1 kudos
MIT License and Fine-tuning
Some questions related to fine-tuning and the MIT License, I read the MIT license but still confusing about some points.If I fine-tune the Dolly-v2 model, say using LoRA and my own dataset,Do I "own" the fine-tuned model?Am I allow to change the name...
- 6436 Views
- 2 replies
- 1 kudos
- 1 kudos
I am not sure I agree with the discussion so far. While none of here are lawyers, I think it's fairly straightforward to reason about the licensing.You have created a combined, derivative work from the Dolly weights in this case. You have copyright i...
- 1 kudos
- 1837 Views
- 0 replies
- 0 kudos
Databricks Rstudio Init Script Deprecated
OK so I'm trying to use Open Source Rstudio on Azure Databricks.I'm following the instructions here: https://learn.microsoft.com/en-us/azure/databricks/sparkr/rstudio#install-rstudio-server-open-source-editionI've installed the necessary init script ...
- 1837 Views
- 0 replies
- 0 kudos
- 2221 Views
- 2 replies
- 1 kudos
Getting FileNotFoundException while using cloudFiles
Hi,Following is the code i am using the ingest the data incrementally (weekly).val ssdf = spark.readStream.schema(schema) .format("cloudFiles").option("cloudFiles.format", "parquet").load(sourceUrl).filter(criteriaFilter)val transformedDf = ssdf.tran...
- 2221 Views
- 2 replies
- 1 kudos
- 9806 Views
- 6 replies
- 3 kudos
Resolved! Using databricks for end to end flow? rather than using ADF for extracting data
Currently, in our company we are using ADF+DATABRICKS for all batch integration. Using ADF first data is copied to ADLS gen 2 (from different sources like on prem servers, ftp solution file sharing, etc), then it is reformatted to csv and it is copie...
- 9806 Views
- 6 replies
- 3 kudos
- 3 kudos
@-werners- Is there any benefit of doing the extract part in databricks itself? Unlike our current architecture, where we first load to adls using adf. I guess it is worth doing all end to end using databricks if there is better processing, lower lat...
- 3 kudos
- 1746 Views
- 1 replies
- 0 kudos
RBAC, Security & Privacy controls
Could you please share us best practices on implementation of RBAC, Security & Privacy controls in Databricks
- 1746 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, Could you please check on https://docs.databricks.com/en/lakehouse-architecture/security-compliance-and-privacy/best-practices.html and see if this helping? Also, please tag @Debayan with your next comment which will notify me. Thanks!
- 0 kudos
- 2246 Views
- 0 replies
- 0 kudos
Records are missing while creating new dataframe from one big dataframe using filter
Hi,I have data in file like belowI have different types of row in my input file, column number 8 defines the type of the record.In the above file we have 4 types of records 00 to 03My requirement is:There will be multiple files in the source path, ea...
- 2246 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1139 | |
| 1027 | |
| 134 | |
| 57 | |
| 42 |
