- 1302 Views
- 1 replies
- 0 kudos
Migration to new Tools
Has anyone else taken a migration journey from SAS to a different language/platform?? Love to hear about your journey…
- 1302 Views
- 1 replies
- 0 kudos
- 0 kudos
@cannonball111 Yes, we have done using Travinto Technologies tool
- 0 kudos
- 2538 Views
- 1 replies
- 0 kudos
Use DataBricks migration tool to export query
Dear all,I tried to use Databricks migration tool (https://github.com/databrickslabs/migrate) to migrate objects from one Databricks instance to another. I realized that notebooks, clusters, jobs can be done but queries can not be migrated by this to...
- 2538 Views
- 1 replies
- 0 kudos
- 0 kudos
@yjiao If you're planning to migrate from your current technology to Databricks, Travinto Technologies' Code Converter Tool is here to make the process seamless. This powerful tool enables you to migrate data, ETL workflows, and reports across platf...
- 0 kudos
- 4827 Views
- 1 replies
- 0 kudos
URGENT - Databricks workspace deletion & recovery
Hi Team,I accidentally deleted our databricks workspace, which had all our artefacts and control plane, and was the primary resource for our team's working environment.Could anyone please help on priority, regarding the recovery/ restoration mechanis...
- 4827 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello data-warriors Note: Users cannot recover the deleted Databricks instance directly from Azure Portal. Deleted databricks instance can be only recovered by opening a support ticket, where our core engineering team will help you to recover the Da...
- 0 kudos
- 3219 Views
- 1 replies
- 0 kudos
Unity catalog Migration
Hi Team,Could you please help me to understand, 1)Why we need to migrate Unity catalog? if we are not migrating what benefits we will not get?2) How to migrate Unity catalog (What all are objects needs to migrate and any tool) ? Regards,Phanindra
- 3219 Views
- 1 replies
- 0 kudos
- 0 kudos
@Phani1 if you're planning to migrate from your current technology to Databricks, Travinto Technologies' Code Converter Tool is here to make the process seamless. This powerful tool enables you to migrate data, ETL workflows, and reports across platf...
- 0 kudos
- 2268 Views
- 2 replies
- 0 kudos
Disable mlflow autologging in a helper notebook
We have a helper function that uses a sklearn estimator. We don't want to to be logged to mlflow.I can dodef myfunc(): import mlflow with mlflow.autolog.ignore: # train model # use model return predictionsBut I get info prints:...
- 2268 Views
- 2 replies
- 0 kudos
- 0 kudos
mlflow.autolog(disable=True, silent=True) fixes the printing. But my other problem with setting autologging back to previous state is still unsolved. I can't find any information about that problem in the docs.
- 0 kudos
- 1970 Views
- 1 replies
- 1 kudos
end-to-end lineage using Unity Catalog
Hi Team,what is the Possibilities of getting end-to-end lineage using Unity CatalogRegards,Phanindra
- 1970 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello Phanindra, Inside Databricks ecosystem, you can easily get end-to-end lineage. The data lineage will be build automatically as you ingest and transform data inside the lakehouse. Here is a technical view on how this is achieved: On the other ha...
- 1 kudos
- 2603 Views
- 1 replies
- 0 kudos
Data Masking issue
Hi Team,After successfully creating a group and populating it with members, I am facing the problem of the is_member() function always returning False, despite the user being part of the group. This issue is specific to my sandbox Databricks Account....
- 2603 Views
- 1 replies
- 0 kudos
- 0 kudos
Can you post your code? I use the 'is_account_group_member' in my code as follows. Also note there is a alternative to Dynamic Views that is preferred. It's named 'Row filtering and Column masking'. Go here to learn more: https://docs.databricks...
- 0 kudos
- 4173 Views
- 6 replies
- 0 kudos
databricks asset bundle error-terraform.exe": file does not exist
Hi,I am getting below error while I am deploying databricks bundle using azure devops release 2024-07-07T03:55:51.1199594Z Error: terraform init: exec: "xxxx\\.databricks\\bundle\\dev\\terraform\\xxxx\\.databricks\\bundle\\dev\\bin\\terraform.exe": ...
- 4173 Views
- 6 replies
- 0 kudos
- 0 kudos
Using Git Bash for bash execution and to setup variables we did this by going to control panel -> system -> environment varibales
- 0 kudos
- 3401 Views
- 3 replies
- 0 kudos
Foreign table to delta streaming table
I want to copy a table from a foreign catalog as my streaming table. This is the code I used but I am getting error: Table table_name does not support either micro-batch or continuous scan.; spark.readStream .table(table_name) ...
- 3401 Views
- 3 replies
- 0 kudos
- 0 kudos
What is the underlying type of the table you are trying to stream from? Structured Streaming does not currently support streaming reads via JDBC, so reading from MySQL, Postgres, etc are not supported. If you are trying to perform stream ingestion fr...
- 0 kudos
- 2455 Views
- 1 replies
- 0 kudos
cross join issue generating surrogate keys in delta table
I used below code to populate target table, it is working as expected but expect for surrogatekey column. After I inserted dummy entry -1 and did run merge code, it is generating the numbers in surrogatekey column like this 1,3,5,7,9 odd numbers, it ...
- 2455 Views
- 1 replies
- 0 kudos
- 0 kudos
This is expected. Identity columns can have gaps, or per the documentation: Values assigned by identity columns are unique and increment in the direction of the specified step, and in multiples of the specified step size, but are not guaranteed to b...
- 0 kudos
- 2143 Views
- 4 replies
- 2 kudos
Expose Databricks Cluster Logs to Grafana
I am trying to send Databricks cluster logs to Grafana using an init_script where I define the system journals to consume. The issue I am facing is that I cannot get the driver logs, standard error, and output to reach Grafana. Is there something spe...
- 2143 Views
- 4 replies
- 2 kudos
- 1512 Views
- 3 replies
- 0 kudos
Certification exam got suspended
Hello Team,My Data engineer associate exam got suspended within seconds of starting without any reason. After starting the exam screen got paused just after 10-20 seconds there was a notice that someone will contact you but no one contacted till the ...
- 1512 Views
- 3 replies
- 0 kudos
- 0 kudos
You need to raise a ticket with Databricks to get to a resolution: https://help.databricks.com/s/contact-us?ReqType=training%22%20%5Ct%20%22_blank
- 0 kudos
- 1583 Views
- 1 replies
- 0 kudos
Liquid Clustering
How can I use column for liquid clustering that is not in first 32 column of my delta table schema.
- 1583 Views
- 1 replies
- 0 kudos
- 0 kudos
We can only specify columns with statistics collected for clustering keys. By default, the first 32 columns in a Delta table have statistics collected. See Specify Delta statistics columns. We can use the below workaround for your use case: 1. Use th...
- 0 kudos
- 1736 Views
- 1 replies
- 0 kudos
Databricks migration
How easy to migrate from snowflake or redshift to Databricks ?
- 1736 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @RajPutta : it very easy to migrate any thing to Databrick only thing required is your team knowledge on databrick platform . below steps is important.DiscoveryAssessment Code conversion and Migration is very importPoCif you want to migrate fro...
- 0 kudos
- 5412 Views
- 2 replies
- 2 kudos
c# connector for databricks Delta Sharing
Any plans for adding c# connector? What are alternate ways in current state?
- 5412 Views
- 2 replies
- 2 kudos
- 2 kudos
I'm having problems getting the REST API calls for Delta Sharing to work. Python and Power BI work fine but the C# code that Databricks AI generates does not work. I keep getting an "ENDPOINT NOT FOUND" error even though config.share is fine.A C# con...
- 2 kudos
