Talend ETL code to Databricks

[Phani1]

Hi Team,

What is the best way to transfer Talend ETL code to Databricks and what are the best methods/practices for migrating Talend ETL's to Databricks (notebook, code conversion/migration strategy, workflow's etc)?

Regards,

Janga

[-werners-]

I am not aware of any migration path for Talend to Databricks.  They can be integrated but migrating from one to the other is likely a 'manual migration' aka redo everything.
But I would definitely check with your Databricks contact person, perhaps they have some tools/ideas to aid in the migration.

[Simon_e]

You can have a look at BladeBridge converters. They support Talend to Databricks route. 
www.bladebridge.com

[thelogicplus]

@Simon_e  try to use travinto.com  . Their product accuracy is more than 90% And most of the SI and GSI are using their tool. We have migrated 200K + talend , datastage ,abi initio , informatica jobs  with 90%+  conversion accuracy, I will highly recommend them. Awesome product and awesome people.

We have tried bridge product as well as but bridge product conversion was not accurate and lot of manual work needed to fix, but when we used travinto Technologies product that was awesome with minimal manual work with 90% of accuracy.

[thelogicplus]

if you're planning to migrate from your current technology to Databricks, Travinto Technologies' Code Converter Tool is here to make the process seamless. This powerful tool enables you to migrate data, ETL workflows, and reports across platforms effortlessly. we’ve successfully migrated Datastate, talend, informatica,abi inition and many more  to Databricks for numerous customers, using travinto technologies tools. With over 50,000 adapters in their toolkit, Travinto Technologies can handle migrations across virtually any platform, ensuring compatibility and efficiency. Whether you're dealing with databases, analytics tools, or reporting platforms

[vamsi_simbus]

Hi @Phani1 ,

Were you able to complete the migration from Talend to Databricks? If yes, please do let me know the process / tools that you have used for the migration and the challenges that you have encountered.

Thanks

Vamsi.

[chennakrishna]

why don't you try using databricks Assistant to build the code for you? 

Apple to Apple conversion not possible with any tool in the market.  They can do for simple jobs. if jobs is complex they will say we will do customization. 

Either way using any AI chatbot to build code for you or any conversion tool. Have to do deep dive after conversion. 

[thelogicplus]

@chennakrishna you can check Travinto Technology tool which is best in tool conversion and No #1 in ETL code conversion . also if you want to automate by yourself try Talend to databricks migration guide  from theLogicplus.com

[pradeep_singh]

Bladebridge had a support for Talend to Databricks conversion but since its acquired by databricks and rebranded as Lakebridge we dont have that support anymore ( Lakebridge for some reason doesn support Talend ). You might want to contact your Databricks representative (Solution Architect or Account executive ) to see if they can help you get some support . They should have a way of making the convertor work with some customization . 

Untill then you can start with exporting your talend jobs as xml files and then using claude code to convert these xmls to databricks pyspark or SQL notebooks . You will have to keep context engineering in mind if your jobs are huge( summarize your jobs and then target individual blocks for conversion instead of converting it all at once) . If you want to learn how to use databricks hosted endpoints from for claude code you can find the details here . 
https://medium.com/@dbxdev/turning-databricks-into-an-ai-pair-programmer-with-claude-powered-coding-...