topic How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data is coming from SAP in Data Engineering

How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data is coming from SAP

amruth — Thu, 02 Mar 2023 11:12:24 GMT

I am not using delta tables my data is from SAP ..how do i retrieve timestamp(history) dynamically from SAP table using databricks SQL

Re: How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data is coming from SAP

-werners- — Thu, 02 Mar 2023 12:22:59 GMT

This depends on what database SAP is running.

First of all databricks SQL needs to have access to the SAP database (HANA, Oracle, DB2, ...)

To query changes to these tables, table logging has to be enabled OR you need to have access to the rollback log (or transaction log or ...).

Chances are small that you are allowed to run such a query.

But perhaps you mean something else with timestamp data?

Re: How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data is coming from SAP

Anonymous — Fri, 17 Mar 2023 05:37:23 GMT

Hi @amruth D

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data i

Dribka — Sun, 26 Nov 2023 11:15:16 GMT

@amruth If you're working with data from SAP in Databricks and want to retrieve timestamps dynamically from a SAP table, you can utilize Databricks SQL to achieve this. You'll need to identify the specific SAP table that contains the timestamp or historical data. Once you've identified the table, you can construct a SQL query in Databricks that includes the appropriate columns and conditions to retrieve the timestamp dynamically. Consider using the SELECT statement to fetch the relevant timestamp column and apply any necessary filters based on your requirements. It's crucial to understand the structure of your SAP table and the naming conventions used for timestamp columns. If there's a specific column indicating historical data, you can use it in your query to retrieve timestamps dynamically. Ensure your Databricks environment is properly configured to connect to your SAP data source, and test your SQL query iteratively to ensure accurate results.

Re: How do i retrieve timestamp data from history in databricks sql not using DELTA table,its data i

felixdmeshio — Tue, 17 Dec 2024 09:24:35 GMT

Hello,

If you’re trying to bring timestamp data or any other SAP Table from SAP (SAP HANA) into Databricks, our SAP HANA to Databricks Connector can help streamline this process. The connector enables you to extract data directly from SAP HANA tables and load it iteratively into Databricks for further processing.

The key benefits are:

Incremental Data Transfer: Particularly helpful for managing very large datasets.
Simple Usage: No complex configurations required.

With the connector, integrating your timestamped data should work seamlessly.

For more details, including implementation steps, please refer to our official documentation. If you need further support, don’t hesitate to contact us at info@dmesh.io.

I hope this helps you integrate SAP data into Databricks!