Databricks

Frank · 09-26-2022

we have an application that takes in raw metrics data like key-value pairs. then we split them into four different table like below`key1, min, max, average`Those four tables are later used for dashboard. What are the design recommendations to this? S...

Frank · 09-26-2022

We have about 12k write/s and 1.5TB/mo compressed S3 data. How can we choose between Serverless vs managed? And what will be good way to project the cost? In serverless, how the machine and hours scaled or scheduled based on the load? If there is a l...

Frank · 09-21-2022

We tried to insert records to Delta table using ORM. It looks like only SQLAlchemy has option to connect to Delta table.We tried the following codefrom sqlalchemy import Column, String, DateTime, Integer, create_engine engine = create_engine("data...

Frank · 09-26-2022

Thanks Ryan for your help and inputs.When you are reading the data, where are you reading it from? Application or Databricks?We got the data from machine learning machines. They will generate the metrics at a constant rate.We need to put those data s...

Frank · 09-26-2022

Hi RyanHow come the Redash can use Flask and SQLAlchemy ORM for Delta Lake?We can use SQL for now but it will be good to know which way is better? The SQL connector is super slow for some reason. 1. Is there other data ingestion method you recommen...

Frank · 09-23-2022

I tried to install the JDBC driver on MAC OS. I am using Python code. Can I still use this driver? Is there a doc on how to install it on Mac? “DatabricksJDBC42.jar” cannot be opened because it is from an unidentified developer.

Frank · 09-23-2022

Hi Ryan Yes the item 2 Python SQL Connector is the first thing I tried before I was asked to explore on the ORM option. So if the SQL connector works, I still need to install the Simba driver? Do I need to install this driver in my local server where...

Frank · 09-22-2022

Below are some examples. There is a library also for that. https://github.com/dropbox/PyHive/issues/305https://dev.to/fruiza/build-a-python-flask-api-for-your-delta-lake-3bgfI guess we are open to other ways. This is only related with the ingestion d...

Databricks

User Stats

User Activity

Design Question

Serverless or Managed

SQLAlchemy ORM Connection String Error

Re: SQLAlchemy ORM Connection String Error

Re: SQLAlchemy ORM Connection String Error

Re: SQLAlchemy ORM Connection String Error

Re: SQLAlchemy ORM Connection String Error

Re: SQLAlchemy ORM Connection String Error