- 4262 Views
- 5 replies
- 5 kudos
Resolved! Recommended ETL workflow for weekly ingestion of .sql.tz "database dumps" from Blob Storage into Unity Catalogue-enabled Metastore
The client receives data from a third party as weekly "datadumps" of a MySQL database copied into an Azure Blob Storage account container (I suspect this is done manually, I also suspect the changes between the approx 7GB files are very small). I nee...
- 4262 Views
- 5 replies
- 5 kudos
- 5 kudos
Hi @Sylvia VB​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 5 kudos
- 1877 Views
- 2 replies
- 0 kudos
Visualization from Python dataframe?
I notice it is very easily to get visualization from sql language inside Databricks. Say you run a SQL query which gives you a table, and you can easily use that table to do its visualization in terms of plots.How about in Python language when we hav...
- 1877 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @THIAM HUAT TAN​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 10410 Views
- 4 replies
- 3 kudos
Resolved! Unable to use CX_Oracle library in notebook
While using cx_oracle python library, it returns the below error:error message: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directoryThe cx_oracle library is dependent on native librari...
- 10410 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Manoj Ashvin​ can you use the below init script and try ? dbutils.fs.put("dbfs:/databricks/oracleTest/oracle_ctl_new.sh",""" #!/bin/bash sudo apt-get install libaio1 wget --quiet -O /tmp/instantclient-basiclite-linuxx64.zip https://download.ora...
- 3 kudos
- 765 Views
- 1 replies
- 0 kudos
- 765 Views
- 1 replies
- 0 kudos
- 0 kudos
from an expert -tldr is that we want to support the critical mass of common charts ootb -- a user shouldn't have to import a new library to use something super common & should be able to benefit from DB provided capabilities. that being said, we know...
- 0 kudos
- 1544 Views
- 1 replies
- 0 kudos
Unable to install kneed library in cluster with DBR version 5.5 LTS
I have an issue to install and use kneed python libary. https://pypi.org/project/kneed/I can install it and check it from log.[Install command]%shpip install kneed[log]Installing collected packages: kneedSuccessfully installed kneed-0.7.0but when I c...
- 1544 Views
- 1 replies
- 0 kudos
- 0 kudos
The kneed library has a dependency and we need to install them as well in order to work:numpy==1.18scipy==1.1.0scikit-learn==0.21.3Once we install the above libraries using GUI, we can run the below command to check the installed library with the cor...
- 0 kudos
- 2781 Views
- 1 replies
- 0 kudos
Unable to construct the sql url as the password is having special characters.
while using the sqlalchemy, unable to connect with sql server from databricks:user='user@host.mysql.database.azure.com' password='P@test' host="host.mysql.database.azure.com" database = "db" connect_args={'ssl':{'fake_flag_to_enable_tls': True}} conn...
- 2781 Views
- 1 replies
- 0 kudos
- 0 kudos
We can use urllib.parse to handle special characters. Here is an example:import urllib.parse user='user@host.mysql.database.azure.com' password=urllib.parse.quote_plus("P@test") host="host.mysql.database.azure.com" database = "db" connect_args={'...
- 0 kudos