- 1336 Views
- 2 replies
- 3 kudos
Resolved! Access Databricks Table with Simple Python3 Script
Hi, I'm super new to Databricks. I'm trying to do a little API scripting against my company's DB instance.I have this supersimple python (ver 3) which is meant to run a remote host. The script tries to a simple SQL query against my Databricks instan...
- 1336 Views
- 2 replies
- 3 kudos
- 3 kudos
@gchandra Yes! This is the documentation I was seeking! Thank you so much
- 3 kudos
- 1570 Views
- 3 replies
- 2 kudos
What is the Best Postman Alternative?
Hey guys, I have been using Postman for quite some time now and getting disappointed recently and want to make a swtich. Is there something better than Postman? I've heard about that APIDog is much easier to use with a much better UI, and support all...
- 1570 Views
- 3 replies
- 2 kudos
- 615 Views
- 1 replies
- 0 kudos
incremental loads without date column
Hi All,We are facing a situation where our data source is Snowflake, and the data is saved in a storage location(adls) in parquet format. However, the tables or data lack a date column or any incremental column for performing incremental loads to Dat...
- 615 Views
- 1 replies
- 0 kudos
- 0 kudos
Ideally you would have some change tracking system (cdc f.e.) on the source tables (Streams in the case of Snowflake, Introduction to Streams | Snowflake Documentation).But that is not the case.So I think you approach is ok. You cannot track what is...
- 0 kudos
- 943 Views
- 7 replies
- 1 kudos
Databricks bundle
Hey, I am new to Databricks, and I am trying to test the mlops-stack bundle. Within that bundle there is a feature-engineering workflow and I have a problem to make it run. The main problem is the following.the bundle specified the target to be $bund...
- 943 Views
- 7 replies
- 1 kudos
- 508 Views
- 1 replies
- 0 kudos
Oracle -> Oracle Golden Gate ->Databricks Delta lake
Hi All,We have a situation where we are collecting data from different Oracle instances.The customer is using Oracle GoldenGate to replicate this data into a storage location.From there, we can use Auto Loader or Delta Live Tables to read Avro files ...
- 508 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Phani1 ,In my opinion this is really good setup. You have push scenario where Oracle GoldenGate is responsible for delivering data into storage, so you don't have to bother about extraction part. And autoloader is the best choice when it comes t...
- 0 kudos
- 181 Views
- 0 replies
- 0 kudos
Delta Lake to Oracle Essbase
Hi All,How can we connect Databricks Delta Lake to Essbase in OCI? We know that Essbase supports JDBC/ODBC. Is it possible to use Python or PySpark to read from Delta Lake and load the data into Essbase? I think using JDBC/ODBC might affect performan...
- 181 Views
- 0 replies
- 0 kudos
- 403 Views
- 0 replies
- 0 kudos
Denodo Connection Parameters.
Hi All,We are establishing a connection from Denodo to Databricks. During the development phase, we utilized a personal access token associated with developer account. However, this approach is not considered a best practice for production environm...
- 403 Views
- 0 replies
- 0 kudos
- 907 Views
- 2 replies
- 1 kudos
Google PubSub for DLT - Error
I'm trying to create a delta live table from a Google PubSub stream.Unfortunately I'm getting the following error:org.apache.spark.sql.streaming.StreamingQueryException: [PS_FETCH_RETRY_EXCEPTION] Task in pubsub fetch stage cannot be retried. Partiti...
- 907 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Retired_mod, it was indeed a permissions issue. Nevertheless, I must admit that the error message is slightly misleading.Thanks.
- 1 kudos
- 1310 Views
- 2 replies
- 0 kudos
Restrict access of user/entity to hitting only specific Databricks Rest APIs
Hi community,Assume I generate a personal access token for an entity. Post generation, can I restrict the access of the entity to specific REST APIs? In other words, consider this example where once I use generate the token and setup a bearer token b...
- 1310 Views
- 2 replies
- 0 kudos
- 0 kudos
@Surajv You have to rely on access control settings on resources and entities (users or service principals or create some cluster policies), rather than directly restricting the API endpoints at the token level.Note: API access based on fine-grained ...
- 0 kudos
- 7328 Views
- 3 replies
- 0 kudos
How list all USERs FROM a especific GROUP USING SQL?
I want to list, using sql editor, all users name from a specific group.Reading documentation, I only learned how to show the groups or the users, using simples filters, like:SHOW GROUPS LIKE '*XPTO*';SHOW GROUPS WITH USER `test@gmail.com`SHOW USERS L...
- 7328 Views
- 3 replies
- 0 kudos
- 0 kudos
I don't think it's possible yet. Unfortunately, I look in all system tables and command and didn't found this kind of things.But with a Python notebook, like what did the AI, you can reconstruct it:first you list all the users withSHOW USERSthen you ...
- 0 kudos
- 919 Views
- 2 replies
- 0 kudos
Resolved! Databricks Certified Associate Developer for Apache Spark 3 Got Suspended. Require Immediate support
Hello @Cert-Team @Certificate Team,Request Id# 00544837I encountered a pathetic experience while attempting my Databricks Certified Associate Developer for Apache Spark 3 certification exam. I had answered more than 44 questions I applied for Data...
- 919 Views
- 2 replies
- 0 kudos
- 1575 Views
- 2 replies
- 1 kudos
Private Python Package in Serverless Job
I am trying to create a Databricks Job using Serverless Compute. I am using wheel file to run the Python Job.The wheel file has setup.py file using which all dependencies are installed. One of the package dependency is a private package hosted on Git...
- 1575 Views
- 2 replies
- 1 kudos
- 700 Views
- 1 replies
- 0 kudos
Unable to clone delta live table
Hello Team,In order to avoid any cost of running workflow for historical data, i am trying DEEP clone in order to copy data of streaming table from PROD workspace to QA workspace but it is giving below error. Please suggest some efficient data copy m...
- 700 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anish_2 ,This is well known limitation of DLT. You can read about it at limitation section in documentation. Conclusion is following. If you want to have ability to use all feature of delta protocol, it's better not to use dlt framework. It's gre...
- 0 kudos
- 3479 Views
- 2 replies
- 0 kudos
File size upload limit through CLI
Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.
- 3479 Views
- 2 replies
- 0 kudos
- 814 Views
- 2 replies
- 0 kudos
CLI is not helpful in exporting Error: expected to have the absolute path of the object or directory
I try to export a job as a DBA in order to create an Asset Bundle according to thishttps://community.databricks.com/t5/data-engineering/databricks-asset-bundle-dab-from-existing-workspace/td-p/49309I am on Windows 10 Pro x64 withDatabricks CLI v0.223...
- 814 Views
- 2 replies
- 0 kudos
- 0 kudos
To export an existing folder under /Workspace/... export-dir command could be used : databricks workspace export-dir /Workspace/Applications/ucx/logs/migrate-tables/run-123-0/ /Users/artem.sheiko/logs
- 0 kudos
