- 7669 Views
- 4 replies
- 7 kudos
Resolved! Architecture choice, streaming data
I have sensor data coming into Azure Event Hub and need some help in deciding how to best ingest it into the Data Lake and Delta Lake: Option 1: azure event hub > databricks structured streaming > delta lake (bronze) Option 2: azure event hub > e...
- 7669 Views
- 4 replies
- 7 kudos
- 7 kudos
If batch job is possible and you need to process data I would use probably:azure event hub from (events after previous job run) > databricks job process as dataframe > save df to delta lakeno streaming or capturing needed in that case
- 7 kudos
- 3138 Views
- 2 replies
- 1 kudos
- 3138 Views
- 2 replies
- 1 kudos
- 1 kudos
A great place to learn more about Databricks integrations with AWS services is https://www.databricks.com/aws There is information on this page regarding integrations with Glue, SageMaker, Redshift and others. Many of these pages also point to our bl...
- 1 kudos
- 14121 Views
- 3 replies
- 1 kudos
- 14121 Views
- 3 replies
- 1 kudos
- 1 kudos
Multiple writers (inserts/appends) present no problems with Delta. You can have two users appending data to a Delta table at the same time without issue. Updates, deletes, merges, and compaction can run into issues when multiple user are trying to d...
- 1 kudos
- 1727 Views
- 1 replies
- 0 kudos
- 1727 Views
- 1 replies
- 0 kudos
- 0 kudos
Not as of this comment, but our product team is aware of this request and are working on getting this implemented soon!Keep in mind that for feature requests and direct updates/communication from our product team, you can post here (or just vote up o...
- 0 kudos
- 2187 Views
- 1 replies
- 0 kudos
- 2187 Views
- 1 replies
- 0 kudos
- 0 kudos
The admin console exists within the workspace and let's you control access and privileges for that specific workspace. An existing admin can get to it from the drop down in the very top right and selecting Admin Console.The first screen you'll land o...
- 0 kudos
- 1308 Views
- 1 replies
- 0 kudos
Can we use SCIM to sync users/groups to Databricks from their on-prem Active Directory. Does Databricks support this?
Can we use SCIM to sync users/groups to Databricks from their on-prem Active Directory. Does Databricks support this?
- 1308 Views
- 1 replies
- 0 kudos
- 0 kudos
Anything that can reach the control plane and use the SCIM API should work. For Azure AD Premium, there is specifically an enterprise App that does this for the customer.
- 0 kudos
- 2148 Views
- 1 replies
- 1 kudos
- 2148 Views
- 1 replies
- 1 kudos
- 1 kudos
Ganglia metrics are available only if the job runs for more than 15 minutes. For jobs that are completed within 15 minutes, the metrics won't be available
- 1 kudos
- 2075 Views
- 1 replies
- 1 kudos
- 2075 Views
- 1 replies
- 1 kudos
- 1 kudos
As of June, 2021, No.However Public Preview features are stable and intended to advance to GA and fully supported by Databricks Support.
- 1 kudos
- 2059 Views
- 1 replies
- 0 kudos
Resolved! Databricks SQL dashboard refresh
In Databricks SQL, can you prohibit a dashboard from being refreshed?
- 2059 Views
- 1 replies
- 0 kudos
- 0 kudos
It looks like this can be done by not granting CAN_RUN to a user/grouphttps://docs.databricks.com/sql/user/security/access-control/dashboard-acl.html#dashboard-permissions
- 0 kudos
- 1141 Views
- 1 replies
- 0 kudos
- 1141 Views
- 1 replies
- 0 kudos
- 0 kudos
One way to increase the CIDR range id the ip list is available or create a complete different work space on same vpc with different subnets
- 0 kudos
- 4090 Views
- 1 replies
- 1 kudos
- 4090 Views
- 1 replies
- 1 kudos
- 1 kudos
Yes. if the on-premise is accessible over the network from the Databricks cluster, then it's possible to connect.
- 1 kudos
- 1261 Views
- 1 replies
- 0 kudos
Okta Integration
My company uses Okta as a SSO provider. Can I integrate Okta with Databricks for a SSO experience?
- 1261 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, okta is among the supported identity providers. Read more here : https://docs.databricks.com/administration-guide/users-groups/single-sign-on/index.html
- 0 kudos
- 1024 Views
- 1 replies
- 0 kudos
- 1024 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes. There is a property called dbus_per_hour that you can add to your cluster policy.See https://docs.databricks.com/administration-guide/clusters/policies.html#cluster-policy-virtual-attribute-pathsHere's an example policy that uses it:https://docs...
- 0 kudos
- 5391 Views
- 1 replies
- 0 kudos
Resolved! My cluster is running an init script, and I want to see what's going on.
After I log in to the workspace, where can I find the logs?
- 5391 Views
- 1 replies
- 0 kudos
- 0 kudos
See public docs: https://docs.databricks.com/clusters/init-scripts.html#cluster-scoped-init-script-logsDon't forget to enable cluster log delivery: https://docs.databricks.com/clusters/configure.html#cluster-log-deliveryNote that this only works for ...
- 0 kudos
- 2567 Views
- 1 replies
- 0 kudos
- 2567 Views
- 1 replies
- 0 kudos
- 0 kudos
Usually, your account URL is where you navigate to log in.
- 0 kudos
