Databricks

dataslicer · 10-31-2022

Using the Azure Databricks R Shiny example, I am easily able to instantiate a R Shiny web front session. The URL that is provided by Databricks provides the proxy session that allows my targeted audience to directly access my `R Shiny` app using tha...

dataslicer · 06-10-2022

+1 as I am local in the peninsula. Thanks for forming this group. 

dataslicer · 04-14-2022

I currently have multiple jobs (each running its own job cluster) for my spark structured streaming pipelines that are long running 24x7x365 on DBR 9.x/10.x LTS. My SLAs are 24x7x365 with 1 minute latency. I have already accomplished the following co...

dataslicer · 02-04-2022

I am using Azure DBX 9.1 LTS and successfully installed the following library on the cluster using Maven coordinates: com.crealytics:spark-excel_2.12:3.2.0_0.16.0When I executed the following line:excelSDF = spark.read.format("excel").option("dataAdd...

dataslicer · 09-27-2021

I am using Azure Databricks Runtime (DBR) 8.3 ML with Python notebook and R cells together.I want to use "tidyverse" and one of the dependency is rlang >= 0.4.10 and the base DBR 8.3 ML provides rlang @ 0.4.9. I successfully upgraded the R package t...

dataslicer · 05-11-2022

You might want to start a new question thread so there are enough area / space to capture the context and issues you are experiencing. That way you get better visibility and support from the community. For reference, this Spark documentation shoul...

dataslicer · 04-15-2022

Using the older library as suggested worked in DBR 10.4 LTS. Thank you. On a separate note, my curiosity in understanding the changes in the underlying datasource v2 API is ongoing. 

dataslicer · 04-15-2022

Thank you so much for characterizing the Photon improvements in different scenarios! I will definitely explore this new path.

dataslicer · 04-14-2022

Thank you for the extra perspectives!Yes, there is already a volume discounts (VM resources purchased ahead of time) negotiated by the company and the cloud provider. Sorry I left that out as I was too focused on the technical options. I am on the sa...

dataslicer · 04-14-2022

Thank you for providing another option to address this issue. I have follow up questions:What should be the dependency to be excluded in this situation?How to exclude such dependency in Databricks runtime environment?Is there a reference you can pro...

Databricks

User Stats

User Activity

Using Azure Databricks to manage my R Shiny front-end, need to know the user via R session

+1 as I am local in the peninsula. Thanks for forming this group. &#xd83d;&#xde00;

Exploring additional cost saving options for structured streaming 24x7x365 uptime workloads

Successfully installed Maven:Coordinates:com.crealytics:spark-excel_2.12:3.2.0_0.16.0 on Azure DBX 9.1 LTS runtime but getting error for missing dependency: org.apache.commons.io.IOUtils.byteArray(I)

upgraded R package rlang to 0.4.11 on DBR 8.3 SC, but sessionInfo() still shows rlang as 0.4.9

Re: Exploring additional cost saving options for structured streaming 24x7x365 uptime workloads

Re: Successfully installed Maven:Coordinates:com.crealytics:spark-excel_2.12:3.2.0_0.16.0 on Azure DBX 9.1 LTS runtime but getting error for missing dependency: org.apache.commons.io.IOUtils.byteArray(I)

Re: Exploring additional cost saving options for structured streaming 24x7x365 uptime workloads

Re: Exploring additional cost saving options for structured streaming 24x7x365 uptime workloads

Re: Successfully installed Maven:Coordinates:com.crealytics:spark-excel_2.12:3.2.0_0.16.0 on Azure DBX 9.1 LTS runtime but getting error for missing dependency: org.apache.commons.io.IOUtils.byteArray(I)