cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

yalei
by New Contributor
  • 4910 Views
  • 1 replies
  • 0 kudos

leaflet not works in notebook(R language)

I saw this notebook: htmlwidgets-azure - Databricks (microsoft.com)However, it is not reproducible. I got a lot errors:there is no package called ‘R.utils’. This is easy to fix, just install the package "R.utils""can not be unloaded". This is not ...

  • 4910 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @yalei du​, Thank you for contacting us regarding the issues you're experiencing with the Leaflet and Dygraphs packages in your R notebook. I understand you have successfully installed the required packages, but the visualizations are not in the n...

  • 0 kudos
JJ_
by New Contributor II
  • 1200 Views
  • 3 replies
  • 0 kudos

ODBC Connection to Another Compute Within the Same Workspace

Hello all!I couldn't find anything definitive related to this issue so I hope I'm not duplicating another topic :). I have imported an R repository that normally runs on another machine and uses ODBC driver to issue sparkSQL commands to a compute (le...

  • 1200 Views
  • 3 replies
  • 0 kudos
Latest Reply
JJ_
New Contributor II
  • 0 kudos

Thanks @Suteja Kanuri​ for your response! I tried all of the steps you mentioned (and many more) but never managed to make it work. My suspicion was that our azure networking setup was preventing this from happening. I have not found this documented ...

  • 0 kudos
2 More Replies
Osky_Rosky
by New Contributor II
  • 3554 Views
  • 2 replies
  • 0 kudos

Combine Python + R in data manipulation in Databricks Notebook

Want to combine Py + Rfrom pyspark.sql import SparkSessionspark = SparkSession.builder.appName("CreateDataFrame").getOrCreate()# Create a sample DataFramedata = [("Alice", 25), ("Bob", 30), ("Charlie", 35), ("Oscar",36), ("Hiromi",41), ("Alejandro", ...

  • 3554 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Oscar CENTENO MORA​ :To combine Py and R in a Databricks notebook, you can use the magics command %python and %rto switch between Python and R cells. Here's an example of how to create a Spark DataFrame in Python and then use it in R:from pyspark.sq...

  • 0 kudos
1 More Replies
nirajtanwar
by New Contributor
  • 1058 Views
  • 2 replies
  • 2 kudos

To collect the elements of a SparkDataFrame and coerces them into an R dataframe.

Hello Everyone,I am facing the challenge while collecting a spark dataframe into an R dataframe, this I need to do as I am using TraMineR algorithm whih is implemented in R only and the data pre-processing I have done in pysparkI am trying this:event...

  • 1058 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Niraj Tanwar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 2 kudos
1 More Replies
SG_
by New Contributor II
  • 2254 Views
  • 1 replies
  • 2 kudos

How to display Sparklyr table in a clean readable format similar to the output of display()?

There exist a Databricks’s built-in display() function (see documentation here) which allow users to display R or SparkR dataframe in a clean and human readable manner where user can scroll to see all the columns and perform sorting on the columns. S...

ewFTT.jpg h3hff
  • 2254 Views
  • 1 replies
  • 2 kudos
Latest Reply
rich_goldberg
New Contributor II
  • 2 kudos

I found that the display() function returned this issue when it came across date-type fields that were NULL. The following function seemed to fix the problem:library(tidyverse) library(lubridate)   display_fixed = function(df) { df %>% ...

  • 2 kudos
Brave
by New Contributor II
  • 2369 Views
  • 6 replies
  • 4 kudos

Resolved! Exporting R data frame variable

Hi all.I am trying to export R data frame variable as csv file.I am using this formula:df<- data.frame(VALIDADOR_FIM)df.coalesce(1).write.format("com.databricks.spark.csv").option("header", "true").save("dbfs:/FileStore/df/df.csv")But isn´t working. ...

  • 2369 Views
  • 6 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @FELIPE VALENTE​  (Customer)​, We haven’t heard from you since the last response from @sherbin w​  (Customer)​ , and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as...

  • 4 kudos
5 More Replies
Dave_Nithio
by Contributor
  • 516 Views
  • 0 replies
  • 1 kudos

Natively Query Delta Lake with R

I have a large delta table that I need to analyze in native R. The only option I have currently is to query the delta table then use collect() to bring that spark dataframe into an R dataframe. Is there an alternative method that would allow me to qu...

  • 516 Views
  • 0 replies
  • 1 kudos
Ryan_Chynoweth
by Honored Contributor III
  • 5075 Views
  • 3 replies
  • 7 kudos

Resolved! Best language to use

Databricks supports SQL, Scala, Python, and R. Is there a most performant language to use on Databricks? I know SQL well but would like to get into one of the other languages and don't know which to focus on.

  • 5075 Views
  • 3 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

It total depends on you? BTW, you can choose Python and SQL

  • 7 kudos
2 More Replies
fhte
by New Contributor
  • 926 Views
  • 2 replies
  • 0 kudos

How to install R GeoLift library on Databrickts

Hi, I am having problems installing the GeoLift library. I am proceeding as per the official instructions: https://facebookincubator.github.io/GeoLift/docs/GettingStarted/InstallingRThis is what I run in the notebook:1) I install this particular vers...

Screenshot 2022-09-14 at 08.59.09
  • 926 Views
  • 2 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Moderator
  • 0 kudos

Hi @Ludmila Kuncarova​,I would like to share the following link to our docs https://docs.databricks.com/libraries/notebooks-r-libraries.html in this link you will be able to find more details on how to install R libraries.

  • 0 kudos
1 More Replies
isaac_gritz
by Valued Contributor II
  • 1985 Views
  • 6 replies
  • 8 kudos

Library Dependency

How to Install Libraries on DatabricksYou can install libraries in Databricks at the cluster level for libraries commonly used on a cluster, at the notebook-level using %pip, or using global init scripts when you have libraries that should be install...

  • 1985 Views
  • 6 replies
  • 8 kudos
Latest Reply
Chris_Shehu
Valued Contributor III
  • 8 kudos

It can be a risky to install libraries without any sort of oversite/security structure to ensure those libraries have no vulnerabilities. I think more caution needs to be added to the wording of these documents to express that. All of the libraries w...

  • 8 kudos
5 More Replies
isaac_gritz
by Valued Contributor II
  • 1064 Views
  • 2 replies
  • 7 kudos

Versions of Spark, Python, Scala, R in each Databricks Runtime

What version of Spark, Python, Scala, R are included in each Databricks Runtime? What libraries are pre-installed?You can find this info at the Databricks runtime releases page (AWS | Azure | GCP).Let us know if you have any additional questions on t...

  • 1064 Views
  • 2 replies
  • 7 kudos
Latest Reply
maxdata
New Contributor II
  • 7 kudos

Wow! Thanks for the help @Isaac Gritz​ !

  • 7 kudos
1 More Replies
kirankv
by New Contributor
  • 480 Views
  • 0 replies
  • 0 kudos

How to get notebookid programmatically using R

Hi, I would like to log the notebook id programmatically in R, Is there any command that exists in R so that I can leverage to grab the notebook id, I tried with python using the below command and grab it without any issues, and looking for similar f...

  • 480 Views
  • 0 replies
  • 0 kudos
Host
by New Contributor
  • 1194 Views
  • 1 replies
  • 0 kudos

hostinc-logo

Hostinc is the best place to match the price and quality of the product at the most affordable price. If you are looking for a server that can make your marketing campaign a huge success here you go with our one of the most powerful Dedicated Server ...

  • 1194 Views
  • 1 replies
  • 0 kudos
Latest Reply
Sovchenko
New Contributor II
  • 0 kudos

Thanks for sharing! Before you hire mobile app developers, you need to carefully study this topic.

  • 0 kudos
flachboard
by New Contributor
  • 2971 Views
  • 4 replies
  • 1 kudos

How do you install R packages?

I've tried this, but it doesn't appear to be working: https://community.databricks.com/s/question/0D53f00001GHVX1CAP/unable-to-install-sf-and-rgeos-r-packages-on-the-clusterWhen I run the following after that init script, I receive an error.library(r...

  • 2971 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hey there @Christopher Flach​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 1 kudos
3 More Replies
dataslicer
by Contributor
  • 1988 Views
  • 3 replies
  • 2 kudos

Resolved! upgraded R package rlang to 0.4.11 on DBR 8.3 SC, but sessionInfo() still shows rlang as 0.4.9

I am using Azure Databricks Runtime (DBR) 8.3 ML with Python notebook and R cells together.I want to use "tidyverse" and one of the dependency is rlang >= 0.4.10 and the base DBR 8.3 ML provides rlang @ 0.4.9. I successfully upgraded the R package t...

  • 1988 Views
  • 3 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @Jim Huang​ ​ , Just a friendly follow-up. Do you still need help or the above responses help you to find the solution? Please let us know.

  • 2 kudos
2 More Replies
Labels