cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

hv
by New Contributor
  • 2405 Views
  • 1 replies
  • 0 kudos

Error-"'Column' object is not callable".

I am trying to lowercase one of the columns(A_description) of a dataframe(df) and getting the error-"'Column' object is not callable".Code: def new_desc():  for line in df:    line = df['A_description'].map(str.lower)  return line new_desc()Have used...

  • 2405 Views
  • 1 replies
  • 0 kudos
Latest Reply
Chaitanya_Raju
Honored Contributor
  • 0 kudos

Hi @Himadri Verma​ Hope this below suggestion will help you in pyspark.Please let me know if you are looking for something elseHappy Learning!!

  • 0 kudos
theSoyf
by New Contributor II
  • 2936 Views
  • 2 replies
  • 1 kudos

How to write to Salesforce object using Spark Salesforce Library

Hi I'm facing an issue when writing to a salesforce object. I'm using the springml/spark-salesforce library. I have the above libraries installed as recommended based on my research.I try to write like this:(_sqldf .write .format("com.springml.spar...

Screen Shot 2022-12-14 at 8.18.07 AM
  • 2936 Views
  • 2 replies
  • 1 kudos
Latest Reply
Gauthy
New Contributor II
  • 1 kudos

Im facing the same issue while trying to write to Salesforce, if you have found a resolution could you please share it ?

  • 1 kudos
1 More Replies
Rishabh264
by Honored Contributor II
  • 687 Views
  • 1 replies
  • 4 kudos

PrivilegesSELECT: gives read access to an object.CREATE: gives ability to create an object (for example, a table in a schema).MODIFY: gives ability to...

PrivilegesSELECT: gives read access to an object.CREATE: gives ability to create an object (for example, a table in a schema).MODIFY: gives ability to add, delete, and modify data to or from an object.USAGE: does not give any abilities, but is an add...

  • 687 Views
  • 1 replies
  • 4 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 4 kudos

thanks sir

  • 4 kudos
gpzz
by New Contributor II
  • 1210 Views
  • 1 replies
  • 3 kudos

pyspark code error

rdd4 = rdd3.reducByKey(lambda x,y: x+y)AttributeError: 'PipelinedRDD' object has no attribute 'reducByKey'Pls help me out with this

  • 1210 Views
  • 1 replies
  • 3 kudos
Latest Reply
UmaMahesh1
Honored Contributor III
  • 3 kudos

Is it a typo or are you really using reducByKey instead of reduceByKey ?

  • 3 kudos
Slalom_Tobias
by New Contributor III
  • 6022 Views
  • 1 replies
  • 1 kudos

AttributeError: 'SparkSession' object has no attribute '_wrapped' when attempting CoNLL.readDataset()

I'm getting the error...AttributeError: 'SparkSession' object has no attribute '_wrapped'---------------------------------------------------------------------------AttributeError Traceback (most recent call last)<command-2311820097584616> in <cell li...

  • 6022 Views
  • 1 replies
  • 1 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 1 kudos

this can happen in 10X version try to use 7.3 LTS and share your observationand if it not working there try to create init script and load it to your databricks cluster so whenever your machine go up you can get advantage of that library because some...

  • 1 kudos
Ancil
by Contributor II
  • 4383 Views
  • 1 replies
  • 3 kudos

Resolved! TypeError: 'JavaPackage' object is not callable - DocumentAssembler() - Spark NLP

Hi,I am doing a grammar check using spark NLP using azure databricks. But am getting TypeError: 'JavaPackage' object is not callable - in DocumentAssembler() intialization line.document_assembler = DocumentAssembler()\ .setInputCol("text")\ .setOutpu...

image.png
  • 4383 Views
  • 1 replies
  • 3 kudos
Latest Reply
Ancil
Contributor II
  • 3 kudos

Hi its working after adding below configs in cluster. Please check URL for more info :- https://nlp.johnsnowlabs.com/docs/en/install#databricks-support

  • 3 kudos
db-avengers2rul
by Contributor II
  • 5281 Views
  • 2 replies
  • 6 kudos

Resolved! AttributeError: 'list' object has no attribute 'columns' - PySpark

Hi All,i am getting the below error when i am ingesting the data from source file , source file is also attached , i have tried in both Community edition and Azure databricks as well getting the same error , can any one suggest me the solution ? # ...

  • 5281 Views
  • 2 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Rakesh Reddy Gopidi​ ​, We haven’t heard from you since the last response from me, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise...

  • 6 kudos
1 More Replies
Raymond_Garcia
by Contributor II
  • 1895 Views
  • 1 replies
  • 1 kudos

Resolved! query array in SQL

Hello I have a databricks question I was not able to answer myselfI have this queryselect count(*) from tablewhere object[0].value is not null and object[0].value.value1 = "s"and created_year = 2022 and created_month = 7 and created_day = 4you can se...

  • 1895 Views
  • 1 replies
  • 1 kudos
Latest Reply
Raymond_Garcia
Contributor II
  • 1 kudos

SELECT count(*)FROM ( SELECT explode(mmycolumn) FROM table WHERE created_year = 2022 and created_month = 7 and created_day = 5)WHERE col.field is not null and col.field.field! = "signal"

  • 1 kudos
chandan_a_v
by Valued Contributor
  • 13966 Views
  • 9 replies
  • 3 kudos
  • 13966 Views
  • 9 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hi @Chandan Angadi​ , We haven’t heard from you on the last response from @Prabakar Ammeappin​ , and I was checking back to see if you have a resolution yet. If you have any solution, please do share that with the community as it can be helpful to ot...

  • 3 kudos
8 More Replies
Jeff1
by Contributor II
  • 963 Views
  • 3 replies
  • 1 kudos

Resolved! Strange object returned using sparklyr

CommunityI'm running a sparklyr "group_by" function and the function returns the following info:# group by event_typeacled_grp_tbl <- acled_tbl %>% group_by("event_type") %>% summary(count = n())                   Length Cl...

  • 963 Views
  • 3 replies
  • 1 kudos
Latest Reply
Jeff1
Contributor II
  • 1 kudos

I should have deleted the post. While your are correct "event_type" should be without quotes the problem was the Summary function. I was using the wrong function it should have been "summarize."

  • 1 kudos
2 More Replies
marsjuli
by New Contributor II
  • 9082 Views
  • 5 replies
  • 3 kudos

Resolved! How to handle <IPython.core.display.HTML object>

Some libraries have intermediate IPython HTML-objects returned to the notebook cell output.Since this happens during training a machine learning model the statements are typically buried within in the library so I cannot easily interfere. (e.g. in or...

grafik.png
  • 9082 Views
  • 5 replies
  • 3 kudos
Latest Reply
marsjuli
New Contributor II
  • 3 kudos

Hi @Kaniz Fatma​ ,thanks for showing me the link. This helps if you are in control of the generated html-object. If the html-content comes from a library, that is where the problems start, because I cannot wrap displayHTML().(I can of course look for...

  • 3 kudos
4 More Replies
amitdatabricksc
by New Contributor II
  • 5187 Views
  • 2 replies
  • 0 kudos

AttributeError: 'NoneType' object has no attribute 'repartition'

I am using a framework and i have a query where i am doing,df = seg_df.select(*).write.option("compression", "gzip') and i am getting below error,When i don't do the write.option i am not getting below error. Why is it giving me repartition error. Wh...

  • 5187 Views
  • 2 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Moderator
  • 0 kudos

hi @AMIT GADHAVI​ ,could you provide more details? for example, what is your data source? how do you repartition? etc

  • 0 kudos
1 More Replies
Labels