Having trouble with ARC (Automated Record Connector) Python Notebook

Isolated — Thu, 12 Oct 2023 21:38:40 GMT

I'm trying to use Databricks ARC (Automated Record Connector) and running into an object issue. I assume I'm missing something rather trivial that's not related to ARC.

#Databricks Python notebook #CMD1 import AutoLinker from arc.autolinker import AutoLinker import arc arc.enable_arc() #CMD2 create dataframe from table data import os data_1 = spark.read.table("temp.list_all") #CMD3 run autolinker autolinker = AutoLinker() attribute_columns = ["first_name", "last_name", "dob", "address_line_1", "zip_code"] #runs fine up to this point autolinker.auto_link( data=data_1, attribute_columns=attribute_columns, unique_id="pid", comparison_size_limit=100000, max_evals=100 )

Then I receive this error when running autolinker.auto_link() and not sure how to troubleshoot.

AttributeError: 'DataFrame' object has no attribute 'sparkSession'

My cluster Runtime Version is 10.4 LTS (includes Apache Spark 3.2.1, Scala 2.12). Within my cluster, I do not have any spark configurations set. I'm not sure if this needs to be changed, and if so, which properties to set. Currently researching....

Re: Having trouble with ARC (Automated Record Connector) Python Notebook

-werners- — Fri, 13 Oct 2023 09:24:48 GMT

ARC requires DBR 12.2 LTS ML

https://github.com/databricks-industry-solutions/auto-data-linkage

topic Re: Having trouble with ARC (Automated Record Connector) Python Notebook in Get Started Discussions

Having trouble with ARC (Automated Record Connector) Python Notebook

Re: Having trouble with ARC (Automated Record Connector) Python Notebook

Re: Having trouble with ARC (Automated Record Connector) Python Notebook