I'm trying to use Databricks ARC (Automated Record Connector) and running into an object issue. I assume I'm missing something rather trivial that's not related to ARC.

#Databricks Python notebook

#CMD1 import AutoLinker
from arc.autolinker import AutoLinker
import arc

arc.enable_arc()

#CMD2 create dataframe from table data
import os
data_1 = spark.read.table("temp.list_all")

#CMD3 run autolinker
autolinker = AutoLinker()

attribute_columns = ["first_name", "last_name", "dob", "address_line_1", "zip_code"]
#runs fine up to this point
autolinker.auto_link(
  data=data_1, 
  attribute_columns=attribute_columns,
  unique_id="pid",  
  comparison_size_limit=100000,
  max_evals=100
)

Then I receive this error when running autolinker.auto_link() and not sure how to troubleshoot.

AttributeError: 'DataFrame' object has no attribute 'sparkSession'

My cluster Runtime Version is 10.4 LTS (includes Apache Spark 3.2.1, Scala 2.12). Within my cluster, I do not have any spark configurations set. I'm not sure if this needs to be changed, and if so, which properties to set. Currently researching....