Hi Team,
In a streamlit app (in databricks) while creating the spark session getting below error, this is happening when running the app via web link.
"[JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number"
Below is the code used in app.py:
import os
import streamlit as st
import pandas as pd
from databricks import sql
import pytz
from datetime import datetime
from pyspark.sql import SparkSession
from pyspark import SparkContext
from pyspark import SparkConf
# Create SparkSession
try:
spark = SparkSession.builder \
.appName("Streamlit App") \
.config("spark.driver.memory", "4g") \
.config("spark.executor.memory", "4g") \
.getOrCreate()
st.write("Spark Session created successfully!")
st.write("Spark Version:", spark.version)
except Exception as e:
st.write(f"Error creating Spark session: {e}")
#Create a sample DataFrame
data = {
'Name': ['Alice', 'Bob', 'Charlie', 'David'],
'Age': [25, 30, 35, 40],
'City': ['New York', 'Los Angeles', 'Chicago', 'Houston']
}
df = pd.DataFrame(data)
# Display the DataFrame in the Streamlit app
st.title('Sample DataFrame')
st.write(df)
# Convert the edited DataFrame to a Spark DataFrame
spark_df = spark.createDataFrame(df)
st.write(spark_df)
