Thanks for all the feedback here! We are actually working on a better file explorer within the platform and one of the items on the roadmap is the ability to sort by "last opened".
@Fran Pérez I did a little research on this and found that currently PYTHONPATH will be overwritten on cluster startup time and there is no way to redefine it at this time. At this point we would recommend using the already defined PYTHONPATH direct...
What documentation are you following here? You shouldn't need to specify PYTHONPATH or PYSPARK_PYTHON as this section is for Spark specific environment variables such as "SPARK_WORKER_MEMORY".
This is because in the example "sales_orders" data is being streamed, joined (using left join) to customers, and being appended to the silver layer table. When a sales_order comes in from a customer that was inserted some time ago (rather than in the...
@Brad Sheridan To do that you have to add the aws_attributes tag within a cluster configuration and there you have the ability to add an instance_profile_arn like so:"clusters": [ { "label": "default", "aws_attributes": { ...
